jyrgenn

More or less since I started programming in Go, I have wanted to know which method I should use for collecting strings: (a) use a bytes.Buffer and WriteString(), or (b) just "add" together a string with +=. The convenience of the latter is appealing, but how would that be performance-wise? So I finally checked it.

package main

import (
	"bytes"
	"fmt"
	"os"
	"time"
)

var startT time.Time

func main() {
	for reps := 10000; reps <= 100000; reps += 10000 {
		fmt.Printf("\nreps: %d\n", reps)
		snippet := os.Args[1]

		startT = time.Now()
		buf1 := bytes.NewBufferString("")
		for i := 0; i < reps; i++ {
			buf1.WriteString(snippet)
		}
		seconds1 :=
			float64(time.Now().Sub(startT)) / float64(time.Second)
		len1 := buf1.Len()
		fmt.Printf("last of %d: %s; %g s\n", len1,
			buf1.Bytes()[len1-10:len1-1], seconds1)

		startT = time.Now()
		buf2 := ""
		for i := 0; i < reps; i++ {
			buf2 += snippet
		}
		seconds2 :=
			float64(time.Now().Sub(startT)) / float64(time.Second)
		len2 := len(buf2)
		fmt.Printf("last of %d: %s; %g s\n", len2,
			buf2[len2-10:len2-1], seconds2)
	}
}

The result was more clear-cut than I had expected:

$ ./strcatz fldsjbcsldfbcdkhasfacde

reps: 10000
last of 230000: dkhasfacd; 0.001061327 s
last of 230000: dkhasfacd; 0.501937235 s

reps: 20000
last of 460000: dkhasfacd; 0.001232219 s
last of 460000: dkhasfacd; 2.42185104 s

reps: 30000
last of 690000: dkhasfacd; 0.00211587 s
last of 690000: dkhasfacd; 6.120059 s

reps: 40000
last of 920000: dkhasfacd; 0.002452257 s
last of 920000: dkhasfacd; 13.718863728 s

reps: 50000
last of 1150000: dkhasfacd; 0.0048127 s
last of 1150000: dkhasfacd; 18.529621865 s

reps: 60000
last of 1380000: dkhasfacd; 0.004334798 s
last of 1380000: dkhasfacd; 24.74539053 s

reps: 70000
last of 1610000: dkhasfacd; 0.005095205 s
last of 1610000: dkhasfacd; 32.982584273 s

reps: 80000
last of 1840000: dkhasfacd; 0.009039379 s
last of 1840000: dkhasfacd; 44.176404262 s

reps: 90000
last of 2070000: dkhasfacd; 0.008148565 s
last of 2070000: dkhasfacd; 53.003958242 s

reps: 100000
last of 2300000: dkhasfacd; 0.008536743 s
last of 2300000: dkhasfacd; 67.390456565 s

The += method is not only slower to begin with, but also goes up more than linear, which is not quite surprising. I do find it surprising, though, that the difference is so large, in the order of 10000. So, I guess, that question is answered: In a place where performance matters at all, don't use += for repeated string concatenation.

The IPv6 tunnel via HE worked fine, but I was not really pleased with the roundtrip times to their access point. So, when I got a new server hosted by my former employer last year, one that comes with a /56 IPv6 space, this gave me another tunneling opportunity.

In between I had trashed the Mikrotik router described in an earlier article after it had bricked itself when I tried to reset the configuration just like it was documented. Last straw and all that. The new one is an EdgeRouter PoE from Ubuiqiti, with which I am mostly happy. It has its weak points, too, but it is openly based on Debian and Vyatta, meaning you don't even have to break out of the configuration CLI to access any Unix commands. (The web GUI is nice and shiny, but very limited in its capabilities.) The CLI is modeled after JunOS, which made me feel at home fast.

Vyatta offers OpenVPN out of the box, so it was easy to set up a tunnel configuration to an OpenVPN instance on my server. This way I have a /60 tunneled to my home, which should be plenty. And other than with HE, the roundtrip times are in the single digits of milliseconds.

In the last days I have put a new server for infrastructure services into operation at home.

It does not run as router or firewall, but has an SSHd for remote logins, DHCP and RADIUS server, DNS resolver, and cron jobs to do all those little things that must be done when my outer IP(v4) address changes, like updating dynamic DNS and reconfiguring the IPv6 tunnel with HE. For a few hours now, fail2ban has also been successfully blocking those pesky brute-force ssh attacks from China and the like.

The hardware is a small and — moderately — low power model from PC Engines, but still with a dual-core 1 GHz AMD CPU (amd64) and 4 GB of RAM, so it is quite capable. I have put in an SLC SSD (relatively expensive, but AIUI not as easily worn out by writing), also with 4 GB, which is enough for normal operation.

http://www.pcengines.ch/apu1c4.htm

Despite being low power (≤ 12 W), that little thing runs quite hot. Internally, CPU and south bridge are thermally connected to the case via an aluminium heat spreader:

http://www.pcengines.ch/apucool.htm

Still, the case gets so hot that I felt another cooling element is in order, as it is already quite warm on the upper boards of the store-room shelf (the left one in the picture):



With that, it runs up to 72 °C on the CPU when it is around 30 °C outside. As the CPU is rated for up to 90 °C, that seems to be okay.

The server is connected to my "core" network, to the WLAN segment, and to the DMZ, where incoming SSH connections are terminated.

As it runs security-critical services, I decided to give OpenBSD a try, for the first time. Not a bad idea — while not as much pre-packaged software is available as for, say, FreeBSD or Debian, most things I want are there, and then I should still be able to install most things from source. Or write them myself, dammit!

OpenBSD feels more like a "traditional" BSD than FreeBSD — the installation is rather like that of other systems 20 years ago; the whole setup feels simpler, more straightforward, with fewer automatic tentacles; updates are done by getting the source for the whole system and recompiling. Without being able to give really informed comments yet, I can say it feels good, solid, familiar, definitely likeable.

BTW, the 4 GB SSD proved to be too small for rebuilding the system, so I had to put /usr/src/ and /usr/obj/ on the file server, NFS-mounted over mere Fast Ethernet. I was afraid that this would slow down the system building by much, but building the userland was done after 5h20m, with 63% CPU utilization. Pleasant surprise!

Update: I have to admit that after some time I fell back to FreeBSD for this machine. While that decreases the OS diversity, it is much easier to update two FreeBSD boxen than one FreeBSD and one OpenBSD box. Also, the tunnel isn't to HE any more, but to my own external server, which is much closer, roundtrip-wise, and handled directly by the router (the middle device in the picture) using OpenVPN.

In summer I gave up the additional "luxury" Internet access (with native IPv6 and fixed IPv4 address) to cut costs, leaving me without IPv6 at home. Now, a few weeks ago, one of our local IPv6 evangelists triggered me to try the free tunnel offering from Hurricane Electric (HE), and so I did. This weekend I put together the remaining pieces, so now I have everything in place again, including tunnel updates when my home IP address changes, and reverse DNS delegation.

Getting the tunnel to work was not that easy. On the HE tunnel broker website the information on how to update the tunnel information (i.e. the web API) is not exactly pushed into your face; googling helps. The first script I found for the Mikrotik router, though, seems to use an outdated version of the API, and then you don't want the router to do that anyway – while it has an HTTP client that you can use in scripts, it does not do https (WTF?!), so it sends your password in clear text. And don't get me started on the scripting language.

Anyway, for (IPv4) dynamic DNS updates I have a script on my home server watch the external IP address anyway, so this could as well trigger a script to update the tunnel when the address changed. This is so much easier in a shell script than with a router script...

In case anyone else needs something like it, this is the script:

#!/bin/ksh
# update HE ipv6 tunnel with Mikrotik router

USER=he_user                    # HE account username
PASS=hepassword                 # HE account password
HOST=12345678                   # HE tunnel ID
URL="https://ipv4.tunnelbroker.net/nic/update?username=$USER&password=$PASS&hostname=$HOST"
TNIF=sit1                       # Mikrotik router's tunnel interface name
ROUTER=mt_router                # router hostname
ADMIN=admin                     # router admin account
SSHKEY=$HOME/.ssh/id_dsa_$ROUTER # ssh identity key file
SSH="ssh -i $SSHKEY $ADMIN@$ROUTER"

curl -s -k "$URL" | while read mode addr; do
        case "$mode" in
                good) $SSH "/interface 6to4 set [find name=sit1] local-address=$addr"
                      logger "$0: new address $addr";;
                nochg) logger "$0: address unchanged $addr";;
                *) logger "$0: unknown response $mode $addr";;
        esac
done

Of course, be sure to understand what this does before you use it. Needs curl, and the approriate ssh key file in place. The ssh key must be good for admin access at the router.

(From the discussion on an article on Google Plus)

There are always different opinions about a programming language. IMO it is a matter not just of personal taste, but also of the personal balance of priorities you have. For instance, if you particularly want terseness, freedom of expression, and lots of pragmatic shortcuts, Perl is just the thing. If you need real type safety and a firm grip on the module interfaces for a large team, Java or Ada may be the right thing. (I happen to like both, BTW.)

For me, Go may have just the right balance of type safety, pragmatic shortcuts (partly implicit strong typing! automatic memory management!), good performance, and powerful language features (closures! channels! interfaces! goroutines! multiple values!).

I lean to the stricter side of programming, such that I see it as a weakness of, say, Python or Ruby that variables do not have to be declared explicitly. Doing it with a simple ":=" like in Go, on the other hand, is so elegant I could squeak!

In a similar vein I am not a friend of the implicit string<->number equivalence that has become so popular since Perl (I guess) opened that particular box. I am very happy that Go does not follow that practice, but can, apparently, provide well-controlled implicit conversions where they are useful.

C has served me well for nearly a quarter of a century, in more than just the technical sense, and I still like it. But many things are so tedious to do in C. Implementing complex data structures, handling memory management, and constantly aiming carefully for the space between the toes takes, after an initial rush in the first years, much of the fun from programming and is quite tiring in the end.

Go may be just the thing to avoid many of C's tediousnesses, while keeping most of C's expressive power plus quite some of its own, and give me back the fun I have been missing.

http://golang.org/

Update: In the meantime I have learned that Go does no implicit conversion between strings and integers, but instead uses a common interface for print formatting, which is even better.

Update of the update: Later I learned that there is not (and cannot be) one common interface for print formatting implemented by all types and objects, but the formatter code looks at the value via reflection to decide how it should be formatted. There is built-in knowledge for simple types, an interface that can be used for those struct types that implement it, and generic code to explore other struct types via reflection.

The need to switch ISPs finally pushed me to configure the Juniper SRX100 router.

As my current ISP, KGT New Media, is giving up their consumer Internet access over T-DSL product and has canceled the contract to end of August, I am a bit under pressure to get everything running with a different ISP. So, back to Titan Networks, although their offer is not quite what I was looking for. For € 24.50 per month, about the same price as with KGT, I get not a traffic flat rate, but a volume of 25GB, with extra traffic for € 5.50/GB. This should usually be enough, but in the past I have had a huge traffic peak once, which suddenly cost me additional 90 Euros. But there are not very many ISPs offering IPv6 for end customer prices to choose from.

Of course, before I switch completely, particularly all the DNS entries for- and backwards, I want to make sure everything works. This gave me another opportunity and additional motivation to finally tackle the SRX100, and I did.

While the Cisco 1712 still runs with KGT, the SRX100 is now running the Titan connection, although in a kind of "client-only" mode, without allowing incoming connections. Making incoming traffic possible requires much more firewall-fu than the little I have already understood. This is really not easy.

Doing the basic configuration -- forwarding IPv4 and IPv6 between the core and the WLAN network and the PPPoE connection to the ISP -- was moderately simple. Junos configuration is indeed a bit less of a pain in the back than IOS. I especially like the method of modifying a configuration until it is done and only then committing it to be activated. Otherwise it would have been more difficult or required a reboot to do reconfigurations that would have cut me off from the router in mid-change.

I also think the explicitly hierarchical configuration makes sense as a way of structuring everything; when I dive into some hierarchy level, I can concentrate on just that and show just that bit, for instance. Ah, yes, you can show the configuration while editing, isn't that just amazing? (I probably have only missed that with IOS, but to me it's still a difference.) And then there are the little things, like being able to go back in the pager (while viewing configuration or the like). I like it.

One thing had me busy for a while, though: There is no possibility to use IPv6 with vlan interfaces. This restriction still puzzles me, but apparently it is intentional, or at least specified. That I was not able to set an IPv6 address on a vlan interface from the CLI but could do that from the web interface added to my confusion. But even if an address has been set on a vlan interface, it cannot actually be used. Took me quite a while to find the final answer.

In the end I gave up and configured the interfaces not as switching group members, but as IP interfaces, and then everything worked. Well, except for the switching, of course -- I need a separate switch now where a port-based vlan on the SRX100 should have been sufficient. That is annoying.

Apart from that and the still unresolved incoming traffic issue, everything works fine now.

Perhaps I will finally just switch the Cisco over to Titan, and then the SRX100 to the currently unused T-Online connection -- I used it briefly for testing the SRX100 and found it that instead of the 30 ms roundtrip to my external server, it gave me 8! The T-Online access is with IPv4 only (currently; IPv6 probably next year) and with changing addresses. But that is fine for the clients, while the server can still use the fixed-address IPv6 and IPv4 access over the Cisco and Titan.

Another Lisp classic that I have wanted to read for decades. I have only just begun, so I have not much to say about it except that it is out of print and I had to buy it through a used-book merchant and not exactly cheap, and that I hope to find a few things in it that can help me to make my new Lisp interpreter better.

After two-and-a-half years, the 12" Powerbook G4, which I had bought already used, two years old, began to feel really old. It had probably been a mistake in the first place to buy a used computer from a line that was already obsolete when they built the last models. (On top of that, the CD/DVD drive was already mostly broken when I bought it, but I noticed that too late to give it back or claim compensation from the seller.)

It mostly felt old playing some kinds of videos. DivX and MPEG-4 in larger formats was too much, as well as some flash video stuff from the net. YouTube was fine, but some others, e. g. those from SPIEGEL Online, were not. And as I had not seen the (technical) point in upgrading from OS X 10.4 Tiger to 10.5, the selection of available software had already begun to shrink noticeably.


After the 2010 tax return it was time now for an up-to-date device again. Months ago I had already resolved to buy a MacBook or 13" MacBook Pro. (The bigger ones don't appeal to me, in particular not at their price.) The white plastic MacBook would have been enough with the RAM upgrade, but with the small Pro costing only 60 Euros more than the MacBook with 4 GB RAM (which the Pro already has), it was the Pro. Good choice. It came with OS X 10.6.3 "Snow Leopard" and runs 10.6.4 now after the first update.

As always, the migration to new hardware and OS version, even from the same manufacturer and in the same product line tradition, brings some, let's say, discoveries. "Same same, but different" or even "sometimes happy, sometimes sad."


Software Update has become much more intrusive. With Tiger, it checked for updates in the background and showed its dock icon only when there was something to do and it needed confirmation from the user. Now it shows the dock icon already when it only checks for updates. When it installs software for which a restart is required, it first asks for restart permission (which is okay), but then immediately shuts down everything and only then begins to install the software, which had previously been done in the background.


Only a short while ago, but still with Tiger, I discovered and learned to appreciate Terminal.app's "New Remote Connection" dialog as a fast and convenient way to open an ssh session to another machine. But now, with Snow Leopard, it wants to start ssh connections by default with SSH protocol version 1, which, for good reason, does not work with any of my servers; after each program restart I have to switch that to automatic or version 2. I have not found anything in the preferences (and I do mean Library/Preferences/com.apple.Terminal.plist) or the application bundle that looked like I could change this default. [Thankfully his has been fixed in OS X 10.6.7 -- ssh is now called without any options by default.]


Other than with my 10.4 installation, IPv6 is no longer consistently preferred with some services - telnet, ssh, http. Sometimes IPv4 is used, sometimes IPv6. I have not yet recognized a pattern. This may well be an application issue, but still it is strange.


X11 seems to work completely different from before.

Regardless if X11 is started or not, each Terminal window has a DISPLAY in its environment that contains the pathname a UNIX domain socket (e. g. /tmp/launch-ghLYjm/org.x:0); the socket exists, but is non-functional if X11 is not running. That confused my mechanism of dectecting the existence of an X server; xdpyinfo simply kept blocking on this socket. No fun. Ok, that could be fixed with an only slightly annoying timeout.

When I try to start X11 myself, it doesn't. Or, sometimes it does. But most of the time, some processes start, but nothing happens in terms of a usable X server.

I thought that my .xinitrc and (rather historic) .xserverrc might cause the problem, but moving them to the side has not really improved the situation. Instead, even without me having done anything (except perhaps checking the socket $DISPLAY for aliveness), it tries every few seconds to start up an X server, fails, tries again, ... you get the idea.

The non-functional DISPLAY variable in the environment causes outgoing ssh logins to fail if ForwardX11 is set to yes in ssh configuration, because the remote host tries to connect to the X server at first. Took me a while to find out that this was the reason why Unison failed to connect to another host.

I guess it is intended like this: Some X11 client connects to the socket $DISPLAY, a monitoring process notices this, starts an X server, passes the socket file descriptor to the X server, and lo! X11 applications can be started just like native ones. Clever, if it would only work.

There is something in the system.log, but I cannot make anything of it.


The new Quicktime Player looks awful. All black!

What is this fascination with black, anyway? When I put the dock on the side, where it belongs (IMO), it turns black! With a bit of transparency, yes, but black. That is ugly compared to the thin and airy dock of 10.4 (and predecessors).


A translucent menubar! WTF? At least I can switch this particular idiocy off again.

There seems to be a general trend towards needless design changes. The new dock (if it is on the bottom of the screen) so three-dimensional with a partial reflection of the icons - wow, that is so much eye candy that I want to take the toothbrush to my eyes. (But black?) Is this a "Yes, we can!" attitude, and "Just because we can"? That sucks.

The rounded upper corners of the windows are less rounded now. I can live with that. The amount of roundness taken from there has apparently been applied to the corners of the pull-down and pop-up menus.

The upper menubar corners are no longer rounded at all. Why did they give up one of the most visible design features of the Macintosh since 1984? Are we no longer nice-looking and a bit cute? This seems to be the most needless design change of all, given that the space previously occupied by the rounded corners has not been put to any other good use.


I like the hardware. Only two annoyances here: the glossy screen (I prefer to use the bathroom mirror when I need a shave) and the sharp edges of the case - because this is where my wrists are when I am typing with the laptop on my belly, lying in my bed. And this is at least 97% of the time when I use it. And the edges are really sharp.


The glossy screen does make for a more brilliant display, with a deeper black, yes. (Hum, does that correlate to that obsession with black I seemed to notice earlier?) Apart from the reflections of my face that I could live without, the display is indeed crisp. I like. (Only after having seen the screen of the new iPhone from a very close distance, I say it could use something more like that in terms of resolution.)

I like the wide format as a good compromise. It lets me put the dock on the side (the black dock) and still have it not steal too much from the screen width. There isn't too much height to spare anyway. The display is now better for watching films not in 3:4, small surprise.


I liked the keyboard of the Powerbook more, but this one is better than I expected. I was afraid that the more or less flat keys offer less guidance to the fingers that the more profiled ones of the Powerbook, but it is not as bad, no insecure feeling. The price paid for being able to shave off 2 millimeters (rough estimate) from the height of the keyboard is not too high. I really like the key illumination, although there is more light coming out from under the sides of the keys than through them.


I love the case. It is gorgeous. The rounded corners, the smooth undisturbed matte surfaces, the flat body - wow. It is often said that Apple sells their hardware more due to its design than its technical qualities. Sure, with this kind of design!


Although I have bought the slowest one, this little machine is screamingly fast compared with the G4. Moments where I had to wait a bit with the old Powerbook are now gone. Good.


The automounter seems to work with less hassle now. I once had a working setup with handcrafted mountpoints via Netinfo with 10.4, but that broke at some point, and I couldn't revive it this way or the other. With 10.6, the /net/$SERVER/ thingy works just like that without any setup required. Joy!


The battery lasts long.


In the end, I am quite happy with the new MacBook Pro. The X11 thing is a real annoyance, all others are minor. The new toy is fast and overall a joy to use.

Update 2011-08-12: After fiddling around over several days it turned out that there was (a) an incompatibility with the decades-old ~/.xserverrc and (b) checking for the existence of an X server at $DISPLAY in my ~/.profile kept it from working. Understandably so, considering that is done during the startup of that exact X server. Why X11 initialization starts a login shell -- perhaps to have the environment variables set up properly -- and has a non-empty $PS1 in there I shall probably never know. At least [ -t 0 ] is false, so I can exclude the check for that case.

[Today this is kind of obsolete since I use the Cisco 1712 for IPv6 now. I find this device not perfect, but it does everything I want. At least it could if I knew how to configure everything.]

As I have a separate ADSL modem, I am interested only in Ethernet-to-Ethernet routers, and I would prefer one with a point-and-click interface. I want it to be able to speak IPv6 over PPPoE on the WAN side, route to more than one /64 network over the LAN (i. e. have two separate interfaces on the inside, or be able to go through another router), and allow WAN access to services on the LAN specified by IPv6 address and port.

Comments are welcome, particularly with newer information or real-life experience, to <ni@jnickelsen.de>.

These are the devices I have some more information about, in no particular order:

• DrayTek Vigor 2130: Covers most things. Currently only one /64 on the inside, no DNS or NTP over v6, no RIPng. Otherwise an attractive device for the common SOHO setup. Cool web gui live demo on the web.


• Cisco: the (EOLed) 831 is the cheapest Cisco IOS router with IPv6 support; the 871 the cheapest with Fast Ethernet on the WAN side. The 831 is available on eBay for less than EUR 100, the 871 for over EUR 200. Very feature-rich, but IOS is in my eyes too arcane for someone how does not work with it daily. Update: Got a 1712 from eBay; together with shipping and a separately bought PSU about 90 Euros. Works fine and can do everything but the dishes, but see my comment above about IOS. I do not use most of the features because it is really hard work to dig out the correct configuration commands, and if you make a mistake, things may suddenly cease to work.


• Cisco SB WRVS4400N: From the former Linksys product line. According to the manual it can speak IPv6 on the WAN interface only through tunnels. WTF?


• OpenWRT: this linux-based open source system runs on a variety of hardware platforms. I have it running on a Linksys (now Cisco) WRT54G. It is (in parts) a hassle to set up and it is less well documented and less reliable than I like. I see it only as a temporary solution.


• AVM Lab Version: a public beta test firmware for their Fritz!Box 7270. Cannot make LAN services available to the WAN. [Update: in between it can.][Update: in between the IPv6 support is part of the official released product. Three cheers!]


• D-Link DIR-825: As I understand the user manual it can have only one flat /64 network on the inside.

Another possibility is to use a general-purpose computer as a router. Linux and *BSD come to mind as possible platforms. PC Engines and Soekris sell small PC-architecture boards designed for this type of use, and some software distributions are made for this purpose (OpenWRT being one of them). But in my experience a commercial router appliance means much less work to set up and operate.

Three days ago I wrote about mail from my system being rejected due to missing reverse DNS entries and complained about rh-tec's customer service interface for creating these entries. I was just a few keypresses ago from sending them another gripe about that thing again, when I had the idea of giving it another try.

This web interface lists the network you get from them (2001:1a50:5097::/48 in my case), and then you can enter "the desired RDNS entry" (my translation of the german text) in a single text field. WTF? I have still not understood how this is supposed to work, but this time I entered the name of the domain everything in there is under, and where my authoritative name servers are in and responsible for, w21.org.

Now (after some time in which they supposedly checked the entry manually) it worked. And not even how I expected it, that perhaps everything in the network is resolved to w21.org, but everything I have in my DNS servers as PTR records is suddenly there! Find me dumbstruck. Especially as I don't know how it actually works - when I look up the NS records for 7.9.0.5.0.5.a.1.1.0.0.2.ip6.arpa, I find all my name servers in there, but I am not able to follow the delegation chain from ip6.arpa upward. Is this really weird, or am I missing something?

Others have in between pointed out that the delegation chain can indeed be followed, but (a) one of their two authoritative name servers does not know about it and (b) both are not reachable via IPv6. Bugger. [jni 2010-03-31]

Anyway, no more gripe about rh-tec, and kudos to them for making this work so simple I still cannot figure out how. Might as well stay with them until the traffic exceeds the 3 GB volume included in the price of € 0. (Yes, that's a zero.)

And, to get back to the original issue, my mail should no longer be rejected now due to missing PTR records. Only I don't know if anybody has tried to send one to IPv6-listed MXs yet.