jyrgenn: Blurred head shot from 2007 (Default)
More or less since I started programming in Go, I have wanted to know which method I should use for collecting strings: (a) use a bytes.Buffer and WriteString(), or (b) just "add" together a string with +=. The convenience of the latter is appealing, but how would that be performance-wise? So I finally checked it.
package main

import (

var startT time.Time

func main() {
	for reps := 10000; reps <= 100000; reps += 10000 {
		fmt.Printf("\nreps: %d\n", reps)
		snippet := os.Args[1]

		startT = time.Now()
		buf1 := bytes.NewBufferString("")
		for i := 0; i < reps; i++ {
		seconds1 :=
			float64(time.Now().Sub(startT)) / float64(time.Second)
		len1 := buf1.Len()
		fmt.Printf("last of %d: %s; %g s\n", len1,
			buf1.Bytes()[len1-10:len1-1], seconds1)

		startT = time.Now()
		buf2 := ""
		for i := 0; i < reps; i++ {
			buf2 += snippet
		seconds2 :=
			float64(time.Now().Sub(startT)) / float64(time.Second)
		len2 := len(buf2)
		fmt.Printf("last of %d: %s; %g s\n", len2,
			buf2[len2-10:len2-1], seconds2)

The result was more clear-cut than I had expected:
$ ./strcatz fldsjbcsldfbcdkhasfacde

reps: 10000
last of 230000: dkhasfacd; 0.001061327 s
last of 230000: dkhasfacd; 0.501937235 s

reps: 20000
last of 460000: dkhasfacd; 0.001232219 s
last of 460000: dkhasfacd; 2.42185104 s

reps: 30000
last of 690000: dkhasfacd; 0.00211587 s
last of 690000: dkhasfacd; 6.120059 s

reps: 40000
last of 920000: dkhasfacd; 0.002452257 s
last of 920000: dkhasfacd; 13.718863728 s

reps: 50000
last of 1150000: dkhasfacd; 0.0048127 s
last of 1150000: dkhasfacd; 18.529621865 s

reps: 60000
last of 1380000: dkhasfacd; 0.004334798 s
last of 1380000: dkhasfacd; 24.74539053 s

reps: 70000
last of 1610000: dkhasfacd; 0.005095205 s
last of 1610000: dkhasfacd; 32.982584273 s

reps: 80000
last of 1840000: dkhasfacd; 0.009039379 s
last of 1840000: dkhasfacd; 44.176404262 s

reps: 90000
last of 2070000: dkhasfacd; 0.008148565 s
last of 2070000: dkhasfacd; 53.003958242 s

reps: 100000
last of 2300000: dkhasfacd; 0.008536743 s
last of 2300000: dkhasfacd; 67.390456565 s

The += method is not only slower to begin with, but also goes up more than linear, which is not quite surprising. I do find it surprising, though, that the difference is so large, in the order of 10000. So, I guess, that question is answered: In a place where performance matters at all, don't use += for repeated string concatenation.
jyrgenn: Blurred head shot from 2007 (Default)
Just for fun I timed a program that I have developed in my spare time, the Lisp interpreter lingo, written in Go, on a number of computers. This measures basically single-thread performance, presumably with some emphasis on memory access, as the interpreter does a lot of pointer chasing. Mainly I wanted to compare my newly upgraded home server mellum with others.

The first four of the computers listed in the table are my own, the first three at home, the fourth an external rented server. All others are my employer's and are operated by our group.


mellum35057151.00E3-1220 v343.1FreeBSD 10.3
naibel4971927.05T40E APU21FreeBSD 10.3
wrixum16591912.11Core 2 Duo22.4OS X 10.11.4
holt18490071.90Opteron 138542.7Debian Jessie
Brunei13756742.55E5-2620 v3122.4Debian Jessie
Island15480872.26X5650122.67Debian Jessie
Bermuda21399851.64i5-240043.1Debian Jessie
qcm0516227772.16E5-2690 v2203Debian Jessie
qcm0613554492.59E5-2690 v3242.6Debian Jessie
qcm0713915232.52E5-2690 v3242.6Debian Jessie
qcw5041664560.84i5-459043.3Debian Jessie
dgm0714736662.38X5650122.7Debian Wheezy


The listed number of cores is the total in the machine, without hyperthreading.

The program I ran is the interpreter lingo, commit 5aa9fa8cd136efd05e0adcbb9474f0aa6fe1fe64, built with the current Go 1.6.2 – to be precise, a run of make benchmark10 in the lingo directory, which factorises the number 100000000001 with the (rather naïvely implemented) Lisp program factor.lisp.

The number at "evals/s" states how many Lisp expressions have been evaluated per second. I have used the best number of a few runs each (at least two). Apart from qcm05 and qcm07 the machines were very lightly loaded, such that each had a "free" CPU.

I am a bit surprised that, apart from the workstation qcw50, my computer with a relatively cheap and nearly three-year-old CPU comes out ahead of nearly everything I could get my hands on, and not only the old ones (Island, our workgroup server, and Bermuda, my workstation), but also the newer ones. Now that computer has only one CPU and only four cores in total; especially the qcm0[5-7], meant for serious number crunching, have much more. Still amazing.

But I am even more surprised that my oldish MacBook Pro wrixum (13", mid-2010) keeps itself up so bravely. It has not only a CPU design from nearly eight years ago, but was also the slowest of the product line when I bought it.

Update: an additional result from rbarclay (see comments)

Update: More results are welcome! If you want to build from source, look into the comments for detailed instructions. If you want to use a pre-built binary for FreeBSD, Linux, or OS X on the amd64 architecture, download the appropriate one of the following files, unpack it, change into the lingo directory, and run <code>make benchmark10</code>. See the output for the "evals/s" value.

additional results
rbarclay28504421.23FX-835084Debian Jessie
Update: An article Modern Microprocessors – A 90 Minute Guide! by Jason Robert Carey Patterson is interesting in this context.

jyrgenn: Blurred head shot from 2007 (Default)
Today I thought of that version cum configuration control system ("ShapeTools") with Makefile-like input files I had the pleasure to work on 20+ years ago.

One user, a co-student of mine, came whining (again!) about some alleged bug in the "shape" program that had allegedly deleted his (alleged) source files. No way, we said.

No, it didn't. It only tended to skip the very last character of the Shapefile. Which didn't do any harm (ever), because that (always) was a newline character, right?
        rm -f core *.o $(PROGRAM) #*# *~

(For totally unrelated reasons, I implemented a new Shapefile parser not long after that bug report. And I still never end a "clean" rule in a Makefile with "*~".)
jyrgenn: Blurred head shot from 2007 (Default)
In the last days I have put a new server for infrastructure services into operation at home.

It does not run as router or firewall, but has an SSHd for remote logins, DHCP and RADIUS server, DNS resolver, and cron jobs to do all those little things that must be done when my outer IP(v4) address changes, like updating dynamic DNS and reconfiguring the IPv6 tunnel with HE. For a few hours now, fail2ban has also been successfully blocking those pesky brute-force ssh attacks from China and the like.

The hardware is a small and — moderately — low power model from PC Engines, but still with a dual-core 1 GHz AMD CPU (amd64) and 4 GB of RAM, so it is quite capable. I have put in an SLC SSD (relatively expensive, but AIUI not as easily worn out by writing), also with 4 GB, which is enough for normal operation.


Despite being low power (≤ 12 W), that little thing runs quite hot. Internally, CPU and south bridge are thermally connected to the case via an aluminium heat spreader:


Still, the case gets so hot that I felt another cooling element is in order, as it is already quite warm on the upper boards of the store-room shelf (the left one in the picture):

from left to right: the new small server with heat sink attached, the router, the switch connecing router and DSL modem

With that, it runs up to 72 °C on the CPU when it is around 30 °C outside. As the CPU is rated for up to 90 °C, that seems to be okay.

The server is connected to my "core" network, to the WLAN segment, and to the DMZ, where incoming SSH connections are terminated.

As it runs security-critical services, I decided to give OpenBSD a try, for the first time. Not a bad idea — while not as much pre-packaged software is available as for, say, FreeBSD or Debian, most things I want are there, and then I should still be able to install most things from source. Or write them myself, dammit!

OpenBSD feels more like a "traditional" BSD than FreeBSD — the installation is rather like that of other systems 20 years ago; the whole setup feels simpler, more straightforward, with fewer automatic tentacles; updates are done by getting the source for the whole system and recompiling. Without being able to give really informed comments yet, I can say it feels good, solid, familiar, definitely likeable.

BTW, the 4 GB SSD proved to be too small for rebuilding the system, so I had to put /usr/src/ and /usr/obj/ on the file server, NFS-mounted over mere Fast Ethernet. I was afraid that this would slow down the system building by much, but building the userland was done after 5h20m, with 63% CPU utilization. Pleasant surprise!

Update: I have to admit that after some time I fell back to FreeBSD for this machine. While that decreases the OS diversity, it is much easier to update two FreeBSD boxen than one FreeBSD and one OpenBSD box. Also, the tunnel isn't to HE any more, but to my own external server, which is much closer, roundtrip-wise, and handled directly by the router (the middle device in the picture) using OpenVPN.
jyrgenn: Blurred head shot from 2007 (Default)
I am interested in programming languages in general. In particular, I was always intrigued by how another programming language can offer me new possibilities to express my programs and allow me to grow as a programmer.

I learned my first one in school. The computers there — a room full of PET 2001s, when they were new — had only BASIC, which frustrated me soon. At the university, I started out with a course in Pascal and did quite some programming on the side in it. Pascal filled many of the gaps that I had found annoying with BASIC too soon.

For fun I learned the basics of FORTRAN, but never had any real use for it. Modula-II came along, even for largish programming assignments, but didn't really catch my interest. I found Ada more interesting, but had little opportunity to use it outside of the process control course. I looked a bit into Forth, but again had no real application to get some practice.

It was when I got to C that I was finally hooked. That was, finally, "the real thing", in a way, and a language that served me well, not only in the technical sense — for most of my professional life, it was one of the main things that kept me well-fed.

There were other interesting languages I learned at the university, Tcl, for instance, not the greatest language, but a very easily embeddable interpreter. For a while, I put one into every major program I wrote. I learned a little of Prolog, but not enough, which I regret.

But I was fascinated by functional languages and got a bit more productive in that field — Lisp, Hope, ML mainly. Lisp was the only one that I built an implementation for myself — or, rather, more than one. First for an assignment, together with a co-student, in Modula-II. We did not like some of the requirements in that course, and not so much the implementation language, so afterwards, we did a similar one in C. Years later, I made a Lisp interpreter in Java, and still later another one in C. All these are not really complete — in particular the garbage collector of the latter is a bit too eager and collects away things it shouldn't —, but both do implement a small but "real" Lisp, one that can use recursion and higher-order functions and has the basic builtins available. In between I have written one in Go, which is the most complete of all of them, although still in the My Favourite Toy Language category rather than a useful programming environment.

As mentioned, C was instrumental to most of the professional jobs I had, and the one I currently have. But others came into view, mainly Perl. Perl has even become the default language for me when I want to try something or have to implement just a bit of functionality. This is not because I value Perl so highly for its technical merits, but it is available everywhere, everybody knows it, so many things are admittedly much easier to do in Perl than in C, and consequently Perl has become a kind of habit. I am not the biggest fan of Perl, though; I find it inelegant and clumsy in places, and seductively encouraging questionable programming habits in others. Still, often it gets things done with relatively little effort.

There are others that I found interesting on the way, but have not found enough time (and practical use) to really learn them — Lua, SNOBOL, and APL (or J, rather) come to mind. I will have to work with JavaScript soonish, but I am strongly meh about it.

Then I saw more and more of Go. An article about it by Rob Pike finally made me dig into it, something I had been wanting to do for a while. Now that seems to be a fine language, with great ideas built into it, while still catering to the habits of programmers who grew up with C and its descendants. Go has the potential to give me back some of the fun I head with C 20 years ago, by combining ease and pragmatism (like in Perl) with a, finally, elegant language (although not as elegant as Lisp or the more modern functional languages). I'll see; for now I haven't done more than a few sample programs and the abovementioned Lisp interpreter in Go.

Then there is Haskell. I became curious about Haskell already in the early 90s, when I had contact with other functional languages as a student (see above). Someone passed me an article about Haskell in, I think, the ACM SIGPLAN Notices. Haskell was still new then, but in between it has matured for a few decades and is still there, which I assume is a good sign. As I always liked functional programming, this may be something to go with.

Now the biggest obstacle in adopting a new programming language for myself is not the difficulty of learning it and getting up to speed for real tasks, but other people. While my workplace has, to my regrets, a culture of people mostly doing their development projects alone, it is still considered important that someone else will be able to fix things when the original author is on vacation, or to do further development after the original author has left. I agree with that, of course. But that makes it difficult to adopt a new programming language when the others are just not interested in doing the same. And alas, it appears they are not.

Besides shell scripts, we work with C and — mostly — Perl, but I would love to do things in Go or Haskell. And I would like to do that at work, to make my work easier and more interesting. But as there is no one to take over a project done in one of these languages, I cannot do that. (There is one who would be interested enough in Go, I guess, but he is a student and will leave us in a year or two.) That is quite frustrating. Perhaps I should try to initiate a kind of consensus which language we should adopt next — but I am afraid there is too little interest to leave the beaten tracks of C and Perl. After all, they have already adopted Perl as a new laguage not even twenty years ago, so why do something like that again so soon?
jyrgenn: Blurred head shot from 2007 (Default)
In the last few days I have been familiarizing myself with the Go programming language and found that in general a very pleasant experience. Up to now, a few areas were a bit unfamiliar, but doable, others outright delightful.

Yesterday, I wanted to do something that involved writing a smallish program. Instead of going for the usual Perl, I wanted to try it in Go. The functionality involved writing a timestamp to a file, so I looked for the strftime() equivalent and found this:

Seriously, Go, time.Format()?

While strftime() may be "a bad interface" in someone's eyes (not in mine -- I always found it perfectly adequate), I can see time.Format() only as a persiflage of how bad an interface can be if carried to the extreme.

Especially this gets me, about strftime(): "no one remembers all the letters, so the only way to use it is with documentation in hand." And with time.Format(), am I supposed to remember -- instead of the partly arbitrary format letters of strftime() -- the parts of the example timestamp, which are all totally arbitrary? I mean, what were they drinking?
jyrgenn: Blurred head shot from 2007 (Default)
(From the discussion on an article on Google Plus)

There are always different opinions about a programming language. IMO it is a matter not just of personal taste, but also of the personal balance of priorities you have. For instance, if you particularly want terseness, freedom of expression, and lots of pragmatic shortcuts, Perl is just the thing. If you need real type safety and a firm grip on the module interfaces for a large team, Java or Ada may be the right thing. (I happen to like both, BTW.)

For me, Go may have just the right balance of type safety, pragmatic shortcuts (partly implicit strong typing! automatic memory management!), good performance, and powerful language features (closures! channels! interfaces! goroutines! multiple values!).

I lean to the stricter side of programming, such that I see it as a weakness of, say, Python or Ruby that variables do not have to be declared explicitly. Doing it with a simple ":=" like in Go, on the other hand, is so elegant I could squeak!

In a similar vein I am not a friend of the implicit string<->number equivalence that has become so popular since Perl (I guess) opened that particular box. I am very happy that Go does not follow that practice, but can, apparently, provide well-controlled implicit conversions where they are useful.

C has served me well for nearly a quarter of a century, in more than just the technical sense, and I still like it. But many things are so tedious to do in C. Implementing complex data structures, handling memory management, and constantly aiming carefully for the space between the toes takes, after an initial rush in the first years, much of the fun from programming and is quite tiring in the end.

Go may be just the thing to avoid many of C's tediousnesses, while keeping most of C's expressive power plus quite some of its own, and give me back the fun I have been missing.


Update: In the meantime I have learned that Go does no implicit conversion between strings and integers, but instead uses a common interface for print formatting, which is even better.

Update of the update: Later I learned that there is not (and cannot be) one common interface for print formatting implemented by all types and objects, but the formatter code looks at the value via reflection to decide how it should be formatted. There is built-in knowledge for simple types, an interface that can be used for those struct types that implement it, and generic code to explore other struct types via reflection.
jyrgenn: Blurred head shot from 2007 (Default)
Lua has been in the back of my head for a while, cannot remember why. Now that I have looked at it a bit by reading this book (and the Lua 5.1 Reference Manual, which is refreshingly short), I seem to be quite fond if it. I had no opportunity to do much with it yet, but there are quite a few things to like.

Functions are first-class values. Functions can be defined as closures. Environments are (kind of) first-class values, too. The syntax is a bit like Modula (while ... do ... end and the like). Typing is dynamic. Variables are created implicitly, which admittedly simplifies a few things, but I don't like it very much. Nothing of these darned $'s and %'s and @'s of Perl, which is good. And not its irregularities of syntax, which is much better. Lua is probably too small for irregularities anyway.

Tables can have metatables that control accesses to existing and particularly to non-existing fields; this is put to some interesting uses. For instance, packages and OOisms are done through tables as well, openly, but with some clever syntactic sugar around it. I like that. Prototype-based inheritance, not real classes, although you can pretend they are there.

Well, yes. Now I should probably go and do the canonical exercise, write a Lisp interpreter in it. Give me the time, someone, please.


jyrgenn: Blurred head shot from 2007 (Default)

January 2018

2829 3031   


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags