Function Types in Go (golang)
jordan orelli
<p>Most developers that are familiar with dynamic scripting language like Ruby, JavaScript, or Python have been exposed to <a href="http://en.wikipedia.org/wiki/Higher-order_function">higher-order functions</a>. Coming from a scripting background, it can be hard to translate that knowledge into Go, since the type system seems to get in the way. On the other hand, coming from a statically typed, primarily object-oriented language like C++, C#, or Java, the problem may be the opposite: static type systems are less of a stumbling block, but the usage of higher-order functions may be less intuitive. For programmers with functional experience, most of this knowledge may come across as very pedestrian, but hopefully this article will at least demonstrate how to use Go’s type system with respect to functions.</p>
<p>In this article, we’ll look at a few situations where function types in Go can be very useful. The reader is not assumed to be an experienced Go programmer, although a cursory knowledge of Go will certainly be helpful in digesting the material.</p>
<h3>Anonymous Function Closures</h3>
<p>Off the bat, Go supports <a href="http://en.wikipedia.org/wiki/Anonymous_function">anonymous functions</a> and <a href="http://en.wikipedia.org/wiki/Closure_(computer_science)">closures</a>. To me, Go’s support for anonymous functions is most reminiscent of the anonymous function support in JavaScript; it is not the flimsy kind of anonymous function support you get with Python’s lambdas (one statement only) or Ruby’s situation where there are multiple ways to create closures (I’m not a rubyist, so the whole proc/block/lambda/Method thing seems very convoluted to me). Anyway, declaring anonymous functions in Go is very lightweight:</p>
<pre><code>func() {
fmt.Println("hello")
}
</code></pre>
<p>As an expression, that’s fairly useless. Generally you want to save it to a variable, stick it in a data structure, or pass it to another function. It’s fairly common to see this type of thing hanging out in Go code:</p>
<pre><code>fn := func() {
fmt.Println("hello")
}
</code></pre>
<p>We now have a variable <code>fn</code> that is a function; it is of type <code>func()</code>. It can be invoked like any other function by saying <code>fn()</code>, or reassigned to any other <code>func()</code> that you might be interested in. Of course, since we have support for closures, we can also reference some data that was declared in the function’s containing scope:</p>
<pre><code>x := 5
fn := func() {
fmt.Println("x is", x)
}
fn()
x++
fn()
</code></pre>
<p>The output of this chunk of code would be:</p>
<pre><code>x is 5
x is 6
</code></pre>
<p>Fairly standard. You can <a href="http://play.golang.org/p/RLWmn1CJyR">run that example here</a>. So far, this is mostly the same as how it looks in JavaScript, except statically typed.</p>
<h3>Function Collections</h3>
<p>Of course, we can use functions in all the same places that we can use regular data types. We can, for example, create a <a href="http://golang.org/ref/spec#Slice_types">slice</a> of functions, chose a function at random, and then execute it. We define the type “binFunc”, which is a binary function; it takes two integers, and returns one integer. This isn’t strictly necessary for this example, but typing <code>binFunc</code> over and over again is much more convenient than typing <code>func(int, int) int</code> everywhere you see it.</p>
<pre><code>type binFunc func(int, int) int
func main() {
// seed your random number generator.
rand.Seed(time.Now().Unix())
// create a slice of functions
fns := []binFunc{
func(x, y int) int { return x + y },
func(x, y int) int { return x - y },
func(x, y int) int { return x * y },
func(x, y int) int { return x / y },
func(x, y int) int { return x % y },
}
// pick one of those functions at random
fn := fns[rand.Intn(len(fns))]
// and execute it
x, y := 12, 5
fmt.Println(fn(x, y))
}
</code></pre>
<p>(<a href="http://play.golang.org/p/j3VYsFJeBC">run this example</a>)</p>
<p>Of course, this specific application is obscenely contrived, but it illustrates a core concept in Go: functions are <a href="http://en.wikipedia.org/wiki/First-class_function">first class</a>.</p>
<h3>Functions as Fields</h3>
<p>Similarly, we might be interested in using a function type for a struct field. Doing so would allow us to attach additional information to a function, such as a label that we can access at runtime:</p>
<pre><code>type op struct {
name string
fn func(int, int) int
}
func main() {
// seed your random number generator
rand.Seed(time.Now().Unix())
// create a slice of ops
ops := []op{
{"add", func(x, y int) int { return x + y }},
{"sub", func(x, y int) int { return x - y }},
{"mul", func(x, y int) int { return x * y }},
{"div", func(x, y int) int { return x / y }},
{"mod", func(x, y int) int { return x % y }},
}
// pick one of those ops at random
o := ops[rand.Intn(len(ops))]
x, y := 12, 5
fmt.Println(o.name, x, y)
fmt.Println(o.fn(x, y))
}
</code></pre>
<p>Functions can also be stored in <a href="http://golang.org/ref/spec#Map_types">maps</a>, so you could do something similar with <code>map[string]binFunc</code>.</p>
<h3>Recursive Function Types</h3>
<p>Another interesting aspect of function types is that they allow us to define recursive function types, which are function types that operate on themselves. That is, a function type that either takes it’s own type as a parameter, or uses its own type as a return value. You can use this technique to do lots of things if you contrive hard enough, so I’ve contrived a way of using recursive functions to perform a <a href="http://en.wikipedia.org/wiki/Random_walk">random walk</a>, just for the sake of illustration. Yes, this is an utterly contrived example. To start, we might say that a walk function is a function that takes a pointer to an integer, and returns a walk function:</p>
<pre><code>type walkFn func(*int) walkFn
</code></pre>
<p>It is expected to increment the integer that is being pointed to, and then return a function describing the next step that should be taken. An example of a valid <code>walkFn</code> would be the following <code>walkEqual</code> function:</p>
<pre><code>func walkEqual(i *int) walkFn {
*i += rand.Intn(7) - 3
return walkEqual
}
</code></pre>
<p>The effect of running this function would be that the integer that <code>i</code> points to would be randomly modified by +-3, and then the function would return itself. We could then perform a random walk in the following fashion:</p>
<pre><code>fn, progress := walkEqual, 0
for i := 0; i < 20; i++ {
fn = fn(&progress)
fmt.Println(progress)
}
</code></pre>
<p>Of course, this isn’t very fun. Let’s involve two more functions, <code>walkForward</code> and <code>walkBackward</code>, which will guarantee that we walk in a “forward” (i.e., positive) direction, and “backward” (i.e., negative) direction. Each function will execute itself, and then return one of the other two functions, randomly. We’ll start with a little utility function named <code>pickRandom</code> that will randomly choose between its input functions, and return that function:</p>
<pre><code>func pickRandom(fns ...walkFn) walkFn {
return fns[rand.Intn(len(fns))]
}
</code></pre>
<p>Then we define our three “walk” functions as follows:</p>
<pre><code>func walkEqual(i *int) walkFn {
*i += rand.Intn(7) - 3
return pickRandom(walkForward, walkBackward)
}
func walkForward(i *int) walkFn {
*i += rand.Intn(6)
return pickRandom(walkEqual, walkBackward)
}
func walkBackward(i *int) walkFn {
*i += -rand.Intn(6)
return pickRandom(walkEqual, walkForward)
}
</code></pre>
<p>Executing the walk is exactly as it would be before, but now notice that <code>walkEqual</code> can return either <code>walkForward</code> or <code>walkBackward</code>. You can <a href="http://play.golang.org/p/621lCnySmy">run this example on the Go Playground</a> to see how it works as a full program.</p>
<p>This particular example is deliberately contrived, so as to keep the focus on the mechanics of recursive functions. In the standard library, this technique is used to make a <a href="http://en.wikipedia.org/wiki/Lexical_analysis">lexer</a> in the <code>text/template</code> package (<a href="http://golang.org/src/pkg/text/template/parse/lex.go?s=3130:3254#L117">view in stdlib source here</a>). A full run-down on how that works is covered in Rob Pike’s talk <a href="http://www.youtube.com/watch?v=HxaD_trXwRE">Lexical Scanning in Go</a>.</p>
<h3>Function Types as Interface Values</h3>
<p>Function types in Go can also have methods. It’s hard to see, at first, why this would be useful. There are two side effects to the fact that function types can have methods in Go: firstly, since any type that can have methods can satisfy an interface, function types in Go can also be boxed into valid interface types. Secondly, since methods in Go can have either pointer or value receivers, we can use methods on function pointers to switch the function being pointed to in the method’s calling context. That’s a bit heady at first, so let’s go through those two things in order.</p>
<p>First, the most obvious way to make a function type also satisfy an interface is to pick an interface with only one method. Keeping it simple, let’s define an <code>add</code> function like we did above, but we’ll give it an <code>Error() string</code> method. In Go, any type that has an <code>Error() string</code> method is a valid <code>error</code> type, so our function can serve as both a function <em>and</em> an <code>error</code>.</p>
<pre><code>type binFunc func(int, int) int
func add(x, y int) int {
return x + y
}
func (f binFunc) Error() string {
return "binFunc error"
}
func main() {
var err error
err = binFunc(add)
fmt.Println(err)
}
</code></pre>
<p>You can run this totally useless example <a href="http://play.golang.org/p/Dzwu9Xaqiy">on the Go Playground</a>.</p>
<p>It’s worth noting that in this case, we had to perform a type conversion to convert the function <code>add</code>, which is of type <code>func(int, int) int</code> to type <code>binFunc</code>. This may seem like an onerous misfeature of the Go runtime, but what if there was <em>another</em> valid conversion for <code>func(int, int) int</code> that would implement <code>error</code>:</p>
<pre><code>type loudBinFunc func(int, int) int
func (f loudBinFunc) Error() string {
return "THIS ERROR IS A LOT LOUDER"
}
</code></pre>
<p>If both <code>binFunc</code> and <code>loudBinFunc</code> are defined, the runtime wouldn’t know how to convert our <code>add</code> function. Even if there is only one valid implementation of an interface for a given type, the runtime will not perform the conversion for us automatically. It seems very annoying at first, but it makes it easier to write bug-free code.</p>
<p>In the standard library, we can see an example of using a function type to satisfy an interface in the <code>net/http</code> package. <a href="http://golang.org/pkg/net/http/#Handler">The <code>http.Handler</code></a> type is the interface used by the standard library to register http handlers:</p>
<pre><code>type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
</code></pre>
<p>There’s also an <a href="http://golang.org/pkg/net/http/#HandlerFunc"><code>http.HandlerFunc</code></a> type, which is just a wrapper around <code>func(http.ResponseWriter, *http.Request)</code> that satisfies <code>http.Handler</code>. This allows us to use either a function <em>or</em> a struct as a web handler. The end result is that we get both the <a href="http://golang.org/pkg/net/http/#Handle"><code>http.Handle</code></a> and <a href="http://golang.org/pkg/net/http/#HandleFunc"><code>http.HandleFunc</code></a> functions, which makes it a bit more convenient to write web handlers.</p>
<p>Now that we’ve established how methods on functions allow us to satisfy an interface, I’ll show a slightly more esoteric usage that turns out to be remarkably useful: let’s stick some functions in a <a href="http://en.wikipedia.org/wiki/JSON"><code>json</code></a> document.</p>
<p><a href="http://golang.org/pkg/encoding/json/">The <code>encoding/json</code> package</a> in the standard library is the normal way to encode and decode json documents. This package is interface-based; it defines an interface, <a href="http://golang.org/pkg/encoding/json/#Unmarshaler"><code>json.Unmarshaler</code></a>, that is to be used in the unmarshaling of json documents:</p>
<pre><code>type Unmarshaler interface {
UnmarshalJSON([]byte) error
}
</code></pre>
<p>That is, if you define this method, the <code>encoding/json</code> package will use this method when unpacking json data; otherwise, it attempts to figure out what to do on its own (and generally gets it correct). Let’s stick with the <code>binFunc</code> example, but this time, we’ll stick some binary functions in a json document. We’ll start with the tired old <code>binFunc</code> definition:</p>
<pre><code>type binFunc func(int, int) int
</code></pre>
<p>Since functions themselves do not have any persistent data, we need somewhere to register names. We can use a <code>map[string]binFunc</code> to store our functions with their associated names:</p>
<pre><code>var fnRegistry = map[string]binFunc{
"add": func(x, y int) int { return x + y },
"sub": func(x, y int) int { return x - y },
"mul": func(x, y int) int { return x * y },
"div": func(x, y int) int { return x / y },
"mod": func(x, y int) int { return x % y },
}
</code></pre>
<p>In Go, if we attempt to retrieve a value from a <code>map</code> with a nonexistent key, the retrieval operation will return <a href="http://dave.cheney.net/2013/01/19/what-is-the-zero-value-and-why-is-it-useful">the zero value</a> for the given type (if you’re asking “but what if we legitimately put <code>nil</code> into that map?” then <a href="http://golang.org/doc/effective_go.html#maps">see here</a>). For function types, the zero value is always <code>nil</code>, so we can just test against nil to see if we’ve retrieved a value. Now we need to actually implement the <code>UnmarshalJSON</code> method on our <code>binFunc</code> type. Since we want to make changes that are seen by the caller of the function, we want to implement <code>UnmarshalJSON</code> with a pointer receiver, and since we know we want a string, we just piggy-back of the existing string unmarshaling capabilities of the standard json package.</p>
<pre><code>func (fn *binFunc) UnmarshalJSON(b []byte) error {
// start by unmarshaling the name of the function that we want
var name string
if err := json.Unmarshal(b, &name); err != nil {
return err
}
// get the function out of our function registry
found := getBinFunc(name)
if found == nil {
// return a descriptive error if we can't find the function
return fmt.Errorf("unknown function in (*binFunc)UnmarshalJSON: %s", name)
}
// dereference the pointer receiver, so that the changes are visible to the caller
*fn = found
return nil
}
</code></pre>
<p>With that defined, we can now unmarshal a <code>binFunc</code> from a <code>json</code> document. <a href="http://play.golang.org/p/tD0bJOFgux">See here for the full, runnable example</a> that unpacks just one function, and <a href="http://play.golang.org/p/hwd9TeOzg7">here’s an example</a> of using this to unpack a list of functions. Of course, this is just the tip of the iceberg, but it shows what types of things can be done with this technique.</p>
<h3>Channels of Functions</h3>
<p>There’s one last pattern that is very unique to Go, and that’s the result of what happens when you combine function types with <a href="http://golang.org/ref/spec#Channel_types">channels</a>. A full description of Go’s concurrency model is out of scope for this post, but for the unfamiliar looking to get their toes wet with Go’s concurrency model, I recommend reading <a href="http://golang.org/doc/effective_go.html#concurrency">the concurrency section in Effective Go</a>, or if you have the time for it, watch the very excellent Google I/O talk <a href="http://www.youtube.com/watch?v=f6kdp27TYZs">Go Concurrency Patterns</a>. From this point forward, some knowledge of Go is assumed.</p>
<p>Since channels are primitives in Go, and we can make channels of any other type, we can even make channels of functions. Since functions can be anonymous, and functions are closures, we can combine these three properties to make channels of anonymous function closures. The definition for this channel type is obvious once you realize that it is possible: <code>chan func()</code>. Keeping in the theme of this post, let’s start by making a slice of functions, but we’ll make each one a closure:</p>
<pre><code>x := 10
fns := []func(){
func() { x += 1 },
func() { x -= 1 },
func() { x *= 2 },
func() { x /= 2 },
func() { x *= x },
}
</code></pre>
<p>We’ll also nab another piece of code we looked at earlier, and device a mechanism for picking one of those functions at random:</p>
<pre><code>func pickFunc(fns ...func()) func() {
return fns[rand.Intn(len(fns))]
}
</code></pre>
<p>Next, we make a channel of <code>func()</code> values:</p>
<pre><code>c := make(chan func())
</code></pre>
<p>And we define a function that takes as its parameters a function channel, a number describing how many functions it should write into this channel, and then a set of candidate functions:</p>
<pre><code>func produce(c chan func(), n int, fns ...func()) {
defer close(c)
for i := 0; i < n; i++ {
c <- pickFunc(fns...)
}
}
</code></pre>
<p>At the start of this function, we use <code>defer close(c)</code> to ensure that our channel gets closed when we’re done producing values. From there, we just use a regular <code>for</code> loop, pick a function at random using <code>pickFunc</code>, and then write it into the channel that we’ve been given. On the receiving end, all we need to do is read values from the channel using <a href="http://golang.org/ref/spec#For_statements">a <code>range</code> clause</a>, then immediately execute these functions upon being read. By adding a sleep after each read operation, we’re able to implement a rate-limiting device for arbitrary closures.</p>
<pre><code>for fn := range c {
fn()
fmt.Println(x)
time.Sleep(delay)
}
</code></pre>
<p>Putting this all together, we arrive at the following program:</p>
<pre><code>package main
import (
"fmt"
"math/rand"
"time"
)
var delay = 200 * time.Millisecond
func pickFunc(fns ...func()) func() {
return fns[rand.Intn(len(fns))]
}
func produce(c chan func(), n int, fns ...func()) {
defer close(c)
for i := 0; i < n; i++ {
c <- pickFunc(fns...)
}
}
func main() {
// time is frozen on Playground, so this is always the same.
rand.Seed(time.Now().Unix())
x := 10
fns := []func(){
func() { x += 1 },
func() { x -= 1 },
func() { x *= 2 },
func() { x /= 2 },
func() { x *= x },
}
c := make(chan func())
go produce(c, 10, fns...)
for fn := range c {
fn()
fmt.Println(x)
time.Sleep(delay)
}
}
</code></pre>
<p><a href="http://play.golang.org/p/2YVKIj6PaW">Run this example on the Go playground.</a></p>
<h3>Conclusion</h3>
<p>Functions are great!</p>