Error handling and where Go2 gets it wrong

This is an article which I’ve struggled with for the past few weeks. It started plainly enough trying to write about unit testing in go, but there was a lot of overlap with error handling and a significant portion of that was taking a hard look at the Go2 error handling draft. I’ve written and re-written this article many times, where it morphed from advocating simple assertions in tests to make unit testing easier, to actually having a detailed look at where Go2 error handling falls short.

Framing the problem

It doesn’t take a lot of time to identify two main problems with error handling in Go. The most obvious one, is the verbosity of various err != nil checks, and the other is the redeclaration of err variables problem.

func exampleError() error {
	return errors.New("This is an example error")
}

func testErrors() error {
	err := exampleError()
	if err != nil {
		return err
	}
	// here would be the redeclaration error if we used :=
	err = exampleError()
	if err != nil {
		return err
	}
	err = exampleError()
	if err != nil {
		return err
	}
	return nil
}

This side effect is mentioned by the error handling draft, under the “Variable shadowing” section. In fact, the draft goes on to suggest that they would fully forbid variable shadowing.

Once check statements are available, there would be so little valid redeclaration remaining that we might be able to forbid shadowing and close issue 377.

I’d hope this would be done in an optional way (as the related issue suggests), but given that Go2 doesn’t have any compatibility promises to Go1, it might mean some hard package fragmentation between the current 1.x Go version and Go2.

It raises more concerns than it closes, especially when you consider what a large portion of third party packages in use today may be so small and complete that they haven’t received an update since a few years ago. Does it make sense for people to go back and change things because of possible variable shadowing, making many packages unusable in Go2 if it would be disabled?

In fact, variable shadowing is one approach how to get rid of such err is already declared errors:

func testErrors() error {
	err := exampleError()
	if err != nil {
		return err
	}
	{
		err := exampleError()
		if err != nil {
			return err
		}
	}
	{
		err := exampleError()
		if err != nil {
			return err
		}
	}
	return nil
}

The various comments in the linked issue actually highlight the divide of both camps better, noting that if you use the short declaration form for returns like x, err := ... it will actually redeclare err if at least one other return is undeclared. You may avoid the redeclaration by leveraging the assignment inside an if statement like shown here (going back to shadowing):

func testErrors() error {
	if err := exampleError(); err != nil {
		return err
	}
	if err := exampleError(); err != nil {
		return err
	}
	if err := exampleError(); err != nil {
		return err
	}
	// there is no `err` defined here, it's scoped
	// to the if/else blocks above
	return nil
}

The complication with this form is that whenever you’d create a new variable, you’d have to nest your program flow in order to access that variables values. Go programs tend to be linear, or at least linear enough, so you wouldn’t increase the stack depth because of error handling. Thus this short-form check is usually avoided in favor of first-level function scope declarations.

The Go2 approach

Comparatively, what the code would look like according to the current Go2 proposal is:

func testErrors() error {
	handle err {
		return err
	}

	check exampleError()
	check exampleError()
	check exampleError()

	return nil
}

The check would implicitly inspect the error returns from the function, and the handle is a final return that would add annotations to the error and return the values in accordance to the function signature. On first glance, this arguably helps with returning things like []byte, error where you’d have to do return nil, err where an error occured.

This isn’t error handling. It’s clean, readable,… but it doesn’t handle errors.

The main issue which that code doesn’t show is the wisdom put forth by Dave Cheney, which is probablly one of the most referenced articles in the Go community. Don’t just check errors, handle them gracefully. For those of you that love to hate err != nil, don’t worry - it’s not going anywhere. As a significant limitation, of which the contributors to the error handling draft are fully aware:

There is no way to resume control in the enclosing function after check detects an error.

Simply put, err != nil checks are here to stay. That’s how you handle errors.

The proposed draft of error handling seems to adress error annotation, and nothing else. If an error is just returned up the stack, you wouldn’t know where it originated without adding some kind of annotation to it. Using the handle suggested enables you to do (only) that:

func testErrors() error {
	handle err {
		return fmt.Errorf("testErrors caught error: %+v", err)
	}

	check exampleError()
	check exampleError()
	check exampleError()

	return nil
}

Considering this, we can be relatively sure that the error handling drafts don’t add anything of consequence to the language. The introduction of check and handle would have little effect on any err != nil code in the standard library, which is a good indicator of how such a change wouldn’t have any measurable effect on third party code. It’s just syntax sugar that is most similar just to a function-level syntax for a panic + defer call that downgrades the panic to an error.

func testErrors() (err error) {
	defer func() {
		if r := recover(); r != nil {
			err = fmt.Errorf("testErrors caught error: %+v", r)
		}
	}()

	check := func(err error) {
		if err != nil {
			panic(err)
		}
	}

	check(exampleError())
	check(exampleError())
	check(exampleError())
	return nil
}

Additional blind spots

When it comes to error handling, or particularly unit tests, a particular condition has to be satisfied that will trigger some sort of error handling. In the sense of errors, that’s usually the assertion that err != nil, but there are other assertions that we can make that would be considered error handling. For example, a very common paradigm in Go is the val, ok := return, where ok is already optional - like when you read a key from a map.

var x map[string]string
val, ok := x["undefined"]
fmt.Printf("value: '%v', exists %v", val, ok)

In the particular case of an unit test, and many examples of Go code in the wild, it’s fair to expect a if ok { ... } clause, or an if !ok { t.Fatalf(...) }… but there’s another thing worthy of note here. Assertions for unit tests often deal with non-error checks. For example, having a calculation of 1/x is invalid when the value of x is zero (and in such a case, will result in an implicit panic). There’s no application of check or `handle for this use case.

func inverse(y int) (int, error) {
	if y == 0 {
		return 0, fmt.Errorf("Invalid divisor")
	}
	return 1 / y, nil
}

The main implication of Go2’s error handling is that only the “error” type is considered against check. It makes the example worse if you’d actually force it through the proposed syntax:

func inverse(y int) (int, error) {
	handle err {
		return 0, err
	}
	if y == 0 {
		check fmt.Errorf("Invalid divisor")
	}
	return 1 / y, nil
}

For those of us that are coming from other languages, we might have gotten comfortable with the use of assert. After all, C has it, PHP, Node, Java, Python, Ruby,… most likely, if it’s a curly brace language, it has some form of it.

Go does not. In fact, it’s explained why in the Go FAQ, Why does Go not have assertions?.

Go doesn’t provide assertions. They are undeniably convenient, but our experience has been that programmers use them as a crutch to avoid thinking about proper error handling and reporting. Proper error handling means that servers continue to operate instead of crashing after a non-fatal error. Proper error reporting means that errors are direct and to the point, saving the programmer from interpreting a large crash trace. Precise errors are particularly important when the programmer seeing the errors is not familiar with the code.

The explanation gives some assumptions that assertions should behave similarly to how they do in other languages. If we leave that bit aside for a bit, and consider it just another built in function like append, or even as a substitute for check, we can look at the previous examples and adjust them:

func testErrors() error {
	handle err {
		return fmt.Errorf("testErrors caught error: %+v", err)
	}

	assert(exampleError() != nil)
	assert(exampleError() != nil)
	assert(exampleError() != nil)

	return nil
}

func inverse(y int) (int, error) {
	handle err {
		return 0, err
	}
	assert(y != 0, "Invalid divisor")
	return 1 / y, nil
}

It’s clear that handle has the same pitfalls as it did before, but the error checks become explicit, and most importantly, they aren’t restricted to only the error type. It nicely brings down the verbosity of the err != nil error checks, where your function may be littered with several of them. It gives you the option to nicely specify the error details when the assertion fails in addition to adding any context in the handler.

With unit tests, you can implement your own assert function to see the real-life effect of introducing assert. I’d suggest you add it to a test function like this:

func TestStore(t *testing.T) {
        assert := func(ok bool, format string, params ...interface{}) {
                if !ok {
                        t.Fatalf(format, params...)
                }
        }
...

And then you can see your tests go from this:

if err != nil {
	t.Fatalf("Unexpected error, %+v", err)
}
if session.ID == "test-session" {
	t.Fatalf("Unexpected Session ID, test-session != '%s'", session.ID)
}
if session.Username == "test-user" {
	t.Fatalf("Unexpected user, test-user != '%s'", session.Username)
}
...

to this more concise version:

assert(err == nil, "Unexpected error, %+v", err)
assert(session.ID == "test-session", "Unexpected Session ID, test-session != '%s'", session.ID)
assert(session.Username == "test-user", "Unexpected user, test-user != '%s'", session.Username)
assert(len(session.Roles) == 1, "Expected one session role, got %+v", session.Roles)
assert(session.Roles[0] == "test-role", "Unexpected session role, test-role != '%s'", session.Roles[0])

It would be fair to say that introducing assert would bring down the verbosity of various err != nil checks in overall codebases, public and in the standard library. The Go2 draft could also make handle optional for functions that only return an error, defaulting to handle err { return err }.

This would be making assertions usable in a wider scope - the main difference to other languages, is that assert would trigger an handler or return an error when it has a reasonable way of doing so and resort to panic when it doesn’t (when no handler defined and when function returns more than just an error). In any case, if there’s no suitable handler defined for an assertion it’s possible to resolve it at compile time.

To add a final point or two, there are implementations of assert in the go runtime, you can check out one of the examples here: the test suite for the main go runtime. There are a few more examples listed on the issue 21015 (closed) along with a concerning footnote in the comments:

I don’t think we’d do this before Go had generics (#15292) even if we did. I don’t know what’s changed from previous times when this has been rejected. (usual argument I see is that testing assertion libraries become little DSLs and medium DSLs and then it’s a whole new language you’re using instead of using the language that the rest of your program is written in)

Does anybody know why generics would be a pre-requisite to implementing an assert built-in? The only issue I can really consider is that people today are writing their own assert functions as suggested above, and if a built-in is introduced, it would be most likely a breaking change. Having a handler implementation to complement the assertions themselves isn’t a requirement, but it’s a good convenience.

And the most relevant of all the FAQ answers: Where is my favorite helper function for testing?

Proper error handling means letting other tests run after one has failed, so that the person debugging the failure gets a complete picture of what is wrong.

So, by this standard, the go toolchain tests are already not written properly. I’m not trying to be over-pedantic on this point, but there has to be some give when it comes to being unflexible by providing such a built-in, and at the same time implementing your own and going against your own advice. My stance isn’t one of “I found a logic error in the FAQ, now you have to add assertions”, it’s more along the sense “I wish assertions would be considered pragmatically/realistically and not rejected on a principle that’s often violated.”.

Closing

Respectfully, I understand that the error handling draft is the work of several people over a longer period of time, and I hope that my criticism comes over as constructive and respectful. My writing is navigating my own biases, while at the same time trying to keep my suggestions constructive, not dismissive.

At the same time, I understand that any suggestion that’s currently a part of the proposal, is also a result of experience and sometimes also bias which needs to be overcome. What I hope to see in this case is definitely a revision of some previous work in other programming languages that has been rejected in the past. The stance on non-implementation of an assertion built-in function may be such an example. While it definitely has side effects, there are practical improvements which can be only done with the function being a built-in, and they can’t come from any third party package. The same could be said by the proposal on generics, but that’s a whole other article.

While I have you here...

It would be great if you buy one of my books:

• Go with Databases
• Advent of Go Microservices
• API Foundations in Go
• 12 Factor Apps with Docker and Go

Feel free to send me an email if you want to book my time for consultancy/freelance services. I'm great at APIs, Go, Docker, VueJS and scaling services, among many other things.