Ljubljana, Slovenia

Store config in the environment

One of the most important 12 Factor Application principles is dictating how your configuration should be declared and passed to your application. If you’re using a database instance, the location and credentials for that database instance should come from the environment where your application is run. This means that it should be passed via an environment variable, via a command line argument, or even via a configuration file which must be passed as a command line argument.

The “core” of this requirement of a 12 factor app is to remove experienced bad practice. People defining constants in the code have disabled themselves from changing the environment without rebuilding the whole application. They may have commited database credentials into source control - even with policies regarding database usernames and passwords, this data may be sensitive in other cases, as for example, exposing AWS credentials. Even keeping a configuration file with such information is sensitive - it may be added to source control by mistake, causing the same problem.

All configuration for environment needs to be managed outside of your application. Imagine a basic requirement that your application can be open sourced or the source code may just be bought by a third party. You don’t want to expose your AWS or other sensitive credentials. It’s always worth to ask yourself: can I publish my application code to the public, without getting burned?

It’s a rethorical question: people unknowingly published their AWS credentials and woke up to find out that their Amazon bill is in the thousands. It’s the story of Ryan Hellyer, a WordPress developer who managed to leak his credentials to GitHub and ended up with a $6K bill, another got a $50K bill and such stories are not uncommon. Amazon might waive some fees here, but having best practices in place that avoid your exposure is priceless.

Flags

When it comes to the standard lib, there’s the flag package, which implements command-line flag parsing. The third party extension to this is the namsral/flag package, which is a drop in replacement for the standard lib package with the addition that it parses files and environment variables.

package main

import (
	"fmt"
	"github.com/namsral/flag"
)

func main() {
	var age int

	flag.IntVar(&age, "age", 0, "age of gopher")
	flag.Parse()

	fmt.Print("age:", age)
}

The flags example shows just how easy it is to pass ENV variables and use them inside Go programs. Using docker, you can pass the ENV variables with the -e option, like this:

#!/bin/bash
docker run --rm -e "AGE=35" -v $(pwd):/go/src/app -w /go/src/app golang go run flags.go

Passing config to packages

With the flag package there’s the option that individual packages will define what kind of flags they need. This might become a bit of a problem - after all, you can’t prevent that two different packages will need the same configuration by name. What you need to enforce is where you will define your flags and how you will pass them to the correct location where they are needed.

The suggested practice is to declare flags in the main package, where they are also injected into packages as needed. As the flags are defined in one place, there is less chance with conflicts with other packages. Some public packages of course define their own flags - as such, I’d recommend filing issues against the offending packages to provide an idiomatic way for setting configuration flags via some kind of functional options API.

Best practices

I would recommend that you should define your flags in the func main() block. By doing this, they become unavailable for reading/writing by other functions from the global scope, and you have full control how this configuration is passed to packages and other functions. The side effect of this is also that packages have a declared interface over which configuration is set.

My current approach is to provide functional options to packages, as they require it. For example, the redis.so from my common package (not the best name) looks like this:

package common

import "time"
import "github.com/garyburd/redigo/redis"

type Redis struct {
	conn                                      redis.Conn
	address                                   string
	connectTimeout, readTimeout, writeTimeout time.Duration
}

type RedisOption func(*Redis)

func RedisAddress(address string) RedisOption {
	return func(do *Redis) {
		do.address = address
	}
}

func RedisConnectTimeout(timeout time.Duration) RedisOption {
	return func(do *Redis) {
		do.connectTimeout = timeout
	}
}

func RedisReadTimeout(timeout time.Duration) RedisOption {
	return func(do *Redis) {
		do.readTimeout = timeout
	}
}
func RedisWriteTimeout(timeout time.Duration) RedisOption {
	return func(do *Redis) {
		do.writeTimeout = timeout
	}
}

func NewRedis(options ...RedisOption) *Redis {
	redis := &Redis{
		address:        "redis:6379",
		connectTimeout: time.Second,
		readTimeout:    time.Second,
		writeTimeout:   time.Second,
	}
	for _, option := range options {
		option(redis)
	}
	return redis
}

The functions that return a RedisOption are passed as variadic arguments to the NewRedis constructor function. The constructor takes care of reasonable defaults, while honoring passed RedisOption structs to allow customisation of all options.

This way worked best for me, but your examples may be less complex. In such cases, passing whatever you need as function arguments might suffice for your case. If you look at chapter7/main.go you can see that the port flag is used without any special interfaces.

godotenv

GoDotEnv is a port of the dotenv package available with Ruby. This package loads ENV variables from a .env file which should be located in the root of your project. Keep in mind, that this file shouldn’t be commited in your git repository, but should come to your development, staging or production environments from your infrastructure management solution or simply as part of your testing or deployment environments.

To use the package, import github.com/joho/godotenv and add the foolowing line to the beginning of your main function. It should be added before using the namsral/flag package:

godotenv.Load()

We can use the flags example from the start of the chapter and set the required configuration flag into the .env file which we create.

echo AGE=1337 > .env

Environment variables have precedence over the variables declared in .env; This enables you to provide sane defaults via the .env file and then provide higher-level overrides via declared environment variables. This means if you will provide -e AGE=35 with the docker command line, it will ignore the value written in your .env file.

This is a part of the third chapter of 12 Factor Applications with Docker and Go. If you’d like to follow along, consider buying a copy. The book samples as they are written are published in my books repository on GitHub.

While I have you here...

It would be great if you buy one of my books:

I promise you'll learn a lot more if you buy one. Buying a copy supports me writing more about similar topics. Say thank you and buy my books.

Feel free to send me an email if you want to book my time for consultancy/freelance services. I'm great at APIs, Go, Docker, VueJS and scaling services, among many other things.