Ljubljana, Slovenia

Networking basics with Docker

If you run several docker containers, you might have already come to the requirement, that they talk to eachother via network. After all, your web application container should use your database container to store to and retrieve data from. This means, that in some way, your webserver container should talk to your database container.

One of the simplest (and ill-advised, deprecated ways) is to link containers. For example:

  1. Start up the db1 container as you would
  2. Start up the web1 container with --link db1:db1

This way, you can talk to db1 from web1 over the network. It’s however slightly awkward if you would want to start the web1 container before you would start the db1 container. There are many pains with using the --link option, which why somewhere down the line, docker introduced the concept of a network definition. You can run docker help network to find out more about it, or just read the longer official documentation.

What you will want to do, is create your own network like this:

docker network create -d bridge --subnet 172.25.0.0/24 party

This will create a network named party. The good thing about this network is that docker provides a DNS server for service discovery on this user-defined network. This means that when you add servers to the network, they can see each-other as soon as they are alive.

The following script starts two containers, wh1 and wh2, attached to this user-defined network. The container is a nginx web server, which will run until you stop it.

#!/bin/bash
NAMES="wh1 wh2"
for NAME in $NAMES; do
        docker rm -f $NAME
        docker run --net=party -d -h $NAME --name $NAME jrelva/nginx-autoindex
done

After running, you can see that the containers can connect to each-other.

# docker exec -it wh1 ping wh2 -c 2
PING wh2 (172.25.0.3): 56 data bytes
64 bytes from 172.25.0.3: icmp_seq=0 ttl=64 time=0.125 ms
64 bytes from 172.25.0.3: icmp_seq=1 ttl=64 time=0.167 ms
--- wh2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.125/0.146/0.167/0.000 ms

# docker exec -it wh2 ping wh1 -c 2
PING wh1 (172.25.0.2): 56 data bytes
64 bytes from 172.25.0.2: icmp_seq=0 ttl=64 time=0.152 ms
64 bytes from 172.25.0.2: icmp_seq=1 ttl=64 time=0.189 ms
--- wh1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss

But, as soon as you turn off one container, good bye name resolution:

# docker stop wh2
# docker exec -it wh1 ping wh2
ping: unknown host

I don’t know exactly where it happened in my professional history of system administration and development, but somewhere along the line I discovered the /etc/hosts file to be the most reliable way to provide host name resolution for active applications on the servers. You don’t need a DNS server, and literally, never ever in all my history, has this method failed for me.

All servers which I deploy in production have been assigned IP adresses in a private LAN network. If a server goes offline for any reason, I can be 100% sure that it will have the same IP when it will come back online. Applications handle any kind of fail-over/backup procedures if the host is not reachable. But it’s IP is always known.

Statically assigning IPs to your docker containers

Some applications just need to know the IP of the server, while not requiring actual connectivity to the server. For example, nginx upstreams need to resolve to a valid IP when you start the service itself. It’s not important if this IP is alive at this time or not. There are fallback mechanisms in nginx, haproxy and other services that use fallback/backup locations when the primary one is not available. Or you just might start up the back-end service docker container after the load-balancer/reverse proxy.

Docker provides the --ip option with the docker run command. This means that you can assign a static IP to your docker container. It’s your choice how you manage the IPs, but we generally manage them in some sort of system inventory. Technically, this can be anything, we developed our own system. For example, at Tubmlr, they develop and use Collins.

Limitations

Knowing the IPs and container names, it’s possible to generate a hosts file for your containers. While this is the obvious next step to provide some kind of persistant name resolution. It may be possible just to put the hosts file to each container with a volume mount, but I’m not a fan of extending my docker run options ad nauseam.

Eventually you will hit an obvious issue - the docker network is contained on the host only, meaning that when you will need to scale to several hosts, you will have to manage network connectivity between them.

Docker also provides multi-host networking which is the next step from this tutorial. It’s configuration is a bit longer, but generally nothing much changes from the single-host method. All you need to do is pair your hosts, and create an overlay network, much in the same way we created the bridge network above. After the network is created, the containers have connectivity between them, even if they are running on different hosts.

Closing words

User-defined networking in docker is a real-world usage pattern. It’s not practical nor advised to run all your software within one container. It’s the responsibility of the database to store and retrieve data - and you shouldn’t be forced into using a specific database just because it’s bundled with a generic LAMP container which works. If something is easy, it doesn’t mean that it’s right.

Choose your database. Choose your web servers. Choose your application stack. And take some time to lay out a plan how you will provide connectivity between them. A list of manually assigned IPs to containers is a good start. Continue this line of thought by thinking what is your bare minimum of what you need, and scale out to multiple hosts from there.

While I have you here...

It would be great if you buy one of my books:

I promise you'll learn a lot more if you buy one. Buying a copy supports me writing more about similar topics. Say thank you and buy my books.

Feel free to send me an email if you want to book my time for consultancy/freelance services. I'm great at APIs, Go, Docker, VueJS and scaling services, among many other things.