How many times have you asked a customer to run a test with some specific tools, just to have to fight countless compatibility problems due to the particularities of the customer’s environment?
Yeah, I’ve been there too.
Containers should be designed to be immutable and run in the same way any time you deploy them, regardless of the environment.
This article will give you some guidelines to containerize any application. Once your application runs in a container, you don’t have to care anymore about compatibility issues, libraries or DLL not available, etc
The containerization steps
The process takes just a few steps, which I’ll briefly describe in this section.
Step 1: Get your container development environment ready!
Step 2: Choose a base image for your container.
Step 3: Find out how to run your tool inside a container
Step 4: Convert the previous steps into a Dockerfile.
Step 5: Test your container image.
Step 6: Push your container image to a repository.
Setting up your container development environment
I recommend downloading and installing Docker for Windows, which is free: https://www.docker.com/docker-windows
Depending on which OS does your tool run, you have to choose to run Windows Containers or Linux Containers (you can switch between both at any time):
In this guide we will show you how to containerize a Linux tool.
Now run docker run hello-world to check that Docker has been correctly installed:
$ docker run hello-world Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world 9a0669468bf7: Pull complete Digest: sha256:0e06ef5e1945a718b02a8c319e15bae44f47039005530bc617a5d071190ed3fc Status: Downloaded newer image for hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. (...)Output trimmed for brevity(...)
Docker will download and run a simple container that shows the above message.
Choosing a base image for your container
You can create containers from scratch, but that’s out of scope for this guide. In our case we will choose a base image and build functionality on top of it.
A base image could be just a container running e.g. full Ubuntu:
$ docker run -it ubuntu bash Unable to find image 'ubuntu:latest' locally latest: Pulling from library/ubuntu ae79f2514705: Pull complete c59d01a7e4ca: Pull complete 41ba73a9054d: Pull complete f1bbfd495cc1: Pull complete 0c346f7223e2: Pull complete Digest: sha256:6eb24585b1b2e7402600450d289ea0fd195cfb76893032bbbb3943e041ec8a65 Status: Downloaded newer image for ubuntu:latest root@a3e5e569e2ab:/# cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
The above downloads and runs an Ubuntu 16.04.3 LTS container, executing the command “bash” on start, which will give us an interactive shell (-it). It took about 15 seconds to download and started immediately on my laptop.
This is the size of the Ubuntu container we’re running:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE Ubuntu latest dd6f76d9cc90 6 days ago 122MB
There are much smaller alternatives, but for the sake of simplicity we will use this container as a base image for our purposes.
Find out how to run your tool inside a container
The next step we need to do is to test our tool inside a container. For that purpose, we will just run the Ubuntu container as we did on the previous step and once we’re in, we will install and run our tool.
I have chosen to containerize iPerf3 as it’s a tool we all use often. What I’ll do now is:
- Get into an Ubuntu container
- Download and install iPerf
- Test it
$ docker run -it ubuntu bash Unable to find image 'ubuntu:latest' locally latest: Pulling from library/ubuntu ae79f2514705: Pull complete c59d01a7e4ca: Pull complete 41ba73a9054d: Pull complete f1bbfd495cc1: Pull complete 0c346f7223e2: Pull complete Digest: sha256:6eb24585b1b2e7402600450d289ea0fd195cfb76893032bbbb3943e041ec8a65 Status: Downloaded newer image for ubuntu:latest root@34e7118c6bb1:/# apt update Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB] Get:2 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] (...)Output trimmed for brevity(...) root@34e7118c6bb1:/# apt install iperf3 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libiperf0 The following NEW packages will be installed: iperf3 libiperf0 0 upgraded, 2 newly installed, 0 to remove and 4 not upgraded. Need to get 58.5 kB of archives. After this operation, 238 kB of additional disk space will be used. Do you want to continue? [Y/n] Get:1 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libiperf0 amd64 3.0.11-1 [50.4 kB] Get:2 http://archive.ubuntu.com/ubuntu xenial/universe amd64 iperf3 amd64 3.0.11-1 [8090 B] (...)Output trimmed for brevity(...) Setting up iperf3 (3.0.11-1) ...
I have an iPerf3 server listening on another machine on IP address 10.1.4.5, so I’ll just try to run a test against it:
root@34e7118c6bb1:/# iperf3 -c 10.1.4.5
Connecting to host 10.1.4.5, port 5201
[ 4] local 172.17.0.3 port 47296 connected to 10.1.4.5 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 124 MBytes 1.04 Gbits/sec 1513 233 KBytes
[ 4] 1.00-2.00 sec 116 MBytes 972 Mbits/sec 1742 147 KBytes
(...)Output trimmed for brevity(...)
[ 4] 9.00-10.00 sec 116 MBytes 976 Mbits/sec 1497 129 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec 14378 sender
[ 4] 0.00-10.00 sec 1.15 GBytes 985 Mbits/sec receiver
iperf Done.
YAY!
Note: Outbound traffic will be translated to the host’s IP address, but inbound traffic needs to be explicitly permitted.
Convert the previous steps into a Dockerfile
A Dockerfile is the file that defines how Docker should build a container.
The first thing we need to specify in our Dockerfile is the base image:
FROM ubuntu:latest
The above specifies we should use the image ubuntu on its latest version. Whatever comes after the colon is called “tag” and it’s used to define specific versions.
This simplifies things for us on building time, but it means that every time someone builds our container they will be pulling the latest ubuntu version (which will be changing overtime!). These are potentially breaking changes, so my recommendation would be to choose a specific version. Let’s change it as follows:
FROM ubuntu:16.04
16.04 is a valid tag as per https://hub.docker.com/_/ubuntu/ and it’s the current stable version.
Now let’s make sure users can find and contact us if they have any issues running our container:
LABEL maintainer="pedro.perez@microsoft.com"
(and for vanity Internet points 😊)
Finally, time to get some action. We will instruct Docker to run the commands we also ran to download and install iPerf3. If you scroll above you’ll find we first had to update apt repositories (you need to do this as the local package lists inside the container are empty by default) and then proceeded to install iPerf3.
RUN apt update \ && apt install iperf3 -y
Note: We specified -y to force YES to any question as you won’t be able to interact with your container during build time.
And the last step would be to instruct Docker what command should it ran by default:
ENTRYPOINT ["iperf3"]
Save this all in a file called “Dockerfile” on your local computer:
FROM ubuntu:16.04 LABEL maintainer="pedro.perez@microsoft.com" RUN apt update \ && apt install iperf3 ENTRYPOINT ["iperf3"]
Note: If you use Visual Studio Code, it supports Dockerfiles and will help you with inline help and spotting any issues.
Build your container image using your Dockerfile
We’re nearly there! Our next step is to get Docker to build a container following the instructions on our Dockerfile. The command we need is docker build -t repository/container-name and specify the path to our Dockerfile. In our case it will be the current folder, hence will just write a dot “.”
The repository in my case is my username in Docker Hub. The container name could be anything you’d like. It is recommended to make it self-explanatory. I have chosen “ubuntu-iperf3”
$ docker -t pedroperezmsft/ubuntu-iperf3 build . Sending build context to Docker daemon 2.048 kB Step 1 : FROM ubuntu:16.04 ---> dd6f76d9cc90 Step 2 : LABEL maintainer "pedro.perez@microsoft.com" ---> Using cache ---> abb0c99425b4 Step 3 : RUN apt update && apt install iperf3 -y ---> Running in 6f436c32e642 Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB] (...)Output trimmed for brevity(...) Setting up iperf3 (3.0.11-1) ... ---> 1ad73703c4d5 Removing intermediate container 6f436c32e642 Step 4 : ENTRYPOINT iperf3 ---> Running in 98995dca9ade ---> 853b98f9719f Removing intermediate container 98995dca9ade Successfully built 853b98f9719f
Let’s check the image has been created:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE pedroperezmsft/ubuntu-iperf3 latest 853b98f9719f 5 minutes ago 161.7 MB
Test your container image
Now it’s time to deploy a container from our container image above:
$ docker run pedroperezmsft/ubuntu-iperf3 -c 10.1.4.5 Connecting to host 10.1.4.5, port 5201 [ 4] local 172.17.0.3 port 50084 connected to 10.1.4.5 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 123 MBytes 1.03 Gbits/sec 2390 183 KBytes [ 4] 1.00-2.00 sec 116 MBytes 971 Mbits/sec 2055 287 KBytes (...)Output trimmed for brevity(...) [ 4] 9.00-10.00 sec 118 MBytes 990 Mbits/sec 1367 261 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec 15077 sender [ 4] 0.00-10.00 sec 1.15 GBytes 984 Mbits/sec receiver iperf Done.
Yay again!
We have a working container. Now the next step would be to test it as a server. As I’ve mentioned previously, inbound traffic to our container must be explicitly permitted. We do this by mapping external (host) ports to internal (container) ports. In our case we will be mapping external port 5201 to internal port 5201 when starting the iPerf3 server:
$ docker run -p 5201:5201 pedroperezmsft/ubuntu-iperf3 -s
Note: We can pass parameters to iperf3 just by adding them at the end of docker command.
Now I’ll start a client from a remote machine and point it to our container. The IP address I should point it to is the one from the host (10.1.4.4) because that’s where we mapped the port. This is an analogous situation to a PAT.
$ iperf3 -c 10.1.4.4 Connecting to host 10.1.4.4, port 5201 [ 4] local 10.1.4.5 port 44978 connected to 10.1.4.4 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 125 MBytes 1.05 Gbits/sec 97 693 KBytes [ 4] 1.00-2.00 sec 119 MBytes 995 Mbits/sec 191 761 KBytes [ 4] 2.00-3.00 sec 119 MBytes 1.00 Gbits/sec 196 827 KBytes (...)Output trimmed for brevity(...) [ 4] 9.00-10.00 sec 118 MBytes 993 Mbits/sec 547 1.04 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.17 GBytes 1.00 Gbits/sec 2181 sender [ 4] 0.00-10.00 sec 1.16 GBytes 1.00 Gbits/sec receiver iperf Done.
Yay once more!
Push your container image to a repository
Let’s recap for a second:
You have successfully built a container image that runs iPerf3 and you have successfully tested it for both inbound and outbound traffic. Now the next step is to make this image available to deploy on other machines. Otherwise this would happen if you try to deploy it on another computer:
$ docker run -p 5201:5201 pedroperezmsft/ubuntu-iperf3 -s Unable to find image 'pedroperezmsft/ubuntu-iperf3:latest' locally Pulling repository docker.io/pedroperezmsft/ubuntu-iperf3 docker: Error: image pedroperezmsft/ubuntu-iperf3:latest not found.
The image does not exist locally, but it doesn’t exist in Docker Hub either. Let’s fix that!
Note: This step implies you have an account created in Docker Hub https://hub.docker.com/ if you don’t, go ahead now and do it.
We are going to push our image to Docker Hub, inside our own repository. As previously mentioned, our repository’s name is pedroperezmsft – please change this for your own as created in Docker Hub.
Just before pushing I’d like to rebuild our container to specify a tag. As we didn’t specify one before, Docker assigned “latest” as the tag. Now I’d like to make sure I specify iPerf3’s version on the tag:
$ docker build -t pedroperezmsft/ubuntu-iperf3:3.0.11 ./container/ Sending build context to Docker daemon 2.048 kB Step 1 : FROM ubuntu:16.04 ---> dd6f76d9cc90 Step 2 : LABEL maintainer "pedro.perez@microsoft.com" ---> Using cache ---> abb0c99425b4 Step 3 : RUN apt update && apt install iperf3 -y ---> Using cache ---> 1ad73703c4d5 Step 4 : ENTRYPOINT iperf3 ---> Using cache ---> 853b98f9719f Successfully built 853b98f9719f $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE pedroperezmsft/ubuntu-iperf3 3.0.11 853b98f9719f 22 minutes ago 161.7 MB
That was easy.
Let’s now push our image:
$ docker push pedroperezmsft/ubuntu-iperf3:3.0.11 The push refers to a repository [docker.io/pedroperezmsft/ubuntu-iperf3] 7aec54326e0e: Preparing 174a611570d4: Preparing f51f76255b02: Preparing 51db18d04d72: Preparing f1c896f31e49: Preparing 0f5ff0cf6a1c: Waiting unauthorized: authentication required
Oops!
It seems we first need to authenticate against Docker Hub with the credentials we have chosen when we signed up on the website:
$ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: pedroperezmsft Password: Login Succeeded
Now we’re authenticated. Let’s try to push the image again to the repository:
$ docker push pedroperezmsft/ubuntu-iperf3:3.0.11 The push refers to a repository [docker.io/pedroperezmsft/ubuntu-iperf3] 7aec54326e0e: Pushed 174a611570d4: Pushed f51f76255b02: Pushed 51db18d04d72: Pushed f1c896f31e49: Pushed 0f5ff0cf6a1c: Pushed 3.0.11: digest: sha256:2652039f39c8825a71771e2f52842c10395fad10c07254d932e4d30e281077f8 size: 1569
Now I can go to any other computer and pull the image. As it’s in a public repository, you don’t need any authentication to do this. You just need Docker:
$ docker pull pedroperezmsft/ubuntu-iperf3:3.0.11 3.0.11: Pulling from pedroperezmsft/ubuntu-iperf3 f6fa9a861b90: Pull complete 2d93875543ec: Pull complete 407421ef3e7e: Pull complete ea9ffec33008: Pull complete c695ce24f66e: Pull complete 0153a4f5cc7f: Pull complete Digest: sha256:2652039f39c8825a71771e2f52842c10395fad10c07254d932e4d30e281077f8 Status: Downloaded newer image for pedroperezmsft/ubuntu-iperf3:3.0.11
I have specified version 3.0.11 when pulling the image, but you can also push/pull without a tag, which will give you the “latest”:
$ docker pull pedroperezmsft/ubuntu-iperf3 Using default tag: latest latest: Pulling from pedroperezmsft/ubuntu-iperf3 f6fa9a861b90: Pull complete 2d93875543ec: Pull complete 407421ef3e7e: Pull complete ea9ffec33008: Pull complete c695ce24f66e: Pull complete 0153a4f5cc7f: Pull complete Digest: sha256:2652039f39c8825a71771e2f52842c10395fad10c07254d932e4d30e281077f8 Status: Downloaded newer image for pedroperezmsft/ubuntu-iperf3:latest
As you can see, the hash is the same. That’s because version 3.0.11 of my container is actually also the latest.
You can now find the container image here: https://hub.docker.com/r/pedroperezmsft/ubuntu-iperf3/
Closing notes
In this article, we have learned how to build a container from an already existing image by using a Dockerfile and a local development environment. We have also learned how to make the container available to download from anywhere by anyone.
This is just the start of your containerization journey. I would recommend as next steps to find out how to make smaller containers and find out how to containerize applications that are not so easy to install inside the container (e.g. not available as a deb/rpm package and/or needing compilation).
As a curiosity for the reader, I’ll share here the Dockerfile for my ntttcp-linux container https://hub.docker.com/r/pedroperezmsft/ntttcp-linux/
FROM gliderlabs/alpine:3.6 LABEL maintainer="pedro.perez@microsoft.com" RUN apk add --update openssl \ && wget https://github.com/PedroPerezMSFT/ntttcp-container/blob/master/binaries/ntttcp-musl-1.3.0?raw=true \ -O /run/ntttcp \ && chmod +x /run/ntttcp ENTRYPOINT ["/run/ntttcp"]
You’ll see I have used a different base image and my steps are different to the ones we have followed. The main challenge to containerize ntttcp was that the application is not available as a package, so I had to compile it myself inside the base container, then extract the binary and make it available in a publicly reachable URL for it to be pulled during build time.
One of the reasons I chose a different base image is because Alpine is a much smaller image, thus resulting in tiny containers compared to the one we created:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE pedroperezmsft/ubuntu-iperf3 latest 853b98f9719f 37 minutes ago 161.7 MB pedroperezmsft/ntttcp-linux latest 9790c7acb7b1 26 hours ago 8.612 MB
That’s a container 20x smaller, which means it deploys faster. It also helps with container density as you should be able to deploy more containers on the same host if disk space is a constraint.
My containers
You can find all my public containers in Docker Hub’s registry in my repo: https://hub.docker.com/u/pedroperezmsft/
Some of my containers download pre-compiled binaries from my Github repos. So far here are the two containers that do it:
One thought on “Containerize your tools!”