Containerize your tools!

How many times have you asked a customer to run a test with some specific tools, just to have to fight countless compatibility problems due to the particularities of the customer’s environment?

Yeah, I’ve been there too.

Containers should be designed to be immutable and run in the same way any time you deploy them, regardless of the environment.

This article will give you some guidelines to containerize any application. Once your application runs in a container, you don’t have to care anymore about compatibility issues, libraries or DLL not available, etc

The containerization steps

The process takes just a few steps, which I’ll briefly describe in this section.

Step 1: Get your container development environment ready!

Step 2: Choose a base image for your container.

Step 3: Find out how to run your tool inside a container

Step 4: Convert the previous steps into a Dockerfile.

Step 5: Test your container image.

Step 6: Push your container image to a repository.

Setting up your container development environment

I recommend downloading and installing Docker for Windows, which is free: https://www.docker.com/docker-windows

Depending on which OS does your tool run, you have to choose to run Windows Containers or Linux Containers (you can switch between both at any time):

selectwincontainers

In this guide we will show you how to containerize a Linux tool.

Now run docker run hello-world to check that Docker has been correctly installed:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
9a0669468bf7: Pull complete
Digest: sha256:0e06ef5e1945a718b02a8c319e15bae44f47039005530bc617a5d071190ed3fc
Status: Downloaded newer image for hello-world:latest


Hello from Docker!

This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:

1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
3. The Docker daemon created a new container from that image which runs the    executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.

(...)Output trimmed for brevity(...)

Docker will download and run a simple container that shows the above message.

Choosing a base image for your container

You can create containers from scratch, but that’s out of scope for this guide. In our case we will choose a base image and build functionality on top of it.

A base image could be just a container running e.g. full Ubuntu:

$ docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
ae79f2514705: Pull complete
c59d01a7e4ca: Pull complete
41ba73a9054d: Pull complete
f1bbfd495cc1: Pull complete
0c346f7223e2: Pull complete
Digest: sha256:6eb24585b1b2e7402600450d289ea0fd195cfb76893032bbbb3943e041ec8a65
Status: Downloaded newer image for ubuntu:latest


root@a3e5e569e2ab:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"

The above downloads and runs an Ubuntu 16.04.3 LTS container, executing the command “bash” on start, which will give us an interactive shell (-it). It took about 15 seconds to download and started immediately on my laptop.

This is the size of the Ubuntu container we’re running:

$ docker images
REPOSITORY   TAG                 IMAGE ID            CREATED             SIZE
Ubuntu    latest              dd6f76d9cc90        6 days ago            122MB

There are much smaller alternatives, but for the sake of simplicity we will use this container as a base image for our purposes.

Find out how to run your tool inside a container

The next step we need to do is to test our tool inside a container. For that purpose, we will just run the Ubuntu container as we did on the previous step and once we’re in, we will install and run our tool.

I have chosen to containerize iPerf3 as it’s a tool we all use often. What I’ll do now is:

  • Get into an Ubuntu container
  • Download and install iPerf
  • Test it
$ docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
ae79f2514705: Pull complete
c59d01a7e4ca: Pull complete
41ba73a9054d: Pull complete
f1bbfd495cc1: Pull complete
0c346f7223e2: Pull complete
Digest: sha256:6eb24585b1b2e7402600450d289ea0fd195cfb76893032bbbb3943e041ec8a65
Status: Downloaded newer image for ubuntu:latest

root@34e7118c6bb1:/# apt update
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
(...)Output trimmed for brevity(...)

root@34e7118c6bb1:/# apt install iperf3
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libiperf0

The following NEW packages will be installed:
  iperf3 libiperf0

0 upgraded, 2 newly installed, 0 to remove and 4 not upgraded.

Need to get 58.5 kB of archives.

After this operation, 238 kB of additional disk space will be used.

Do you want to continue? [Y/n]

Get:1 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libiperf0 amd64 3.0.11-1 [50.4 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial/universe amd64 iperf3 amd64 3.0.11-1 [8090 B]

(...)Output trimmed for brevity(...)
Setting up iperf3 (3.0.11-1) ...

I have an iPerf3 server listening on another machine on IP address 10.1.4.5, so I’ll just try to run a test against it:

root@34e7118c6bb1:/# iperf3 -c 10.1.4.5
Connecting to host 10.1.4.5, port 5201
[  4] local 172.17.0.3 port 47296 connected to 10.1.4.5 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   124 MBytes  1.04 Gbits/sec  1513    233 KBytes
[  4]   1.00-2.00   sec   116 MBytes   972 Mbits/sec  1742    147 KBytes

(...)Output trimmed for brevity(...)

[  4]   9.00-10.00  sec   116 MBytes   976 Mbits/sec  1497    129 KBytes

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.15 GBytes   987 Mbits/sec  14378             sender
[  4]   0.00-10.00  sec  1.15 GBytes   985 Mbits/sec                  receiver

iperf Done.

YAY!

Note: Outbound traffic will be translated to the host’s IP address, but inbound traffic needs to be explicitly permitted.

Convert the previous steps into a Dockerfile

A Dockerfile is the file that defines how Docker should build a container.

The first thing we need to specify in our Dockerfile is the base image:

FROM ubuntu:latest

The above specifies we should use the image ubuntu on its latest version. Whatever comes after the colon is called “tag” and it’s used to define specific versions.

This simplifies things for us on building time, but it means that every time someone builds our container they will be pulling the latest ubuntu version (which will be changing overtime!). These are potentially breaking changes, so my recommendation would be to choose a specific version. Let’s change it as follows:

FROM ubuntu:16.04

16.04 is a valid tag as per https://hub.docker.com/_/ubuntu/ and it’s the current stable version.

Now let’s make sure users can find and contact us if they have any issues running our container:

LABEL maintainer="pedro.perez@microsoft.com"

(and for vanity Internet points 😊)

Finally, time to get some action. We will instruct Docker to run the commands we also ran to download and install iPerf3. If you scroll above you’ll find we first had to update apt repositories (you need to do this as the local package lists inside the container are empty by default) and then proceeded to install iPerf3.

RUN apt update \
  && apt install iperf3 -y

Note: We specified -y to force YES to any question as you won’t be able to interact with your container during build time.

And the last step would be to instruct Docker what command should it ran by default:

ENTRYPOINT ["iperf3"]

Save this all in a file called “Dockerfile” on your local computer:

FROM ubuntu:16.04
LABEL maintainer="pedro.perez@microsoft.com"

RUN apt update \
  && apt install iperf3

ENTRYPOINT ["iperf3"]

Note: If you use Visual Studio Code, it supports Dockerfiles and will help you with inline help and spotting any issues.

Build your container image using your Dockerfile

We’re nearly there! Our next step is to get Docker to build a container following the instructions on our Dockerfile. The command we need is docker build -t repository/container-name and specify the path to our Dockerfile. In our case it will be the current folder, hence will just write a dot “.”

The repository in my case is my username in Docker Hub. The container name could be anything you’d like. It is recommended to make it self-explanatory. I have chosen “ubuntu-iperf3

$ docker -t pedroperezmsft/ubuntu-iperf3 build .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM ubuntu:16.04
 ---> dd6f76d9cc90
Step 2 : LABEL maintainer "pedro.perez@microsoft.com"
 ---> Using cache
 ---> abb0c99425b4
Step 3 : RUN apt update   && apt install iperf3 -y
 ---> Running in 6f436c32e642

Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]

(...)Output trimmed for brevity(...)
Setting up iperf3 (3.0.11-1) ...

 ---> 1ad73703c4d5

Removing intermediate container 6f436c32e642
Step 4 : ENTRYPOINT iperf3
 ---> Running in 98995dca9ade
 ---> 853b98f9719f
Removing intermediate container 98995dca9ade

Successfully built 853b98f9719f

Let’s check the image has been created:

$ docker images
REPOSITORY                     TAG     IMAGE ID            CREATED             SIZE
pedroperezmsft/ubuntu-iperf3   latest  853b98f9719f        5 minutes ago       161.7 MB

Test your container image

Now it’s time to deploy a container from our container image above:

$ docker run pedroperezmsft/ubuntu-iperf3 -c 10.1.4.5
Connecting to host 10.1.4.5, port 5201
[  4] local 172.17.0.3 port 50084 connected to 10.1.4.5 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   123 MBytes  1.03 Gbits/sec  2390    183 KBytes
[  4]   1.00-2.00   sec   116 MBytes   971 Mbits/sec  2055    287 KBytes

(...)Output trimmed for brevity(...)

[  4]   9.00-10.00  sec   118 MBytes   990 Mbits/sec  1367    261 KBytes

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.15 GBytes   987 Mbits/sec  15077             sender
[  4]   0.00-10.00  sec  1.15 GBytes   984 Mbits/sec                  receiver

iperf Done.

Yay again!

We have a working container. Now the next step would be to test it as a server. As I’ve mentioned previously, inbound traffic to our container must be explicitly permitted. We do this by mapping external (host) ports to internal (container) ports. In our case we will be mapping external port 5201 to internal port 5201 when starting the iPerf3 server:

$ docker run -p 5201:5201 pedroperezmsft/ubuntu-iperf3 -s

Note: We can pass parameters to iperf3 just by adding them at the end of docker command.

Now I’ll start a client from a remote machine and point it to our container. The IP address I should point it to is the one from the host (10.1.4.4) because that’s where we mapped the port. This is an analogous situation to a PAT.

$ iperf3 -c 10.1.4.4
Connecting to host 10.1.4.4, port 5201
[  4] local 10.1.4.5 port 44978 connected to 10.1.4.4 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   125 MBytes  1.05 Gbits/sec   97    693 KBytes
[  4]   1.00-2.00   sec   119 MBytes   995 Mbits/sec  191    761 KBytes
[  4]   2.00-3.00   sec   119 MBytes  1.00 Gbits/sec  196    827 KBytes

(...)Output trimmed for brevity(...)

[  4]   9.00-10.00  sec   118 MBytes   993 Mbits/sec  547   1.04 MBytes

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.17 GBytes  1.00 Gbits/sec  2181             sender
[  4]   0.00-10.00  sec  1.16 GBytes  1.00 Gbits/sec                  receiver

iperf Done.

Yay once more!

Push your container image to a repository

Let’s recap for a second:

You have successfully built a container image that runs iPerf3 and you have successfully tested it for both inbound and outbound traffic. Now the next step is to make this image available to deploy on other machines. Otherwise this would happen if you try to deploy it on another computer:

$ docker run -p 5201:5201 pedroperezmsft/ubuntu-iperf3 -s
Unable to find image 'pedroperezmsft/ubuntu-iperf3:latest' locally
Pulling repository docker.io/pedroperezmsft/ubuntu-iperf3
docker: Error: image pedroperezmsft/ubuntu-iperf3:latest not found.

The image does not exist locally, but it doesn’t exist in Docker Hub either. Let’s fix that!

Note: This step implies you have an account created in Docker Hub https://hub.docker.com/ if you don’t, go ahead now and do it.

We are going to push our image to Docker Hub, inside our own repository. As previously mentioned, our repository’s name is pedroperezmsft – please change this for your own as created in Docker Hub.

Just before pushing I’d like to rebuild our container to specify a tag. As we didn’t specify one before, Docker assigned “latest” as the tag. Now I’d like to make sure I specify iPerf3’s version on the tag:

$ docker build -t pedroperezmsft/ubuntu-iperf3:3.0.11 ./container/
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM ubuntu:16.04
 ---> dd6f76d9cc90
Step 2 : LABEL maintainer "pedro.perez@microsoft.com"
 ---> Using cache
 ---> abb0c99425b4
Step 3 : RUN apt update   && apt install iperf3 -y
 ---> Using cache
 ---> 1ad73703c4d5
Step 4 : ENTRYPOINT iperf3
 ---> Using cache
 ---> 853b98f9719f

Successfully built 853b98f9719f


$ docker images
REPOSITORY                     TAG      IMAGE ID            CREATED             SIZE
pedroperezmsft/ubuntu-iperf3   3.0.11   853b98f9719f        22 minutes ago      161.7 MB

That was easy.

Let’s now push our image:

$ docker push pedroperezmsft/ubuntu-iperf3:3.0.11

The push refers to a repository [docker.io/pedroperezmsft/ubuntu-iperf3]
7aec54326e0e: Preparing
174a611570d4: Preparing
f51f76255b02: Preparing
51db18d04d72: Preparing
f1c896f31e49: Preparing
0f5ff0cf6a1c: Waiting

unauthorized: authentication required

Oops!

It seems we first need to authenticate against Docker Hub with the credentials we have chosen when we signed up on the website:

$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: pedroperezmsft
Password:

Login Succeeded

Now we’re authenticated. Let’s try to push the image again to the repository:

$ docker push pedroperezmsft/ubuntu-iperf3:3.0.11
The push refers to a repository [docker.io/pedroperezmsft/ubuntu-iperf3]
7aec54326e0e: Pushed
174a611570d4: Pushed
f51f76255b02: Pushed
51db18d04d72: Pushed
f1c896f31e49: Pushed
0f5ff0cf6a1c: Pushed

3.0.11: digest: sha256:2652039f39c8825a71771e2f52842c10395fad10c07254d932e4d30e281077f8 size: 1569

Now I can go to any other computer and pull the image. As it’s in a public repository, you don’t need any authentication to do this. You just need Docker:

$ docker pull pedroperezmsft/ubuntu-iperf3:3.0.11
3.0.11: Pulling from pedroperezmsft/ubuntu-iperf3
f6fa9a861b90: Pull complete
2d93875543ec: Pull complete
407421ef3e7e: Pull complete
ea9ffec33008: Pull complete
c695ce24f66e: Pull complete
0153a4f5cc7f: Pull complete

Digest: sha256:2652039f39c8825a71771e2f52842c10395fad10c07254d932e4d30e281077f8

Status: Downloaded newer image for pedroperezmsft/ubuntu-iperf3:3.0.11

I have specified version 3.0.11 when pulling the image, but you can also push/pull without a tag, which will give you the “latest”:

$ docker pull pedroperezmsft/ubuntu-iperf3
Using default tag: latest
latest: Pulling from pedroperezmsft/ubuntu-iperf3
f6fa9a861b90: Pull complete
2d93875543ec: Pull complete
407421ef3e7e: Pull complete
ea9ffec33008: Pull complete
c695ce24f66e: Pull complete
0153a4f5cc7f: Pull complete

Digest: sha256:2652039f39c8825a71771e2f52842c10395fad10c07254d932e4d30e281077f8

Status: Downloaded newer image for pedroperezmsft/ubuntu-iperf3:latest

As you can see, the hash is the same. That’s because version 3.0.11 of my container is actually also the latest.

You can now find the container image here: https://hub.docker.com/r/pedroperezmsft/ubuntu-iperf3/

Closing notes

In this article, we have learned how to build a container from an already existing image by using a Dockerfile and a local development environment. We have also learned how to make the container available to download from anywhere by anyone.

This is just the start of your containerization journey. I would recommend as next steps to find out how to make smaller containers and find out how to containerize applications that are not so easy to install inside the container (e.g. not available as a deb/rpm package and/or needing compilation).

As a curiosity for the reader, I’ll share here the Dockerfile for my ntttcp-linux container https://hub.docker.com/r/pedroperezmsft/ntttcp-linux/

FROM gliderlabs/alpine:3.6
LABEL maintainer="pedro.perez@microsoft.com"

RUN apk add --update openssl \
  && wget https://github.com/PedroPerezMSFT/ntttcp-container/blob/master/binaries/ntttcp-musl-1.3.0?raw=true \
    -O /run/ntttcp \
  && chmod +x /run/ntttcp

ENTRYPOINT ["/run/ntttcp"]

You’ll see I have used a different base image and my steps are different to the ones we have followed. The main challenge to containerize ntttcp was that the application is not available as a package, so I had to compile it myself inside the base container, then extract the binary and make it available in a publicly reachable URL for it to be pulled during build time.

One of the reasons I chose a different base image is because Alpine is a much smaller image, thus resulting in tiny containers compared to the one we created:

$ docker images
REPOSITORY                     TAG         IMAGE ID      CREATED             SIZE
pedroperezmsft/ubuntu-iperf3   latest      853b98f9719f  37 minutes ago      161.7 MB
pedroperezmsft/ntttcp-linux    latest      9790c7acb7b1  26 hours ago        8.612 MB

That’s a container 20x smaller, which means it deploys faster. It also helps with container density as you should be able to deploy more containers on the same host if disk space is a constraint.

My containers

You can find all my public containers in Docker Hub’s registry in my repo: https://hub.docker.com/u/pedroperezmsft/

Some of my containers download pre-compiled binaries from my Github repos. So far here are the two containers that do it:

https://github.com/PedroPerezMSFT/qperf-container

https://github.com/PedroPerezMSFT/ntttcp-container

One thought on “Containerize your tools!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s