Using Docker For Building... Code

A pilot issue of my new mid-of-the-month newsletter.

Typically, Docker is used for building container images and/or running containerized programs. However, it can easily be (ab)used as a generic build tool. Let me show you a few examples how it can come in handy.

Hacking stuff quickly

These days I use Rust only occasionally. In particular, it means there is often no configured development environment around. So, whenever there is a new PR in one of my Rust projects, here is what I do:

The above docker run command starts a temporary build environment where I can validate the changes quickly. Alternatively, if the build artifacts need to be used outside of the builder container, the command can be adjusted as follows:

Obviously, this trick works not only for Rust. Java and Go are two other great candidates, and the technique can be extended beyond compiling tiny programs. For instance, you can minify a bunch of CSS files without spoiling your system with any Node.js tools. As someone who's been advocating for disposable and isolated development environments, I find it very appealing.

The downside, of course, is the time needed for a new container to pull in and compile all the dependencies. So, if your use case is slightly less ephemeral, you should probably think of setting up a proper dev environment.

Leveraging BuildKit superpowers

Since version 18.09 (released in ~2019), Docker relies on BuildKit for building container images. However, BuildKit isn't limited to building images. It defines itself as a toolkit for converting source code to build [arbitrary] artifacts in an efficient, expressive, and repeatable manner.

Ivan Velichko @iximiuz Spot the difference 😉 $ docker build --help > Build an image from a Dockerfile $ docker buildx build --help > Start a build October 15th 2022 2 Retweets 15 Likes

With the docker buildx build command, changing the type of the build artifact becomes as simple as providing the --output flag:

By describing your build procedure in a Dockerfile, you immediately start benefiting from efficient (and potentially remote) caching, isolated build environments, out-of-the-box cross-platform builds, and more. And the build results don't need to be container images! It'll also work for producing normal local files.

Read more about the BuildKit superpowers in this highly practical article by Batuhan Apaydın.

Harnessing Bake - Docker-aware Make

It's a common practice to invoke docker build commands from a Makefile. GNU make can run independent recipes in parallel, but unlike BuildKit, it's not a container-native build system. If the build targets were described in a format that BuildKit understands, it could handle multiple concurrent build requests much more efficiently than make by de-duplicating the build steps, using more advanced caching, remote build instances, etc, etc.

And that's how the new docker buildx bake command was born.

You can think of bake as an HCL-based container-aware make:

Combining bake with the --output trick gives you an efficient, highly concurrent, and isolated (from the host system) build tool. Go give it a try!

Random (but slightly related) tweet

Look ma', no Dockerfile!

Ivan Velichko @iximiuz TIL: `docker build` can read Dockerfile from STDIN 🔥 The `docker build -` form also makes the context argument unnecessary. It's just your Dockerfile. Now I'm unstoppable! pic.twitter.com/y4amayjAbG October 16th 2022 21 Retweets 142 Likes

​

Have a productive week ahead!

Ivan