Building Container Images with Img

Img is an open source project initiated by one of the most famous software engineers in this space, Jessie Frazelle, in response to the demand for daemonless and rootless container image building - especially in Kubernetes. Her blog post ‘Building Container Images Securely on Kubernetes’ describes the rationale for creating the Img project.

Essentially, Img is a wrapper around another open source build-related technology called BuildKit, which is embedded within Img as a library. BuildKit is an extremely powerful, multi-purpose build engine, which is part of the Moby project formulated by Docker. Before looking into some of the build features that BuildKit provides, let’s see how Img exposes those features.

Img

Img is squarely aimed at developers who are familiar with the Dockerfile as a source of build steps, and the docker build command as a means of building OCI compliant container images.

Img CLI

From a UX perspective, Img provides a set of commands which mimic equivalent Docker CLI commands that relate to container image building, distribution, and image manipulation. In addition to img build for invoking an actual image build, Img has commands for interacting with container registries; img login, img pull, img push, and so on. When it comes to manipulating images in the local cache, just as with Docker, it’s possible to list, remove, tag and save images. There are even a few handy housekeeping commands for monitoring disk usage, and for keeping on top of the build cache.

For image builds, the img build command replicates the most important sub-set of options that docker build provides, bringing a seamless experience for image authors familiar with Docker. Building an image with Img can be as simple as replacing ‘docker’ with ‘img’.

$ img build --build-arg VERSION=1.0 -t mycorp/myapp .

In this respect, Img is similar to Podman (which we discussed in a previous article), but is constrained to the task of image building, whereas Podman can be used for building images, running containers, and much more.

Build State

Container image authors that use the Docker daemon for building their images are relieved of the responsibility of determining where to store the image data locally; it’s handled by the Docker daemon. If there is no daemon to take care of storage, as is the case with Img, the responsibility falls to the image author. Img stores image data in a directory called img located under the value of the XDG_DATA_HOME variable set for the user, or $HOME/.local/share if XDG_DATA_HOME is not set, or /tmp if neither XDG_DATA_HOME or HOME variables are set. The location is configurable, but the point to remember is that if Img is run containerized, the ‘state’ directory needs to be made available to the container as a volume. By using a volume, the built image and build cache are made available for subsequent invocations of Img.

Rootless Builds

Building images as a non-privileged user is a primary goal of Img, and as such required a number of security-related patches to be made to various upstream projects. These included Docker for running rootless builds in containers, and Kubernetes for running rootless builds in pods.

Img makes use of user namespaces to implement the rootless build feature, and suffers the common limitation of the current lack of support for mounts in unprivileged user namespaces when using overlayfs (except in the Ubuntu distribution). Overlayfs is often the preferred union mount method for assembling a container’s filesystem, and without its support for unprivileged user namespaces, Img has to fall back on copying files instead, which is very inefficient and slow.

Despite this limitation, Img achieves its goal of providing rootless builds, whilst at the same time leveraging some of the advanced build features provided by BuildKit.

BuildKit

BuildKit might be described by some as the next generation build engine for container images and can be thought of as the ‘backend’ component in the build process. As we’ll find out in a future article, it is incorporated into recent versions of the Docker engine, but it’s a standalone project and can be utilised independently of Docker. Img is testament to this fact, but it’s not the only technology that leverages BuildKit. In fact, BuildKit is technology agnostic and can be used to perform build steps for any domain that can be expressed using a ‘frontend’ that converts the steps into BuildKit’s internal representation. So, what’s so special about BuildKit? There are a number of innovative features, but here are a couple that Img exploits.

Parallel Build Execution

The Moby project introduced multi-stage builds into the Docker engine in version 17.05, allowing the segregation of image builds into separate, logical stages. Multi-stage builds are a huge benefit to image authors, but the sequential nature of the execution of build steps in the legacy build engine in the Docker daemon, means that unrelated build stages can’t benefit from the parallel execution of their separate build steps. BuildKit to the rescue!

BuildKit assembles an internal representation of the build steps as a Directed Acyclic Graph (DAG), which enables it to determine which build steps can be executed in parallel. For Dockerfiles that contain numerous build stages, this can significantly reduce the length of time to build completion. This is a significant consequential advantage for Img adopters, provided without any required knowledge or configuration on the image author’s part.

Cross Platform/OS Builds

Img exposes another BuildKit feature through its user interface; cross platform and cross OS builds. Whilst there are some extra considerations to take account of when using this feature, it’s possible to build images for different combinations of architecture and OS on a completely different platform. For example, a linux/amd64 platform can be used to build an image suitable for linux/arm64. Img exposes this feature using the --platform flag.

Img and Kubernetes

The goal of building images securely on Kubernetes provided the original impetus for Img, and a container image is available for running builds in a Kubernetes pod. The image’s Dockerfile is also available for scrutiny and can be used as a basis for creating a bespoke image to account for alternative preferences or constraints.

The key ingredients provided in the image are:

• the creation of a non-privileged user for running the build,
• subordinate uid/gid files that define user namspace mappings for the non-privileged user, and
• the newuidmap and newgidmap binaries for performing the mappings for the user namespaces created each time a container is run for a build step.

There are a number of different layers at work when running rootless builds on Kubernetes with Img. Build steps are executed in BuildKit worker containers nested inside the Img container. The Img container is likely running under the Docker container runtime (or possibly, just containerd), all managed by the Kubernetes Container Runtime Interface (CRI). The task of safely removing the constraint of privileged execution down the layers is non-trivial, and it’s not surprising that some limitations remain.

At present, AppArmor and Seccomp profiles cannot be applied to Img pods for executing rootless builds, although the worker containers can still be protected using these security measures. The Img container in the pod also needs ‘unmasked’ access to its /proc filesystem, and for Kubernetes versions from 1.12 onward, this can be realised using the pod security feature securityContext.procMount. This needs to be set to Unmasked.

The final, practical requirement for using Img on Kubernetes for building container images, is for Img pods to have access to the evolving build state. This is so that separate invocations of Img can make use of the built images and cached layers in the local store. Built images may need pushing to a registry as part of a CI/CD workflow, for example, and build iterations will certainly benefit from reusing previously built layers stored in the cache. The build state can be made available to Img pods using the Kubernetes (persistent) volume abstraction.

Conclusion

It’s difficult to judge how ubiquitous Img is, or how successful it will be, as it’s not backed by a corporation with a vested interest like many open source projects. It does have an active community, however and the project has been instrumental in breaking down the barriers to running rootless containers. For that, it has much kudos.

The build engine used by Img, BuildKit, has many more interesting features that are not currently exposed by Img. If in time, these additional features make their way into Img’s user interface, it will become an even better prospect for container image building - especially for rootless builds in Kubernetes environments.

Are you looking to get your Kubernetes containers into production? Contact Giant Swarm and start your cloud-native journey