Developing for Kubernetes with Tilt

Before the developer's world got hijacked by containers and all things cloud-native, life was relatively straightforward. The inner loop, where the developer iterates over the practice of coding, building, and testing in a local environment, might have varied in its complexity according to technology. But, the complexity was manageable, and code could be pushed, reviewed, and merged before entering the outer loop of continuous integration and delivery (CI/CD).

With the trend to adopt the microservices architecture pattern, the composition of software applications is changing. Instead of a big monolith comprised of many different layers and functions, the application's logic is decomposed into small, discrete, loosely coupled services. The potential benefits of adopting this approach are many, but it also introduces a number of significant challenges — especially for the developer.

Developer Complexity

If you're developing a microservice that is part of a wider logical application entity, your inner loop activity needs to account for some additional complexity. Aside from writing code to implement the fix or features under consideration, you also have to wrestle with a number of different technologies and techniques just to stay productive.

They will include:

• Build an artefact from its source code
• Encapsulate an artefact as a container image (usually using a Dockerfile)
• Manage versions of a container image (via a container registry)
• Test locally against a representative environment (perhaps with Docker Compose)
• Generate workload abstraction configuration (YAML manifests, Kustomize overlays, Helm charts)
• Deploy the service to a local Kubernetes environment for more 'production like' testing
• Observe any telemetry instrumented in the application, and debug issues

That's a lot — I thought I was employed to write code?

It is a lot, and it's the reason why a slew of tools has emerged from the cloud-native community for developing software destined to be run on Kubernetes. Some of these tools are modest in their aims, such as copying file changes to a cluster (see Ksync), or proxying traffic between a local service and peer services running in a cluster (see Telepresence). I say 'modest', not to belittle their considerable capabilities, but in order to differentiate them from other tools that provide a much more comprehensive developer experience.

I'm going to cover some of these tools over a few different blog articles, and we'll start with Tilt.

Tilt

Tilt is an open-source tool that emerged in 2017 from a company called Windmill Engineering. It aims to provide a productive software development environment for anyone whose deployment target is Kubernetes. It doesn't take away the need to write code, of course, but it provides a framework for performing all of the other tasks that are important to a developer. And, like a lot of tooling in the cloud-native era, it uses a combination of configuration and an 'engine' to bring about desired outcomes. The outcomes might be to build a container image, render some YAML from a Helm chart, apply some YAML manifests to a Kubernetes cluster, run a local command, and a whole lot more besides. There's a lot that can be achieved with Tilt, so let's unpack it a little.

Tiltfile

Tilt uses a purpose-built CLI to get things done. But, it also needs a special file called a Tiltfile to work from in order to know what it needs to do. A Tiltfile is analogous to a Dockerfile, in that Tilt's engine consumes the Tiltfile in the same way that Docker's engine consumes a Dockerfile for a container image build. There the similarities end, though, as a Tiltfile is more expressive than a Dockerfile, and has a more general purpose in its utility.

The contents of a Tiltfile are written in Starlark, which is a dialect of Python. And, Tilt provides a number of domain-specific functions, as well as providing a mechanism for extending the available set of functions. It may seem an odd decision to foist yet another language on the overburdened cloud-native developer (to accompany Dockerfile syntax, YAML, Golang text/template, etc.), but there is a good reason. It means you can employ traditional programming constructs in your workflow; variables, loops, conditionals, and so on. If you don't need all the significant power that's at your disposal with Starlark, you can still get by with the bare minimum. The Tilt CLI is used to perform operations, with tilt up being the command to parse the Tiltfile, and then invoke the Tilt engine to act on what's defined.

A basic Tiltfile might look something like the following:

docker_build('ghcr.io/puja108/my-app', '.')
k8s_yaml('./manifests/my-app.yaml')
k8s_resource('my-app', port_forwards=8000)

The first line uses Tilt's built-in docker_build function, which implements a container image build (using Docker). The first argument will be the image's tag, and the second argument is the path to the build context. The k8s_yaml function points to a Kubernetes manifest, and here's where things get interesting; if the manifest references the same image tag, Tilt infers a correlation between the two. Tilt will apply the manifest to a configured Kubernetes cluster using the configured context. The k8s_resource function then allows us to interact with the deployed workload for testing, using a port forward.

Because of Tilt's inferred correlation between the steps defined in the Tiltfile, it bundles the associated work into a resource. And, as Tilt watches for local changes to the files that participate in this resource, it knows what actions to perform when changes occur (for example, a new image build followed by a re-deployment to Kubernetes when a file in the build context changes). This means that the effects of source code changes can be seen without having to manually rebuild an image, push it to a registry, and then redeploy a deployment to Kubernetes. Tilt takes care of it all for you, triggered by the changes you make in your development environment.

User Interface

On invoking tilt up, Tilt opens a web page that provides valuable insight for the developer. This is one of the key features that sells Tilt! It shows the steps and output generated for each resource under its control. For example, it provides the output for the container image build, the image being pushed to a registry, and then the Kubernetes events as the workload is deployed. If anything goes wrong, the user interface provides the necessary feedback for troubleshooting. This saves the developer from having to switch between different contexts to get the information they need — it's all in one place.

Live Updates

Perhaps the most interesting feature is Tilt's live update capabilities, which are expressed in a Tiltfile. It takes time to build container images, push them to registries, pull them to a Kubernetes node, and then replace an existing deployment with a new version. Instead of enduring the delay that flows from this workflow, live update allows for a container to be updated in situ, instead. So, no image build, or push, and no workload redeployment.

The Tiltfile can be used to specify which local directory paths will be synced with the running container's filesystem, so there is granular control available. Where an application service is being coded in an environment that doesn't necessitate a container restart (e.g. a Node app under Nodemon supervision), this makes life very easy. But, even if you need to compile a binary first, Tilt helps to simplify the process. A newly built binary can be copied into a container's filesystem using live update followed by a trigger action that can use the restart process extension to get the binary running in the existing container.

Getting live updates to your application service whilst developing in the inner loop is super important. Without live updates, the cycle of coding, building, and testing is protracted, to the point of becoming unworkable. Tools like Docker Compose have helped to manage this in the past. But, Tilt takes this to another level with its flexibility, its ability to incorporate deployments to Kubernetes clusters, and the valuable dashboard it provides for monitoring the health of the app you're developing.

Conclusion

Tilt has been built for a particular purpose: inner loop development. And, despite its specific focus, it has enormous flexibility. For example, you can define plain YAML, render YAML from Kustomize overlays or Helm charts, or use any other external mechanism to achieve the same thing. There are features that support team working and the developer experience, which aids the collaborative nature of the software development lifecycle.

It's safe to say that Tilt is a young project as I write, but it already has a considerable following in the wider cloud-native community. For example, it’s used by the Kubernetes Cluster API project to enable fast iteration in the development of APIs for cluster provisioning. That’s a great endorsement! Let us know if you're using Tilt to develop apps for Kubernetes, we'd love to hear from you.