GitOps with Jenkins X

In this penultimate article in our GitOps series, we're taking a look at another heavyweight solution to the automated continuous deployment quest: Jenkins X.

Jenkins X shares some commonalities with its namesake, Jenkins — it's an open source project directed at the automated delivery of software applications and hails from the same company, Cloudbees, Inc. But, that's where the similarities end. The Jenkins project (formerly known as Hudson) started its journey back in 2005, well before the cloud native revolution inspired by the containerization of software applications. The recognition that software development practices have been transformed in recent years has inspired the creation of Jenkins X. It attempts to address some of the same issues that its older relative addresses, but uses a cloud-native approach instead. And, of course, for our purposes, it also encompasses GitOps practices.

A little like the GitLab Agent for Kubernetes, the utility of Jenkins X stretches beyond just the GitOps use case. This is in contrast to the purpose of tools like ArgoCD, Flux, and Fleet, whose whole focus is on the last step in a software delivery pipeline; deployment using GitOps principles. Jenkins X allows for defining and running full CI/CD pipelines in-cluster, spinning up preview environments, and promoting application changes using GitOps automation.

Architecture

Jenkins X is essentially a collection of open source cloud native tools deployed to a Kubernetes cluster that are configured to work in unison to provide a cohesive continuous 'everything' experience. And, when I say "collection of tools", I mean a big collection of tools!

Diagram courtesy of Jenkins X project

To get all of these items to function in a coordinated fashion, Jenkins X is opinionated about how you work. For example, it assumes that CI/CD operations are performed using Tekton pipelines that are triggered using the Lighthouse webhook handler. It also assumes that application configuration is defined with Helm charts and that any secrets are managed using the Kubernetes External Secrets operator. Sure, you can change out some of these tools and replace them with something that better suits your needs. But, you'll be left to your own devices to get everything to work in step, and it kind of negates a lot of the value of the Jenkins X proposition.

Two key ingredients for GitOps in this profusion of tooling are the Git Operator and the jx  command-line interface. The former takes care of polling and cloning a Git repository for changes to applications and getting it applied to the cluster. The latter provides a means of managing the entire Jenkins X setup. We'll discuss these further in due course.

Bootstrapping

With so many things to install and configure, it would be easy to feel intimidated by the task of standing up a Jenkins X deployment. The project has this covered, anticipating this and recognizing that it could be a barrier to adoption.

Firstly, Jenkins X facilitates the establishment of the Kubernetes cluster itself and the provisioning of the various tools to the cluster. Terraform is the tool of choice for cluster creation, and modules exist for bootstrapping on Amazon EKS, GKE, and AKS. The modules use Terraform's Kubernetes provider for namespace creation and the Helm provider for installing the Git Operator.

It's also possible to bring your own cluster before bootstrapping Jenkins X into that cluster. Bootstrapping a Jenkins X environment requires a hosted Git repo from which to acquire the configuration. A bunch of pre-defined repo templates are available to get you started, which provide you with the necessary directory structure and files that Jenkins X expects. Some customization of the repo is required to reflect the specific cluster circumstances before the jx admin operator command is invoked to bootstrap the environment.

Whichever route you take to bootstrap Jenkins X, the process ends with the Git Operator deployed to a namespace called jx-git-operator, watching the Git repo that contains the configuration for the cluster. The repo has all of the configuration for each component tool in the architecture, in the form of Helm charts, under the declarative management of Helmfile. Applying a Kubernetes job located at a defined path in the repo results in the charts being run through helm template to generate the various Kubernetes resources, which are written back to the repo in a directory called config-root. These are then applied to the cluster by the Git Operator using kubectl.

$ kubectl -n jx get deployments
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
bucketrepo-bucketrepo          1/1     1            1           66m
docker-registry                1/1     1            1           66m
jx-build-controller            1/1     1            1           66m
jx-pipelines-visualizer        1/1     1            1           66m
lighthouse-foghorn             1/1     1            1           66m
lighthouse-keeper              1/1     1            1           66m
lighthouse-tekton-controller   1/1     1            1           66m
lighthouse-webhooks            1/1     1            1           66m

At the end of the bootstrap process, the various tools that make up the CI/CD environment run in a namespace called jx. The default bootstrap settings also create jx-staging and jx-production namespaces, which are intended to host application environments that correspond to their names. This setup may suffice, but it's more likely that these environments will be located on dedicated clusters, which Jenkins X can cater for.

Phew! Bootstrapping just takes care of getting Jenkins X up and running. More work is required to get the applications that are to be managed in GitOps fashion deployed to the cluster.

Workflow

To start managing applications with Jenkins X, projects can be created or imported using jx project with the quickstart or import sub-commands, respectively. The latter imports an existing project defined in a Git repo, while the former guides you through a list of language/framework options to create a new Git repo with the relevant files from a catalog. Either way, a remote Git repo is established for the application to be managed by Jenkins X.

As part of the project creation, a pull/merge request is also created on the cluster's Git repo for the inclusion of the application to the list of managed applications. Merging the request results in;

• the establishment of a CI/CD pipeline for the application,
• the registration of a webhook on the app's Git repo for triggering the pipeline,
• the creation of a Dockerfile and Helm chart for the application, amongst several other things.

Again, as part of the project creation sequence, the new pipeline is triggered, resulting in an image build and push to a registry and a new pull request on the cluster's Git repo. The new pull request seeks to promote the application to the staging environment (namespace) in the cluster. The change that brings this about is the addition of the app's Helm chart to the staging environment's helmfile, which on merge will result in rendered Kubernetes YAML added to the config-root. The changes are then applied to the cluster by the Git Operator, as it polls the cluster's repo.

Following this initial deployment, the workflow requires application changes to be made in new branches of the app's Git repo. A pull request for merging those changes triggers the Tekton pipeline, and after completion of successful testing, results in establishing a preview environment in the cluster. If all is well when testing the app in its preview environment, the change can be approved, and the pull request will be merged. In turn, this triggers the release pipeline, which includes the creation of a pull request on the cluster's Git repo for promoting the new version of the app to the staging environment. On merge, the cluster's configuration is altered to reflect the change, which is then applied to the cluster. And the cycle continues.

All of these steps are configurable, and it's possible to add or customize pipeline triggers and amend the pipelines themselves.

Conclusion

There is a ton of stuff that we don't have the space to cover! But, suffice to say, given the all-inclusive nature of Jenkins X, it provides a very compelling proposition for DevOps teams looking to establish a strategy for automated continuous deployments. In addition, having a collection of tools that are held together under an administrative umbrella provided by the project means that engineers don't have to ferret around trying to work out how to glue lots of disparate tools together.

Yet, it's opinionated. And, if you don't like the workflow or a particular tool option, and you'd prefer to deviate, then you'll lose the benefits of the crafted solution. That's not to say that it's impossible to deviate; the project even describes how to use ArgoCD with Jenkins X for the GitOps element of the workflow. But, inevitably, this will mean getting to grips with all the complexity inherent in the heterogeneous approach that Jenkins X has pursued and which has been carefully hidden through various abstractions. Ultimately, Jenkins X is ideal for teams looking for an all-in-one solution for their DevOps pipelines. But, if you're looking for a best-of-breed solution for the final step of automated continuous deployments, one of its peer solutions might be more appropriate for you.

As always, we'd love to hear your thoughts on and experiences with Jenkins X. Please do get in touch and give us your point of view!