It seems that the term ‘GitOps’ has gradually crept into the consciousness and vernacular of anyone associated with the cloud-native approach to software delivery. Over the course of a set of articles, I’ll be putting this new approach to managing software applications and infrastructure under the microscope. And, to get things started, I want to pose two questions; what exactly is GitOps, and where did it come from? Before we tackle the first question, let’s try and answer the second.
Where did the term 'GitOps' come from?
It’s generally accepted that the term 'GitOps' was coined by Alexis Richardson, the CEO of Weaveworks, in a blog post in 2017. The post describes how his company adopted a developer-oriented approach to operating Kubernetes clusters, using Git as a single source of truth for declarative configuration. A lot of water has passed under the GitOps bridge since that moment in time, and these days everyone seems to have a take on GitOps. This often involves bending the description or purpose of an existing tool, project, or product, to express it in GitOps lingua franca. This doesn’t help, of course, because we end up with a multitude of different, conflicting definitions, which only serves to muddy the waters.
But, what is it?
So, that leads us to the second question; what is GitOps? In an attempt to inject some formalism into the confusing picture, a number of parties with a vested interest initiated the GitOps Working Group (GWG), whose aim is to "define a vendor-neutral, principle-led meaning of (the term) GitOps". The purpose of the GWG has gained significant interest from companies large and small, including the likes of AWS, Alibaba, IBM, Microsoft, Red Hat, and VMWare. The GWG is also a sub-group of the CNCF Application Delivery Technical Advisory Group, so it holds some credence if you’re bought into the purpose of the Cloud Native Computing Foundation (CNCF).
Part of the work of the GWG is to define a set of principles that describe a system operated using a GitOps approach. It’s a work in progress, but there are essentially four key principles:
- Desired state is expressed declaratively — the desired state of the system under consideration must be expressed declaratively in a form that is machine and human readable and writeable. For anyone familiar with Kubernetes, this is familiar territory, as the principle echoes those required to define application workload resources as the ‘desired state’, using YAML or its subset, JSON.
- Desired state is stored as immutable versions — the declarative expressions of desired state must be stored as immutable versions, with one version easily identifiable from another, each of which contributes to a history. This is where the Git part of GitOps comes into play, and whilst there is no explicit requirement for the storage system to be Git-based (or even VCS-based), practically speaking it seems Git is all pervasive in the modern era. That said, one popular GitOps solution, Flux (which we’ll cover in a later article), supports the use of S3-compatible buckets for desired state storage.
- Actual state is continuously reconciled with the desired state — a system managed according to GitOps principles will have its actual state reconciled with the expressed desired state on a continuous basis. Again, for Kubernetes aficionados, this is not an alien concept, as this is the basis of the Kubernetes controller model. This GitOps principle states that software agents are responsible for initiating automated actions to converge the actual state with the desired state, irrespective of whether the deviance is as a result of version change, or due to configuration drift (untracked change).
- The system is operated according to the principles — the final principle states that the operation of the system is performed according to the principles. That is, change is applied through new versions of the declarative expression of the desired state and automatically reconciled by a software agent. The key aspect of this principle is that changes should not be conducted by humans or machines directly on the system, or outside of the scope of the versioned, declarative definitions of the desired state.
That’s a whole lot of theory to digest, but how do these principles help in the delivery of cloud-native infrastructure and software applications?
How does GitOps help?
The use of a version control system (VCS), such as Git, for managing the code of software applications we create, is deemed a prerequisite for modern continuous integration and delivery (CI/CD) pipelines. But, what about the configuration that describes how those applications are deployed to a Kubernetes cluster? If we’re forward-thinking enough, we might store this declarative configuration in Git also. Often, however, it is stored in a multitude of different places, duplicated, with no clear ownership, and no means for providing a common view of the world. Which version of the app’s image should be deployed in production? What is the URI for the database used in staging? Even if the configuration is managed with Git, its efficacy is diminished as soon as someone makes an imperative change on the live cluster. Is the configuration represented by the live cluster the source of truth, or is that reflected in the Git repo?
It’s these difficult problems of; managing state, the processes that impinge on the state, and the actions of the various actors with a vested interest in the system, that GitOps tries to resolve. In an ideal GitOps world, nobody would have direct access to a cluster; all change would be committed to a Git repo, it’s content automatically retrieved by a software agent, and then applied to the cluster to achieve the desired state.
We also get fine-grained control over change using the inherent approval features of Git-based providers, such as GitHub, GitLab, and the equivalent offerings from the public cloud providers (for example, AWS CodeCommit). Whilst it’s a challenge to work out how to set up Git repos to suit the way that teams work and interact, the inherent features provide control points (peer review, pull/merge request) and a full audit trail of change and the actors involved. The former ensures no ad-hoc change is applied to the system, whilst the latter provides detail about the reason and content of any change. As a by-product, changes that fail can be reverted with relative ease, and systems can be re-established from the source of truth in the event of a disaster.
Summary
There’s much more to endear us to the GitOps pattern; too much to cover in a single blog post. But, in summary, GitOps is a discipline that has a way to go before it reaches relative maturity. It’s developing at speed, fuelled by the explosive growth of containerized workloads and Kubernetes adoption. Whether it’s creating Kubernetes clusters using the Cluster API or managing the deployment of software applications to existing clusters, GitOps promises to bring extra rigor to the process. In the coming articles, we’ll be taking a close look at some of the tools that are driving the interest in, and early adoption of, GitOps principles. Hold on tight!
