Time to Catch a New Train: Flatcar Linux

If you’ve been following developments in the container space, you probably know that CoreOS Container Linux is reaching end of life. The good news is that there’s a new (and kind of old) OS by Kinvolk called Flatcar Container Linux.

The backstory… Kinvolk has been working on projects with CoreOS for a long time and they’ve been the main contributors to rkt. So when Red Hat bought CoreOS in 2018 they decided to make sure that Container Linux will be continued. The result is Flatcar.

Some of you might know that we were early supporters of CoreOS too. Back then, their main product was their operating system. This was an obvious choice since we saw the need to have a small and up-to-date vanilla Linux on which we could run a container platform. BTW, I’ve spoken about this at multiple meetups. I even made it to the very first CoreOS fest in San Francisco. That was also where I met the Kinvolk founders. And then Kinvolk helped us create the first version of our aws-operator.

Immutable infrastructure is an important part of making your container platform scalable and reliable. So having a really small OS was, and still is, important.

This OS doesn’t even have a package management system. Everything needs to be in containers. There is no transition phase on a machine. A machine either runs version A or version B of the operating system. This is key if you want to provide your users with reliable and reproducible releases of your infrastructure product. Treat your operating system like a container image. Reduce the surface as much as possible to have a secure foundation to run the applications and services of your company.

Kinvolk took the sources of CoreOS Container Linux and created Flatcar Container Linux, mainly to provide continuity in the event Red Hat/IBM decided to drop it. The EoL of CoreOS has now been confirmed for May 26, 2020. We don't see the need to migrate to a new OS. All of the arguments about the importance of CoreOS still hold true. Flatcar is the only drop-in replacement.

There is almost no work to be done to replace it. And there isn't even an alternative. Fedora CoreOS doesn’t have a support plan yet and the list of changes one has to do is very long.

What did we need to make the switch?

The obvious thing is that you need to replace the OS images within all your clusters. Whether your bare metal servers are booted via PXE or within the cloud providers. Some images (like AWS China) weren’t there yet, so we talked to Kinvolk about how to set them up. We decided to build our own image pipeline based on the existing tooling.

Our Flatcar images are now uploaded to AWS China and are public. Feel free to use them.

On Azure, we had some trouble replacing the images in our VM scale sets. Azure prevents this because the images are coming from a different publisher. And we don’t have automation in our updates that tears everything down. We found some workarounds to make our upgrades work with a manual step in between. Ultimately, we decided that the effort for automating the step would outweigh the effort of the manual step by far. Our learning: building your own image pipeline on Azure makes sense too.

Other than that we had to swap some flags with coreos_ to flatcar_. But that is basically it. We are running tests at the moment and are close to publishing new releases of our KaaS service with Flatcar Container Linux. This change will be totally transparent to our customers.

We have an even closer relationship with Kinvolk than we had with CoreOS. This was recently made official with a sponsorship agreement. What is more important to us is to have quick and direct communication with the right people in order to get the best support possible. As a result, and an echo of the Giant Swarm support model, we now share a Slack channel with Kinvolk.

The bottom line is that switching to Flatcar is not a lot of work for us. And there are quite a few nice things coming up in Flatcar. The edge channel already previews a lot of new features coming to Flatcar that we have been waiting for. Some of us are using Wireguard already and we’re testing a new VPN setup that might replace our ipsec infrastructure in the future. Also, the alpha and edge releases now have ARM builds, which is awesome.

I hope that all of the above makes it clear that we’re proud to be the first sponsor of the Flatcar Container Linux Project. The Flatcar project will save us a lot of time and effort by not having to switch to a new type of OS. Giant Swarm is happy to back Kinvolk and help them provide this great OS.