How To Run The Cheapest Kubernetes Cluster at $1 Per Day
This guide has been updated on September 2021.
Goals & Motivation
I recently launched an app called Fakes.io where users can create realistic AI-generated photos of people, along with fake information about the person. The app has a simple & easy to use API endpoint that allows programmers to generate images using their favourite programming language.
While building the app, I had a few design considerations in mind, namely:
- How can I run the app as cheaply as possible?
- Am I able to scale out (or in) depending on the amount of traffic?
- Will the API backend have self healing capability so it stays up while I am off on a 7 day hike in the bush?
Naturally a Kubernetes cluster will be ideal for this kind of unpredictable workload while maintaining the highest utilisation efficiency.
New to Kubernetes? Read this web comic by Google.
Traditionally, managed Kubernetes cluster offerings are thought to be expensive — but it doesn’t have to be.
As of December 2020, I have found a way to run a fully managed & highly scalable Kubernetes cluster for as cheap as $1 a day. I am confident that this is the cheapest Kubernetes cluster in existence as of writing.
At $1 a day, I am confident that this is the cheapest Kubernetes cluster in existence as of writing.
I write this with all future developers and startups in mind. No longer you have to worry about over-provisioning your resources for a hobbyist project, or risk the hug of death when your app made the front page of reddit.
We will walk through the theory and pricing comparison for each cloud provider below. If you want to skip straight to the step-by-step tutorial in Azure, click here.
Readers are assumed that they have basic understanding of Kubernetes.
Kubernetes Cluster Provider Comparison
A fully running Kubernetes cluster consist chiefly of the following physical resources:
- Control Plane
- Compute Instance
- Load Balancer
- Disk Storage
- Container Registry
The resources below are priced in USD and are in the US region.
Control Plane
As of writing, the most common fully managed Kubernetes cluster providers are:
Right off the bat we can see that Azure AKS sets itself apart from the rest of the pack by offering managed Kubernetes service for free.
Note that this may change in the future, as it did for Google GCP GKE on 5th March 2020 where Google switched from Free to $0.10 per hour.
Compute Instance
Compute instances usually come in 3 flavours:
- On Demand (most expensive)
- Reserved (less expensive)
- Spot (least expensive)
Spot pricing will provide the cheapest way to run compute instance (up to 90% savings) and that’s what we will go with.
What is the heck is Spot Pricing?
The caveat is you must run a non-spot node in the system pool for Azure. The cheapest VM currently that meets the requirement is Azure’s Standard_B2S at US$1/day.
Most Kubernetes clusters demand at least 3.5 GB of RAM. The following are some of the examples of the cheapest compute instances made available.
I highly recommend Azureprice.net for Azure Price comparison.
I recommend everyone to do their own research to find which instance is suitable for your workload.
One of the disadvantages of spot instances is that if the spot price is above your requested price, the provider will shut down your compute instance to take it back. To avoid this, just set the request price ceiling to the same as on-demand price.
Disk Storage
All instances required a block storage attached to the compute instance. Azure has a minimum requirement of 32 GiB storage to start a cluster, so that’s the configuration we will go with.
Load Balancer
All managed Kubernetes clusters needed at least one network load balancer attached to the virtual network.
Azure has a free load balancer offering called Basic but unfortunately that offering does not work with a node pool with spot instances. Source
It is possible that if you go with Reserved Instances and switch over to Basic offering, it will work out to be cheaper (depending on your configuration). Note that you cannot change your Load Balancer type after you created it.
Bandwidth
Bandwidth is the data transfer in/out of the virtual network. Typically the cloud provider charge per GB usage.
Bandwidth-wise Azure’s offering is cheaper by 37.5%. Remember that every penny counts!
Static IP Address
Azure requires a minimum of one public ip network attached to the load balancer. You may switch to Basic SKU to reduce pricing.
Source: Azure
Container Registry
Container registry is where you store your docker images. While offerings like DockerHub provides free unlimited public repositories, most people would want to keep their images private.
All 3 cloud providers have private container registry offerings, but unfortunately things start to get expensive here as the providers start charging per day or per GB stored, or both!
Source: Azure, AWS, GCP, DockerHub, GitLab
This is where third party container registries really shines. For example DockerHub provides unlimited private repo for $5/month, and GitLab has the better offering of… free 🙃
GitLab’s offering ties to a project code repository. So if have multiple repository, you just need to create multiple “projects” to take advantage of its free offering.
Conclusion
Tally everything up:
Microsoft Azure AKS (Basic) is the clear winner in this scenario, beating all other major cloud providers by offering the cheapest, fully managed, highly scalable Kubernetes Cluster at a rock bottom price of $1.25 a day
If you wish to use a spot node pool, you will need to switch to Microsoft Azure AKS (Standard), which will run you $1.88 a day (before adding spot instances)
Here’s a snapshot of Fakes.io running cost for a week in November 2020.
In Part 2, we will go through the step by step tutorial on how to create the cheapest AKS cluster, possibly in existence.
