Goals & Motivation
I recently launched an app called Fakes.io where users can create realistic AI-generated photos of people, along with fake information about the person. The app has a simple & easy to use API endpoint that allows programmers to generate images using their favourite programming language.
While building the app, I had a few design considerations in mind, namely:
- How can I run the app as cheaply as possible?
- Am I able to scale out (or in) depending on the amount of traffic?
- Will the API backend have self healing capability so it stays up while I am off on a 7 day hike in the bush?
Naturally a Kubernetes cluster will be ideal for this kind of unpredictable workload while maintaining the highest utilisation efficiency.
Traditionally, managed Kubernetes cluster offerings are thought to be expensive — but it doesn’t have to be.
As of December 2020, I have found a way to run a fully managed & highly scalable Kubernetes cluster for as cheap as $1 a day. I am confident that this is the cheapest Kubernetes cluster in existence as of writing.
At $1 a day, I am confident that this is the cheapest Kubernetes cluster in existence as of writing.
I write this with all future developers and startups in mind. No longer you have to worry about over-provisioning your resources for a hobbyist project, or risk the hug of death when your app made the front page of reddit.
We will walk through the theory and pricing comparison for each cloud provider below. If you want to skip straight to the step-by-step tutorial in Azure, click here.
Readers are assumed that they have basic understanding of Kubernetes.
Kubernetes Cluster Provider Comparison
A fully running Kubernetes cluster consist chiefly of the following physical resources:
- Control Plane
- Compute Instance
- Load Balancer
- Disk Storage
- Container Registry
The resources below are priced in USD and are in the US region.
As of writing, the most common fully managed Kubernetes cluster providers are:
Right off the bat we can see that Azure AKS sets itself apart from the rest of the pack by offering managed Kubernetes service for free.
Note that this may change in the future, as it did for Google GCP GKE on 5th March 2020 where Google switched from Free to $0.10 per hour.
Compute instances usually come in 3 flavours:
- On Demand (most expensive)
- Reserved (less expensive)
- Spot (least expensive)
Spot pricing will provide the cheapest way to run compute instance (up to 90% savings) and that’s what we will go with.
Most Kubernetes clusters demand at least 3.5 GB of RAM. The following are some of the examples of the cheapest compute instances made available.
I highly recommend Azureprice.net for Azure Price comparison.
I recommend everyone to do their own research to find which instance is suitable for your workload.
One of the disadvantages of spot instances is that if the spot price is above your requested price, the provider will shut down your compute instance to take it back. To avoid this, just set the request price ceiling to the same as on-demand price.
All instances required a block storage attached to the compute instance. Azure has a minimum requirement of 32 GiB storage to start a cluster, so that’s the configuration we will go with.
We can see that Microsoft Azure has the cheapest block storage offering. You can get it cheaper by switching to HDD, sacrificing performance.
All managed Kubernetes clusters needed at least one network load balancer attached to the virtual network.
Azure has a free load balancer offering called
Basic but unfortunately that offering does not work with a node pool with spot instances. Source
It is possible that if you go with Reserved Instances and switch over to
Basic offering, it will work out to be cheaper (depending on your configuration). Note that you cannot change your Load Balancer type after you created it.
Bandwidth is the data transfer in/out of the virtual network. Typically the cloud provider charge per GB usage.
Bandwidth-wise Azure’s offering is cheaper by 37.5%. Remember that every penny counts!
Static IP Address
Azure has an extra cost where the load balancer requires static IP addresses to work. In my configuration I found that AKS requires 2 static IP address, one upon creation, one created by the kubernetes pod.
Container registry is where you store your docker images. While offerings like DockerHub provides free unlimited public repositories, most people would want to keep their images private.
All 3 cloud providers have private container registry offerings, but unfortunately things start to get expensive here as the providers start charging per day or per GB stored, or both!
This is where third party container registries really shines. For example DockerHub provides unlimited private repo for $5/month, and GitLab has the better offering of… free 🙃
GitLab’s offering ties to a project code repository. So if have multiple repository, you just need to create multiple “projects” to take advantage of its free offering.
Tally everything up:
Microsoft Azure AKS is the clear winner in this scenario, beating all other major cloud providers by offering the cheapest, fully managed, highly scalable Kubernetes Cluster at a rock bottom price of $1.28 a day
Here’s a snapshot of Fakes.io running cost for a week in November 2020.
In Part 2, we will go through the step by step tutorial on how to create the cheapest AKS cluster, possibly in existence.