How To Run The Cheapest Kubernetes Cluster at $1 Per Day

Goals & Motivation

recently launched an app called Fakes.io where users can create realistic AI-generated photos of people, along with fake information about the person. The app has a simple & easy to use API endpoint that allows programmers to generate images using their favourite programming language.

While building the app, I had a few design considerations in mind, namely:

  • How can I run the app as cheaply as possible?
  • Am I able to scale out (or in) depending on the amount of traffic?
  • Will the API backend have self healing capability so it stays up while I am off on a 7 day hike in the bush?
Achieving Nirvana

Naturally a Kubernetes cluster will be ideal for this kind of unpredictable workload while maintaining the highest utilisation efficiency.

New to Kubernetes? Read this web comic by Google.

Traditionally, managed Kubernetes cluster offerings are thought to be expensive — but it doesn’t have to be.

Example of Kubernetes Cluster Cost — Source

As of December 2020, I have found a way to run a fully managed & highly scalable Kubernetes cluster for as cheap as $1 a day. I am confident that this is the cheapest Kubernetes cluster in existence as of writing.

At $1 a day, I am confident that this is the cheapest Kubernetes cluster in existence as of writing.

I write this with all future developers and startups in mind. No longer you have to worry about over-provisioning your resources for a hobbyist project, or risk the hug of death when your app made the front page of reddit.

We will walk through the theory and pricing comparison for each cloud provider below. If you want to skip straight to the step-by-step tutorial in Azure, click here.

Readers are assumed that they have basic understanding of Kubernetes.

Kubernetes Cluster Provider Comparison

A fully running Kubernetes cluster consist chiefly of the following physical resources:

  • Control Plane
  • Compute Instance
  • Load Balancer
  • Disk Storage
  • Container Registry

The resources below are priced in USD and are in the US region.

Control Plane

As of writing, the most common fully managed Kubernetes cluster providers are:

Source: Azure, AWS, GCP

Right off the bat we can see that Azure AKS sets itself apart from the rest of the pack by offering managed Kubernetes service for free.

Note that this may change in the future, as it did for Google GCP GKE on 5th March 2020 where Google switched from Free to $0.10 per hour.

Compute Instance

Compute instances usually come in 3 flavours:

  • On Demand (most expensive)
  • Reserved (less expensive)
  • Spot (least expensive)

Spot pricing will provide the cheapest way to run compute instance (up to 90% savings) and that’s what we will go with.

What is the heck is Spot Pricing?

Most Kubernetes clusters demand at least 3.5 GB of RAM. The following are some of the examples of the cheapest compute instances made available.

I highly recommend Azureprice.net for Azure Price comparison.

I recommend everyone to do their own research to find which instance is suitable for your workload.

One of the disadvantages of spot instances is that if the spot price is above your requested price, the provider will shut down your compute instance to take it back. To avoid this, just set the request price ceiling to the same as on-demand price.

Disk Storage

All instances required a block storage attached to the compute instance. Azure has a minimum requirement of 32 GiB storage to start a cluster, so that’s the configuration we will go with.

Source: Azure, AWS, GCP

We can see that Microsoft Azure has the cheapest block storage offering. You can get it cheaper by switching to HDD, sacrificing performance.

Load Balancer

All managed Kubernetes clusters needed at least one network load balancer attached to the virtual network.

Source: Azure, AWS, GCP

Azure has a free load balancer offering called Basic but unfortunately that offering does not work with a node pool with spot instances. Source

It is possible that if you go with Reserved Instances and switch over to Basic offering, it will work out to be cheaper (depending on your configuration). Note that you cannot change your Load Balancer type after you created it.

Bandwidth

Bandwidth is the data transfer in/out of the virtual network. Typically the cloud provider charge per GB usage.

Bandwidth-wise Azure’s offering is cheaper by 37.5%. Remember that every penny counts!

Static IP Address

Azure has an extra cost where the load balancer requires static IP addresses to work. In my configuration I found that AKS requires 2 static IP address, one upon creation, one created by the kubernetes pod.

Source: Azure

Container Registry

Container registry is where you store your docker images. While offerings like DockerHub provides free unlimited public repositories, most people would want to keep their images private.

All 3 cloud providers have private container registry offerings, but unfortunately things start to get expensive here as the providers start charging per day or per GB stored, or both!

Source: Azure, AWS, GCP, DockerHub, GitLab

This is where third party container registries really shines. For example DockerHub provides unlimited private repo for $5/month, and GitLab has the better offering of… free 🙃

GitLab’s offering ties to a project code repository. So if have multiple repository, you just need to create multiple “projects” to take advantage of its free offering.

Conclusion

Tally everything up:

Microsoft Azure AKS is the clear winner in this scenario, beating all other major cloud providers by offering the cheapest, fully managed, highly scalable Kubernetes Cluster at a rock bottom price of $1.28 a day

Here’s a snapshot of Fakes.io running cost for a week in November 2020.

Daily Cost for Fakes.io
Service Cost Breakdown for Fakes.io

In Part 2, we will go through the step by step tutorial on how to create the cheapest AKS cluster, possibly in existence.

Battle-tested Engineer. Hackathon Addict. Drone Enthusiast.