Dynamically resizing a cluster with the cluster autoscaler

Last updated July 27th, 2021.

Objective

OVHcloud Managed Kubernetes service provides you Kubernetes clusters without the hassle of installing or operating them.

During the day-to-day life of your cluster, you may want to dynamically adjust the size of your cluster to accommodate to your workloads. The cluster autoscaler simplifies the task by scaling up or down your OVHcloud Managed Kubernetes cluster to meet the demand of your workloads.

This guide will cover how to enable and manage the cluster autoscaler on your OVHcloud Managed Kubernetes cluster.

Requirements

Instructions

The cluster autoscaler

Horizontal pod scaling and cluster scaling

One of the hallmarks of cloud native environments is that they must automatically adapt to fluctuating loads. These fluctuations can be for the most part regular, such as day-night or work-weekend cycles, or more ad hoc, such as a Black Friday situation. A well-designed cloud native system must scale up and down to adapt to fluctuations in load without human intervention.

Kubernetes has a powerful set of features to accommodate to shifting workloads, the horizontal pod autoscaling.

The Horizontal Pod Autoscaler automatically scales the number of Pods in a replication controller, deployment, replica set or stateful set based on observed CPU utilization (or, with custom metrics support, on some other application-provided metrics).

But the horizontal pod autoscaling has a hard limit, the number of available nodes in your Kubernetes cluster. It can only add pods to a Replica Set if there is enough available resources in your Kubernetes nodes, and it cannot reduce the size of your cluster if your nodes are underutilized.

In order to go further with the scalability, you need to have a cluster autoscaler. The Cluster autoscaler adds or removes nodes in a cluster based on all pods' requested resources, without human intervention.

The horizontal pod autoscaler and the cluster autoscaler work better when deployed together in the cluster, because the horizontal pod autoscaler focuses on managing pods to allow them to match the demand while the autoscaler manages the number of nodes in the cluster to be sure that the horizontal pod autoscaler can do its role.

The cluster and horizontal pod autoscalers can work together, and are often both deployed in a cluster. When combined, the horizontal pod autoscaler is focused on running the number of pods required to meet application demand. The cluster autoscaler is focused on running the number of nodes required to support the scheduled pods.

Scaling up and down

To scale-up a cluster, the cluster autoscale watches the load on your nodes and detects when your cluster has resource constraints (e.g. it cannot schedule nodes because of lack of resources). When such a situation arise, the cluster autoscaler adds nodes to your node pools to match the demand.

The cluster autoscaler also works the other way around, monitoring underutilized nodes and decreasing the number of nodes, helping to reduce your costs.

The scaling up is conceptually easier than the scaling down. When the autoscaler detects resource constraints, it simply needs to add yet another node (providing it's under the user defined limit of nodes). But in order to scale down, even if most nodes are underutilized, it needs to move all the pods outside of one of the nodes before deleting it.

Some considerations like pods using node selectors, anti-affinity, or restrictive PodDisruptionBudget (PDB).

For more information on this factors disrupting the scaling down of the cluster, please see the What types of pods can prevent CA from removing a node? page.

In order to scale up and down the cluster, the autoscaler uses user-defined parameters for things like limits (what are the upper and lower number of nodes limits in your cluster?), nodes resource thresholds (at what load level should it add or remove a node?) or time intervals (how often should it scale the cluster?). These parameters are described in the Configuring the cluster autoscaler guide.

Enabling the autoscaler

Using Kubernetes API

The easiest way to enable the autoscaler is using the Kubernetes API, for example using kubectl.

As explained in the How nodes and node pools work guide, in your OVHcloud Managed Kubernetes cluster, nodes are grouped in node pools (groups of nodes sharing the same configuration).

Autoscale is configured on a node pool basis, i.e. you don't enable autoscaling on a full cluster, you enable it for one or more of your node pools.

You can activate the autoscaler on several node pools, each of which can have a different type of instance as well as different min and max nodes number limits.

When you create your cluster, you can bootstrap a default node pool in it, and you can add others in the Public Cloud section of the OVHcloud Control Panel or directly using the Kubernetes API.

To list node pools, you can use:

kubectl get nodepools

In my case I have one node pool in my cluster, called nodepool-b2-7, with 3 B2-7 nodes:

$ kubectl get nodepools
NAME            FLAVOR   AUTO SCALED   MONTHLY BILLED   ANTI AFFINITY   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   [...]
nodepool-b2-7   b2-7     false         false            false           3         3         3            3           [...]

As you can see, the AUTO SCALED field is set to false. Let's see why by looking at the node pool description.

You can then get the description of the node pool in YAML format using:

kubectl get nodepools <your_nodepool_name> -o yaml

For my example cluster:

$ kubectl get nodepools nodepool-b2-7 -o yaml
apiVersion: kube.cloud.ovh.com/v1alpha1
kind: NodePool
metadata:
    [...]
  name: nodepool-b2-7
  resourceVersion: "22315717368"
  selfLink: /apis/kube.cloud.ovh.com/v1alpha1/nodepools/nodepool-b2-7
  uid: de606632-3948-46c0-ba91-e34a6de6a1aa
spec:
  antiAffinity: false
  autoscale: false
  desiredNodes: 3
  flavor: b2-7
  maxNodes: 100
  minNodes: 0
  monthlyBilled: false
  scaleDownUnneededTimeSeconds: 600
  scaleDownUnreadyTimeSeconds: 1200
  scaleDownUtilizationThreshold: "0.5"
status:
  availableNodes: 3
  conditions:
    [...]
  currentNodes: 3
  observedGeneration: 2
  upToDateNodes: 3

In the spec section you can see that the autoscale parameter is set to false. In order to enable the autoscaler, you need to patch the node pool to set this field to true.

kubectl patch nodepool <your_nodepool_name> --type="merge" --patch='{"spec": {"autoscale": true}}'

As you can see in my example, patching the node pool definition enables the autoscaler:

$ kubectl patch nodepool nodepool-b2-7 --type="merge" --patch='{"spec": {"autoscale": true}}'
nodepool.kube.cloud.ovh.com/nodepool-b2-7 patched

$ kubectl get nodepools
NAME            FLAVOR   AUTO SCALED   MONTHLY BILLED   ANTI AFFINITY   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   [...]
nodepool-b2-7   b2-7     true          false            false           3         3         3            3           [...]   

During this beta phase, the configuration options for the cluster autoscaler are immutable, set to sensible by-default values. We plan to progressively allow our users to modify some of these parameters to better tailor them to their specific use cases.

When the autoscaler is enabled on a node pool, is uses a by default configuration. To better understand the by-default configuration and its parameters, see the Configuring the cluster autoscaler guide.

Go further

To have an overview of OVHcloud Managed Kubernetes service, you can go to the OVHcloud Managed Kubernetes page.

Otherwise to skip it and learn more about using your Kubernetes cluster the practical way, we invite you to look at our tutorials .

Join our community of users.


Did you find this guide useful?

Please feel free to give any suggestions in order to improve this documentation.

Whether your feedback is about images, content, or structure, please share it, so that we can improve it together.

Your support requests will not be processed via this form. To do this, please use the "Create a ticket" form.

Thank you. Your feedback has been received.


These guides might also interest you...

OVHcloud Community

Access your community space. Ask questions, search for information, post content, and interact with other OVHcloud Community members.

Discuss with the OVHcloud community

In accordance with the 2006/112/CE Directive, modified on 01/01/2015, prices incl. VAT may vary according to the customer's country of residence
(by default, the prices displayed are inclusive of the UK VAT in force).