Hi everyone!! In kubernetes, Horizontal pod autoscaling(HPA) is mostly done either based on CPU or memory. But, in some cases, we’d like to scale the pods based on custom metrics such as ‘number of http requests received per second’. In this tech blog, I will explain how that can be achieved. 

Generally, there are 3 types of metrics in kubernetes. 

1. Resource Metrics: Provides the predefined resource usage metrics such as CPU and memory of Pods and the Nodes.
2. Custom Metrics: provides the custom metrics that are associated with a Kubernetes object
3. External Metrics: provides the custom metrics that are not associated with a Kubernetes object

Resource metrics are available out of the box. Whereas, for using custom metrics and external metrics, we must have the following things in place.

1)prometheus :- It is a data source and a great tool for collecting and recording all the metrics.

2)prometheus adapter: It is a metric API server that integrates with Prometheus as a metric collector. It is needed when you want to use custom metrics or external metrics for scaling the pods.

1)Steps for installing prometheus adapter:

Firstly, run the following commands to add the helm repository named ‘prometheus-community’.

1)helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

2)helm repo update

For using prometheus adapter, we can use a helm chart named ‘prometheus adapter’.

But, before deploying prometheus adapter, you need to first run the following command and get the values of the ‘prometheus adapter’ helm chart. The command to do that is

helm show values prometheus-community/prometheus-adapter > values.yaml

This command will give all the default values that will be used by helm in ‘values.yaml’ file. In that values.yaml, you need to make sure the prometheus.url and prometheus.port values are correct. Your prometheus server url and port need to be given there.

For example, if my prometheus url is ‘http://prometheus-server.prometheus.svc.cluster.local’ and port is 80. you need to override the values in the values.yaml file as given below.

prometheus:
url: http://prometheus-server.prometheus.svc.cluster.local
port: 80
path: “”

2)Defining the custom metric:

In values.yaml file which we got in step 1, you can modify and define all custom rules under ‘rules’ section. In this example, I have defined one custom metric ‘http_requests_received_total’. Before you use the metric, you need to make sure that your application is sending that metric in prometheus. The following custom metric gets the total http requests received and converts it into the http requests received per second.

In a similar way, you can define any other custom metrics.

Now, Run the following command to install the prometheus adapter.

helm upgrade -f values.yaml -i prometheus-adapter prometheus-community/prometheus-adapter -n prometheus

This command will deploy all the things related to prometheus adapter.

3) Apply the horizontal pod autoscaling file.
After defining the custom metrics, we can use that metric in horizontal pod autoscaler file.

hpa.yaml:-

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: test-autoscaler
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: <deployment_name>
minReplicas: 2
maxReplicas: 5
metrics:
– type: Pods
pods:
metric:
name: http_requests_received_per_second
target:
type: AverageValue
averageValue: 500m #i.e 500 milli requests per second, (1 request every 2 seconds)

Use the following command to apply the file

kubectl apply -f hpa.yaml.

Once you apply this file, you should be able to see that your pods are able to scale up based on the custom metric you defined.

Conclusion:

In this article, You have seen how to create prometheus adapter, define the custom metrics and scale up your pods based on custom metrics.

Thanks for reading!! I hope you enjoyed the article.