How To Write A Weather Scaler For KEDA
During my internship, I was given a task to write a weather scaler for KEDA which takes JSON data from a weather API, which is OpenWeather, and scales the pods up and down by comparing the threshold value I specified in the YAML file and the temperature value that the API returns at that moment.
After reading this article you can have a basic understanding of what functions KEDA scalers require and how to write the functions specific to your scaler. The scalers are written in GO.
To see my code, check my Github.
NOTE: This task (weather scaler) was just for the sake of understanding the scaler concepts to be able to contribute to KEDA with a real-purpose scaler.
Setting Up Minikube and Docker, and Cloning KEDA
I used Minikube for local Kubernetes and Docker as container manager. Click on them for the installation.
You can make docker the default driver for minikube if you want
minikube config set driver dockerClone your fork of KEDA:
git clone <YOUR_FORK_OF_KEDA>In the same directory where KEDA has just created, download the keda-2.2.0.yaml file.
Defining the Process
We’re going to have a weather_scaler.yaml file in which we are going to set parameters for scaling and a weather_scaler.go file which is going to be the weather scaler itself.
Let’s analyze what we want from the weather scaler:
- It sends an HTTP request to the API and gets the JSON data
- It gets the preferred temperature value among all the other unnecessary JSON data (e.g. pressure, humidity) and compares it with the threshold value in the YAML file.
Well, it looks easy, doesn’t it? 😉 Then let’s dive into how to create this scaler.
Scaler Functions
Must-Have Functions for A Scaler
Create the weather_scaler.go file under pkg/scalers/.
- GetMetrics
That’s the function that gets the data we want to scale. In the weather scaler, it corresponds to the JSON data.
2. GetMetricSpecForScaling
That’s the function that gets the metrics we want to scale according to. In the weather scaler, it corresponds to the preferred value in the YAML file.
3. IsActive
If this function returns false, and if current replicas are greater than 0, and there are no configured minimum pods, then KEDA scales down to 0. In the weather scaler, it returns true if the current temperature is greater than the threshold value in the YAML file. Otherwise, it returns false.
4. Close
After each poll on the scaler to retrieve the metrics, KEDA calls this function for each scaler to allow closing any resources, like HTTP clients.
5. New<NAME_OF_YOUR_SCALER>Scaler
That’s the function executed first in the scaler. It calls Parse<NameOfTheScaler>Metadata, which is the next one.
6. Parse<NAME_OF_YOUR_SCALER>Metadata
That’s the function that parses the parameters and their values defined in the YAML file to the Metadata struct defined at the beginning of the scaler.
That’s all the 6 required functions to be able to create a scaler. Now, it’s time to define the functions specific to the weather scaler. ☔️
Functions Specific to The Weather Scaler
- GetWeather
This function calls GetJSONData, which is going to be the next function, and decodes returning jsonBlob with json.Unmarshal().
2. GetJSONData
This function sends an HTTP request to the API and reads the returning JSON data. It returns jsonBlob and err to the function it is called from, which is GetWeather().
Now that I suppose you understand the functions part, which was the biggest issue for me 😬, we can move on to the struct types I defined for my weather scaler.
Struct Definitions
weatherScaler struct has metadata that points to another struct named weatherMetadata and httpClient that points to http.Client. Since we send an HTTP request, we use http.Client.
weatherMetadata struct has the thresholdValue we want to scale according to, the host for the API URL, the cityName we want to get the temperature of, the apiKey we need to be able to connect to the API, and the preference data we want to scale, which you are going to see next.
WeatherData struct has a Main struct (These two start with a capital letter because of being JSON). In JSON, the temperature fields are inside the main, that’s why they are nested here, too. There are only 3 temperature variables named Temp, Temp_min, and Temp_max which are integers.
YAML File Configuration
It’s time to look at the YAML file in which we set the Deployment and the ScaledObject.
Create the weather_scaler.yaml file in the same directory where the scaler is (under pkg/scalers/).
Deployment includes the environment for the scaler. I named it weather-deploy.
ScaledObject includes the parameters that the scaler must get. I named it as weather-scaledobject. “minReplicaCount: 1” sets the minimum pod number to 1 when the current temperature is less than the threshold value in the YAML. “maxReplicaCount: 4” sets the minimum pod number to 4 when the current temperature is greater than the threshold value in the YAML. In metadata, there is “host” for the API URL, “cityName” for the city we want to get the temperature of, “apiKey” for the connection to the API, “thresholdValue” for the comparison, and “preference” for one of the 3 temperature values we want to scale according to.
NOTE: After you created the weather_scaler.go and weather_scaler.yaml file under pkg/scalers/, change the getScaler function in pkg/scaling/scale_handler.go by adding another switch case that matches your scaler. Scalers in the switch are ordered alphabetically, so follow the same pattern. In our case, write as shown below.
Build and Deploy
Log in to Docker:
docker loginStart Minikube:
minikube startNow, go to the root of the KEDA and run the following commands in order:
make build
make deploy
IMAGE_REGISTRY=docker.io IMAGE_REPO=<YOUR_DOCKER_HUB_REPO> make publish
IMAGE_REGISTRY=docker.io IMAGE_REPO=<YOUR_DOCKER_HUB_REPO> make deployIf you don’t have Docker Hub account, click to create.
And we need to apply the YAML file:
kubectl apply -f pkg/scalers/weather_scaler.yamlNOTE: If you make a change in the weather_scaler.go file, you must run these commands starting from “make build”.
If everything is okay, now it’s time to see the pods:
kubectl get pod 
You should see the weather-deploy here. Since we described the threshold value as 1 in the YAML file and the current temperature is greater than that, there are 4 pods equal to the maxReplicaCount. If you want only one replica(or pod) to be running, change the threshold value in the YAML file. Run the previous command related to the YAML file every time you change it to apply the changes.
See scaledobject:
kubectl get scaledobject
IsActive returns True because the current temperature is greater than the threshold value
See HPA:
kubectl get hpa
If you get some errors, you may want to check the logs. For that, run the command below to see the name of your keda operator:
kubectl get pod -n keda
You can see the logs with:
kubectl logs -f <NAME_OF_YOUR_KEDA_OPERATOR> -n keda
That’s all! 😌 🎉 I hope this article helped you to understand the basic concepts of a KEDA scaler, which was the main goal of writing this. Feel free to reach out to me in the comments in case there is something you want to correct or ask.
Check KEDA’s Github and its website out to learn more.
