HPA (Horizontal Pod Autoscaler) Operation Principle

HPA operates in three stages: Metrics API -> HPA Controller -> ReplicaSet/Deployment adjustment.

1. Collects current metrics from Metrics Server or Custom Metrics API.
2. HPA Controller calculates the required number of replicas by comparing with the target value.
3. Updates the replicas value of the Deployment/ReplicaSet via a PATCH request.
4. Kube-controller-manager actually scales the number of pods.

Components #

Metric Providers

• cpu/memory: metrics-server
• Prometheus based or applicaton based: custom metrics adapter

API Types

• resource metrics (Memory, CPU)
• custom metrics (QPS, latancy)
• external metrics (kafka lag, sqs length etc..)

In other words, metrics run based on either the default metrics server or our custom-built metrics collection server.

The HPA Controller then adjusts the number of replicas using a calculation formula based on those metrics.

HPA Controller's Core Scaling Calculation Formula #

For CPU-based scaling:

desiredReplicas = ceil( currentReplicas × currentMetricValue / targetMetricValue )

If:

• Current: 3 pods
• Target CPU: 60%
• Actual CPU: 120%

Then:

desired = ceil( 3 × 120 / 60 ) = ceil(6) = 6개

HPA Scale Out/In Conditions #

• Scale Out Conditions
  • Increases if the metric is above the target value.
  • Applies if the calculated replica count is greater than the current count.
  • No default cooldown (upward scaling is applied immediately).
• Scale In Conditions
  • By default, there's a stabilization period to prevent rapid reduction.
  • --horizontal-pod-autoscaler-downscale-stabilization
  • Default: 5 minutes.
  • This means it doesn't immediately scale down if the CPU briefly drops.

HPA Stabilization Logic #

HPA attempts to maintain the largest desired replicas value calculated over the last 5 minutes.

• Prevents drastic fluctuations.
• Controls to prevent sudden scale-in events.

How HPA Adjusts Deployments #

HPA does not directly modify the Deployment; instead, it calls the following API:

PATCH /apis/apps/v1/namespaces/{ns}/deployments/{name}/scale

When this value changes, the Deployment controller creates/deletes new pods.

Operation Cycle (reconciliation loop) #

Default cycle: Checks every 15 seconds (--horizontal-pod-autoscaler-sync-period).

Every 15 seconds:

1. Reads the latest metrics.
2. Calculates scaling.
3. Checks stabilization.
4. PATCHes replicas.

If CPU-based HPA malfunctions and overscales #

• The metrics server provides 15-second snapshots.
  • Autoscales when momentary spikes are reflected.
• Incorrect values increase when CPU throttling occurs.
• If the container request (e.g., request=100m) is too small, CPU usage is calculated as high.

The CPU% calculation formula is as follows:

CPU% = (현재 CPU 사용량 / request 값) × 100

This means if request is set too low, it will scale incorrectly.

Custom Metrics Based flow HPA #

Based on Prometheus adapter:

• Prometheus adapter provides /apis/custom.metrics.k8s.io.
• HPA requests metrics from that API.
• Adapter queries metrics from Prometheus.
• HPA controller calculates and determines replicas.

It's essentially the same.

Parameters #

Min/Max Replicas

minReplicas: 2
maxReplicas: 20

stabilizationWindowSeconds (scale-in limit)

behavior:
  scaleDown:
    stabilizationWindowSeconds: 300

Policy-based adjustment ratio limit

behavior:
  scaleUp:
    policies:
      - type: Percent
        value: 100
        periodSeconds: 60