Skip to main content
Scale Checkly Agent pods automatically in relation to live load. This page covers the KEDA-based recipe; for static capacity planning, see Scaling and Redundancy.

​
The signal

Checkly exposes the checkly_private_location_check_runs gauge through the Prometheus v2 endpoint. Filtered by state and a private_location_slug_name, it provides the count of pending and currently-executing check runs in a single private location — the signal you drive replica count from. The relevant state values are:
  • queued — the check run has been scheduled but not yet picked up by an agent.
  • inflight — the check run is currently being executed by an agent.
The gauge is aggregated on a ~1 minute interval, so checks that start and finish within that window may be excluded — their impact on private location capacity is negligible.

​
KEDA ScaledObject

The ScaledObject below provides sensible defaults — adjust the bounds and scaling behavior to match your check workload.
The query is scoped to a single private location by private_location_slug_name, so create one ScaledObject per private location. It counts only the main queue: uptime monitors and API checks run on the uptime queue, which the agent processes with AGENT_WORKER_COUNT workers instead of JOB_CONCURRENCY slots.
If you deploy agents with the Checkly agent Helm chart, template the ScaledObject alongside your chart values so the autoscaler ships with the deployment.
For a Prometheus instance outside the cluster, add an authenticationRef pointing at a TriggerAuthentication resource with the appropriate credentials.

​
How many pods you’ll get

KEDA queries Prometheus on its polling interval and turns the result into a target pod count. With threshold: "1", that target is roughly the number of queued plus in-flight check runs — one pod per check. The pod count is then kept within minReplicaCount and maxReplicaCount. For example, with threshold: "1", minReplicaCount: 2, maxReplicaCount: 10:

​
Tuning the bounds

  • threshold — set it to match the agent’s JOB_CONCURRENCY. The default JOB_CONCURRENCY is 1, so leave threshold: "1". A higher value packs more checks per pod and can cause scheduling delays for long-running checks.
  • minReplicaCount — keep at 2 or higher so a single agent failure doesn’t take the private location offline. See Scaling and Redundancy.
  • maxReplicaCount — must exceed your expected peak queued + in-flight check runs. If the cap is too low, queued check runs accumulate above it and are dropped once the queue TTL expires: 10 minutes for the main queue on current agents, 6 minutes for uptime monitors and for private locations still running an older agent.
If you set minReplicaCount: 0 to scale to zero when idle, cooldownPeriod becomes important — it controls how long KEDA waits after the trigger goes inactive before scaling the deployment down to zero.

​
Graceful termination

In-flight checks on a terminating pod are rerun on another agent after a 300-second timeout. Set terminationGracePeriodSeconds above this on the agent pod spec so an evicted pod has room to drain before SIGKILL:
Maximum runtime by check type:

​
Verify

  1. Confirm KEDA created the HPA and is reading the metric:
  2. Probe the signal directly:
  3. Schedule a burst of checks against the private location and watch the replica count climb toward maxReplicaCount, then settle back to minReplicaCount once the burst clears.

​
See also