Rocky AI

Rocky AI is Checkly’s AI agent. Rocky AI can help you automatically determine user impact and root cause on any failing check using error messages, code, metrics, OTeL traces, trace routes, packet captures: essentially all artefact generated by a check or test.

​

Rocky AI Capabilities

​

AI Assistant

Use the AI Assistant to plan monitoring and create checks and monitors with a generated check intent.

​

Automatic Check Repair

Set an intent and enable automatic repair for a Browser or Multistep check.

​

Root Cause and User Impact Analysis

• User Impact: Rocky AI analyzes the failure and provides you with a list of impacted user groups and features.
• Root Cause: Rocky AI analyzes the failure and provides you with a root cause analysis.
• Step Summary: Rocky AI summarizes the steps in the check script, making it easier to understand what the check is doing and where it failed. This typically works for any Playwright based check.

Learn More

​

Consumption and Invocations

By default, Rocky AI automatically analyzes every new error group. You can disable automatic analysis in the AI settings to only spend invocations on manual analyses. Each root cause analysis uses one AI invocation from your plan’s allowance, whether it starts automatically or manually. Sending the same analysis to multiple alert channels still uses only one invocation. An automatic check repair uses one additional invocation. Each AI Assistant conversation uses one invocation on its first run. Later messages in that conversation use no more invocations. Initialize from saved check does not use an invocation. When your invocation quota is reached, no overages are charged. Invocations refresh at the start of your next billing cycle. You can monitor your current usage and configure automatic analysis in the AI settings.

​

Error groups

An error group is a deduplicated bucket of check failures that share the same underlying error. Instead of treating every failure as a separate event, Checkly cleans the raw error message and clusters failures with matching cleaned messages into a single group per check. A group records when the error was first and last seen, so it reads as “this specific error has occurred on this check, starting on date X, most recently on date Y.” A new group is created the first time a check fails with an error that doesn’t match any existing group — and that’s the event that triggers an automatic analysis. Because minor variations are normalized away, repeated failures of the same error reuse the existing group and don’t consume additional invocations.

​

Data Privacy and Security

• All AI features are designed to operate on the data you provide or the data generated by your checks. This data is already stored in Checkly, and we do not send any additional data to third-party AI providers.
• Values of secret environment variables are scrubbed from check results when the check runs, before the results are stored and before Rocky AI reads them.
• We do not train any custom models on your data.
• Any data used by the AI features is processed in real-time and not stored for future training.

​

Built-in and Custom Models

By default, Checkly uses OpenAI’s GPT-6.1 Sol model for AI features. You can also bring your own model (BYOM) by selecting a custom model in the AI settings. Custom models run on your own Azure OpenAI account. The available models are GPT-6.1 Sol and GPT-5.1. Provide your Azure OpenAI API key and resource name as environment secrets named CHECKLY_AI_AZURE_API_KEY and CHECKLY_AI_AZURE_RESOURCE_NAME. If your deployment isn’t named after the model, also set CHECKLY_AI_AZURE_DEPLOYMENT_NAME.