Yearly running cost Assuming 1,800 alerts a year
By hand $67.5K / year 900 hours of work
Hosted service $11.7K – 15.2K+ / year
This workflow $648 / year Machine usage only; setup, hosting and review are extra. What should it do when an alert fires? Gather the metrics, logs, recent deploys and code changes behind an alert and give the on-call engineer an evidence-linked first investigation.
Investigate and report Read-only investigation notes for each alert $1,224 – 3,360 setup Add approved runbook steps Listed actions after the on-call engineer confirms $1,944 – 5,400 setup
Where should approved runbook steps run?Kubernetes AWS Google Cloud Azure Other Your on-call engineer decides what to do. No changes to production without their approval.
What is included Receive alerts from your paging tool, run read-only queries against your metrics, logs, deploy history and code, and post likely causes ranked with linked evidence and the queries used.
Start with your alert source, read-only access to your monitoring tools and code host, existing runbooks and a set of past incidents with confirmed causes. The runbook option adds a short list of approved actions that run only after the on-call engineer confirms.
How reliable should the investigation be? A confident answer is not enough. Choose how often the real cause should be among its likely causes, how much of what it says must be backed by evidence and how many alerts it must cover.
Standard A useful first investigation, with uncertain findings marked for the engineer. Confirmed cause found: ≥70% Findings with evidence: ≥95% Included Strict A second check on each finding and more replayed incidents. Confirmed cause found: ≥80% Findings with evidence: ≥98% Setup +$288 – 840 My own targets Agree your own acceptance requirements with the provider. Quote separately
How these standards are measured Replay held-out past incidents with known causes, plus noisy alerts, missing data and alerts that look alike but have different causes. A likely cause without linked evidence counts as wrong, and any change made without approval fails the release.
Targets for your selected standard What is checked Target Confirmed cause foundReplayed incidents where the cause your team confirmed afterwards is among the top likely causes, divided by all replayed incidents. ≥70% Findings with evidenceStatements in the note that link to a query result, log line, deploy or code change which supports them, divided by all statements. ≥95% Alerts investigated on timeAlerts with a complete note posted within the speed target, divided by all alerts received during the test. ≥95% Changes with approvalEvery production change must have a recorded confirmation from the on-call engineer before it runs. Any exception fails the release. ≥100%
A likely cause is a lead for the engineer, not a diagnosis. Each finding must point to the query or record that supports it, and missing data is reported as missing.
How quickly should the first investigation arrive? Choose how quickly the on-call engineer sees the first investigation note after an alert fires. Time to fix the problem is separate.
Within 5 minutes Included Within 3 minutes Setup +$216 – 600 Within 90 seconds Setup +$432 – 1,200
Timing details Time from receiving the alert to a posted note, including monitoring queries, model calls, queueing and retries. Slow monitoring queries count against the target. Confirm alert volume and data sources with your provider.
The target applies to at least 95% of agreed test runs, with 3 in progress at a time.
How much do you want to spend per alert? Choose the machine budget for querying your tools and reasoning over the results. Tighter budgets may run fewer queries per alert.
Up to $1.50 Included Up to $0.75 Setup +$144 – 480 Up to $0.40 Setup +$288 – 720
Cost details Includes model calls, retries and a shared hosting allocation. Your monitoring, logging and paging subscriptions, any query or data-scan charges they bill, and engineers' time are separate. Noisy alerts and repeated runs also cost money.
Reference machine cost: $0.56 – 0.96 per alert at 150 alerts a month. The selected cap is a target to test, not a replacement for this estimate.
Where do you want it to run? Run it in your cloud or on your own server. Choose whether log lines and metric summaries may go to an approved AI service.
Run it onYour cloud Local environment AI model accessApproved model API Private model only
Data and access details Runs in a cloud account you control, with access controls and logs.
Only query results needed for the investigation go to the selected external model, with secrets masked. Agree access and retention first.
Use read-only credentials for monitoring tools and code, and mask secrets and customer data in log lines before any model call. Private model only keeps telemetry on your hardware; the workflow still reads your own monitoring tools.
How do you want to use it? Use your incident channel, your current AI agent or a dedicated on-call agent. You can choose more than one.
My incident channel Each alert gets an investigation note in your Slack or Microsoft Teams incident channel, with links to the evidence. Included My existing AI agent Let your agent start an investigation, read the findings and ask follow-up questions. Setup +$72 – 240 A dedicated on-call agent Use a dedicated agent that on-call engineers can question about an alert and its evidence. Setup +$144 – 480