Yearly running cost Assuming 60,000 sessions a year
By hand $975K / year 15,000 hours of work
Subscription n.a.
This workflow $2.1K / year Machine usage only; setup, hosting and review are extra. How should sessions be collected? Replay agent tool calls, find failed steps and track latency without exposing secrets in your logs.
One session includes up to 20 tool calls.
Review from exports Session replay and issue summary $1,152 – 3,480 setup Instrument a live application Add an authorized system connection $1,368 – 4,200 setup
Which system should it connect to?MCP server Tool-using agent application Observe only. No tool re-execution, production changes or automatic fixes. Redact secrets before telemetry leaves the application.
What is included Use exported traces for the first scope. Event ordering and latency are calculated from records, not invented by AI. The cost unit is a completed session summary, not each tool call.
Use exported traces for the first scope. Event ordering and latency are calculated from records, not invented by AI. The cost unit is a completed session summary, not each tool call.
Stored history: up to 30 days.
Application scope: up to 1 instrumented application.
Model input budget: up to 6,000 tokens across the reference run.
Model output budget: up to 700 tokens across the reference run.
Standard independent test set: at least 100 sessions.
Strict independent test set: at least 200 sessions.
Reference application CPU: 2 cores.
Reference application memory: 4 GB.
What standard do you need? Choose the acceptance targets your provider must demonstrate on your own test material.
Standard Test the defined workload and controls. Expected events captured: ≥99% Correct issue groups: ≥90% Included Strict Tighter targets and a larger independent test set. Expected events captured: ≥99.5% Correct issue groups: ≥95% Setup +$360 – 1,200 My own requirements Agree different targets and scope with your provider. Quote separately
How these standards are measured Use independently recorded reference traces including retries, out-of-order events, queue overflow, process restarts and seeded secret fields. Do not replay production tool actions during testing. Agree the holdout set before tuning. Report each metric with its numerator, denominator and exclusions; abstentions count against coverage. Test all inputs, including failures, rather than only accepted results.
Targets for your selected standard What is checked Target Expected events capturedComplete, correctly ordered events divided by independently recorded expected events, with drops reported rather than hidden. ≥99% Correct issue groupsCorrectly grouped labeled failure sessions divided by all labeled failure sessions, including unknown causes. ≥90% Seeded secrets removedSeeded secret fields absent from stored and exported traces divided by all seeded secret fields; any exposure fails the pilot. ≥100% Approval and data controlsPassed control cases divided by all independently tested unauthorized-action, stale-input and sensitive-data cases; any critical failure rejects delivery. ≥100%
Targets apply to the agreed independent test set, not every possible future input. During the pilot, a person reviews every result. Any unauthorized action or sensitive-data exposure fails acceptance.
How quickly do you need each session? Choose the time from accepted input to a reviewable session. Human review and waiting for source access are separate.
Within 30 seconds Included Within 15 seconds Setup +$144 – 480 Within 10 seconds Setup +$288 – 840
Timing details Measure wall-clock time including queueing, model calls, validation and permitted retries. Report failures and deadline misses. Faster targets need pilot testing; they are not vendor benchmarks.
The target applies to at least 95% of agreed test runs, with 1 in progress at a time.
How much should each session cost? Choose a machine-cost target per session. A lower cap may limit model calls and iterations.
Up to $0.07 Included Up to $0.055 Setup +$144 – 480 Up to $0.045 Setup +$288 – 840
Cost details Includes reference model usage, a retry allowance and allocated hosting at the stated volume. Human review, source-system subscriptions and paid connector access are separate. A tighter target must preserve the selected standards in the pilot.
Reference machine cost: $0.04 – 0.05 per session at 5000 sessions a month. The selected cap is a target to test, not a replacement for this estimate.
Where do you want it to run? Run the workflow in your cloud or on your own machine. Choose separately whether it may use an external AI model.
Run it onYour cloud Local environment AI model accessApproved model API No external model calls
Data and access details Run in your controlled cloud environment with scoped access and retained review records.
Use an approved external model with the agreed data-processing terms and minimum necessary input.
Local hosting does not itself keep data offline. Connected business systems may still use the network. Private inference needs suitable hardware and a separate measured running cost; access and retention rules must be agreed before real data is used.
How do you want to use it? Choose a browser interface, an MCP tool for your current agent, or a standalone agent.
Web page Review each session in a private browser interface. Included Your AI agent Call the workflow through MCP; the same access and approval rules apply. Setup +$72 – 360 Standalone agent Use a dedicated agent with the agreed triggers and controls. Setup +$144 – 480