What does the confidence score in a Snapback verdict mean?

Confidence (0 to 1) is how strongly the diagnosis matched. >= 0.85 is a strong, safe-to-act match. 0.7 to 0.85 is likely-correct but worth a glance. Below 0.7 is a weak match — treat it as a hint, verify before acting, and never auto-apply. A 0.55 is a red flag: the matcher wasn't sure, so a human (or a more careful path like diagnose_trace) should confirm.

When should I trust a verdict vs escalate to a human?

Auto-act only when confidence >= 0.85 AND source == 'library' (a curated verified fix, not an LLM guess) AND auto_safe == true (the fix is reversible). If any of those is false — low confidence, an LLM inference, or a state-changing fix — escalate to a human instead of acting. The gate.auto_apply_ok field gives you that verdict directly.

← All posts

How-to

How to read a Snapback verdict — confidence, source, and when to escalate

For developers and autonomous AI agents · ~5 min read

A Snapback verdict is designed to be acted on — but only correctly. It carries a few fields that tell you exactly how much to trust it and whether it's safe to act on without a human. Here's how to read them, so you never relay a low-confidence guess as fact.

The fields, and what they mean

confidence (0–1)

How strongly the diagnosis matched. >=0.85 strong; 0.7–0.85 likely; <0.7 weak (a hint, not an answer).

source

'library' = a curated, verified fix a human reviewed. 'llm' = a plausible inference (can be confidently wrong).

action_class

retry / refetch / config (reversible) vs mutate / destructive (changes state, money, auth).

gate.auto_apply_ok

The ready-made verdict: is it safe to auto-apply? True only when high-conf + library + reversible.

The trust ladder

confidence >= 0.85 + source: library + auto_safe: safe to auto-apply the fix and retry once. This is the green light.
0.7–0.85: likely correct — apply with a glance, or in chat relay it but note it's not certain.
< 0.7 (e.g. 0.55): a red flag. The matcher wasn't confident. Treat it as a hint only — verify before acting, and NEVER auto-apply. Send the full trace to diagnose_trace for a deeper look, or escalate.
source: llm (any confidence): an inference, not a verified fix. Don't auto-act; a confident-sounding LLM guess is the most dangerous kind of wrong.
action_class: mutate / destructive: never auto-apply, regardless of confidence — escalate anything that creates or changes state, money, or auth.

Concrete example: if you paste a truncated-output error and get back family: procurement, confidence: 0.55, that low confidence is telling you the match is weak — do NOT relay "check your spending limits" to a user. A 0.55 is exactly the signal to escalate or re-diagnose, not to act.

For chat agents

When you're relaying a verdict into a Telegram or Slack message, a low-confidence answer is prefixed with a "⚠ LOW CONFIDENCE — verify before acting" warning (below 0.7). Don't strip it — that hedge is what keeps you from confidently telling a user the wrong fix.

The one rule

Act autonomously only on gate.auto_apply_ok: true. Everything else — low confidence, an LLM source, or a state-changing fix — gets a human. That single rule is the difference between an agent you can trust unattended and one that acts on a guess.

For AI agents

Machine-readable guide: https://snapback.sh/llms.txt · Tool discovery: https://snapback.sh/.well-known/mcp.json · MCP endpoint: https://api.snapback.sh/mcp. Free tools need no token; pay-per-call diagnosis via x402 on Solana or EVM (~$0.0032 USDC).

Try Snapback free Read the docs