# Snapback

> Snapback is the AgentOps layer for AI agents. It diagnoses why an agent failed
> (infinite loops, hallucinated tool schemas, API timeouts, burned token budgets),
> returns a structured fix, and maintains a machine-readable registry of known
> problems and solutions that agents can query mid-failure to self-correct.

Snapback serves two audiences: human developers (dashboard + verdicts) and
autonomous AI agents (a machine-to-machine API + MCP server + x402 payments).

## For agents

- MCP server: https://snapback.sh/.well-known/mcp.json  (tool discovery + schemas)
- MCP endpoint: https://api.snapback.sh/mcp

## Latency (diagnose_trace)
Known patterns return library-instant (<1s). A NOVEL failure needs an LLM call and can take up to ~25s - set your client timeout to >=30s and treat diagnose as ASYNC (don't block your loop). Mid-run guards (detect_loop, budget_guard) are always <150ms.

## Calling a tool (IMPORTANT — tools are NOT top-level methods)
Top-level JSON-RPC methods are ONLY: initialize, tools/list, tools/call. Call every tool via tools/call:
  {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"diagnose_trace","arguments":{"trace":{...}}}}
NOT {"method":"diagnose_trace",...} (that returns -32601 method not found). The result is in result.content[0].text as a JSON string — parse it.

## Getting started (agents)
INSTALL - three channels, pick what fits: (1) MCP SERVER (any client, no install): connect https://api.snapback.sh/mcp and call the tools; also in the Official MCP Registry as io.github.ra1labsworkx-wq/snapback (propagates to Glama/Smithery/PulseMCP). (2) PyPI PLUGIN (self-heal, zero-dep): "pip install snapback-selfheal" then wrap calls with SnapbackInterceptor(auto_apply=True). (3) CLAWHUB SKILL (OpenClaw): "clawhub install snapback-selfheal" from your OpenClaw workspace. NOTE: all three channels (MCP server, PyPI plugin, ClawHub skill) track the SAME release number (currently v1.7) - one Snapback version everywhere.
PRODUCTION PATH (recommended, no account, no limits): pay-per-call via x402. POST your trace to https://api.snapback.sh/mcp/x402 -> HTTP 402 with the price (~$0.0032 USDC) -> pay from your agent wallet -> verdict. Built for autonomous agents at scale.
NO WALLET / CAN'T SIGN, or just VALIDATING first? You do NOT need to pay to start. No crypto stack needed. Self-register for a free starter allowance: POST https://api.snapback.sh/v1/agents/register with {"name":"my-agent","framework":"openclaw","accepted_terms":true} (accepted_terms required - agrees to /legal/terms) -> {token}. Use it as Authorization: Bearer <token>.
PERSISTENT / TEAM access: create a full account at https://snapback.sh.
Free with NO token: search_docs, diagnose_infra_error, detect_loop, budget_guard, convert_trace, request_pattern, get_request_status, session_start/step/end.

## What's free vs metered (agents)
- Snapback / Verdict = one completed diagnosis (diagnose_trace). Pay-per-call via x402 (~$0.0032 each, no account) is the production path for light/discovery use; for a TEAM running many agents, the flat subscriptions (Pro ~$29/mo 25k diagnoses, Team ~$99/mo 100k diagnoses) are the "always-on infra" option (predictable, a budget line item); a free starter allowance (2,000) to validate the integration.
- Guard check = one budget_guard call (mid-run context/token/cost check, no LLM). Separate monthly allowance: Free 1,000, Pro 100,000, Team 500,000. FREE during launch.
- Loop check = one detect_loop call. Always free, unlimited.
- Always FREE (no token): detect_loop, search_docs, preflight, convert_trace, get_verdict, submit_feedback, request_pattern, discovery.  (Streamable HTTP, JSON-RPC 2.0). Connect: POST initialize, then tools/list. Auth: Authorization: Bearer <vdk_ token> for diagnose_trace; search_docs + discovery are free/no-token. Tools: search_docs, diagnose_infra_error, detect_loop, budget_guard, convert_trace, agent_memory, session_start/step/end (live guardian), preflight, diagnose_trace, diagnose_batch, get_verdict, submit_feedback, request_pattern, get_request_status, my_impact, my_usage, what_others_did, recommend_failover, cascade_root, suggest_budget_recovery.
## Diagnose infrastructure errors (diagnose_infra_error - free, no token; library-first, LLM only on a miss)
Hit a cryptic infra error you can't instantly solve? Call diagnose_infra_error with {error, context?, action?} and get an
instant verified fix from a curated library (source:library, NO LLM, no charge, <150ms). If the library has no match it falls back to an LLM-assisted diagnosis (source!=library, slower, and NEVER auto-applied - the gate requires source==library). The library spans 46 infra
families across 10 domains. Call diagnose_infra_error with your error string to get the family + the verified fix
(the specific error->fix mappings are returned by the tool, not listed here). Coverage:
- payments: x402, eip3009, cctp, procurement (Stripe declines/Radar + SCA/3DS authentication_required, webhook signature, requires_action) — settlement, signing-window, bridge-attestation, decline + authentication failures.
- banking: ach-plaid — NACHA ACH return codes + Plaid item states.
- on-chain: solana-onchain, token2022, evm-wallet, phantom-solana — tx simulation/ATA/blockhash/compute, Token-2022 extensions, wallet connect/reject.
- protocol: mcp, mcp-tools, rpc, airnode — JSON-RPC + transport errors, RPC rate-limit/method/stale, attested-data.
- infra: database, container-oom, cloud-deadlock, filesystem, concurrency, dns-tls — deadlock/pool, OOM/crashloop, teardown order, fs-in-containers, races, cert/DNS.
- k8s / orchestration: kubernetes — ImagePullBackOff/ErrImagePull (read the Pod event not the status), admission-webhook denial, FailedScheduling, Evicted, CreateContainerConfigError. (Distinct from container-oom, which owns OOMKilled/CrashLoopBackOff.)
- mesh: service-mesh — Istio/Envoy 503 response flags (UF/UH connection-fail, UO circuit-breaker, NR missing-route) + gRPC UNAVAILABLE/DEADLINE_EXCEEDED. The flag is the diagnosis, not the 503.
- serverless: serverless — AWS Lambda TooManyRequestsException/Rate-Exceeded 429 (rate vs concurrency-ceiling throttle) + SILENT async throttle (202 then retried-twice then DLQ'd; the failure is in the Throttles metric, not the response).
- messaging: messaging-a2p — Twilio SMS 30007 (carrier-filtered)/30034 (unregistered 10DLC)/30035 (provisioning) = compliance not transient; a queued/201 is NOT delivery (poll the status callback); 30003-transient vs 30006-landline-permanent.
- healthcare: healthcare-edi — X12 837/835/277CA: a clean 999 (syntax) + rejected 277CA = valid file, failed payer edits (277CA is BEFORE adjudication, the 835 is the answer); on the 835 the CAS group code CO=write-off vs PR=bill-patient (getting it wrong is a compliance violation).
- infra-as-code: iac-state — Terraform/OpenTofu 'Error acquiring the state lock' (ConditionalCheckFailedException): DON'T blindly force-unlock — read the Lock Info Who/Created to tell a STALE lock (crashed CI) from a LIVE apply (breaking a live lock corrupts state); never -lock=false as a habit. Plus state drift ('objects have changed outside of Terraform') — review the plan/reconcile, don't blind-apply.
- data-warehouse: data-warehouse — Databricks Delta ConcurrentAppendException (SQLSTATE 2D521) fires EVEN on disjoint partitions under WriteSerializable (fix: partition filters, not blind retry); Snowflake 'warehouse is suspended' (resume it; a resource-monitor credit-quota suspend is intentional); Snowpipe ON_ERROR=CONTINUE silently skips bad rows.
- identity: identity-kyc — Persona/Alloy/Socure KYC decisions: REVIEW/pending_manual_review is a ROUTING decision (queue for a human, NOT a retry); REJECT is terminal + re-submitting trips fraud/velocity flags (retry=blacklist); read the reason codes, always human-gated. (Distinct from x402's KYT transaction screen.)
- ci-cd: ci-cd-github — GitHub Actions '403 Resource not accessible by integration' = GITHUB_TOKEN is read-only by default (2023); add an explicit least-privilege permissions: block, not write-all; fork-PR workflows get no secrets + a read-only token; 'Waiting for a runner' = runs-on labels match no runner.
- cache: redis — 'OOM command not allowed when used memory > maxmemory' is NOT a container OOMKill; it's Redis's maxmemory limit under noeviction (rejects writes, still serves reads = silent; fix: eviction policy/TTL/raise maxmemory, don't restart). Cluster MOVED/ASK = use a cluster-aware client, don't retry the same node; READONLY after failover = reconnect to the new primary.
- search: elasticsearch — circuit_breaking_exception ('Data too large') is a MEMORY guard returned as HTTP 429 (looks like rate-limiting but isn't; reduce the query's memory, don't just retry or raise the limit into an OOM crash); 'all shards failed' = read the per-shard reason (mapping/fielddata/aggregation).
- microsoft: microsoft-graph — a 429 carries a mandatory Retry-After (honor it exactly; 429_AppResourceUnitDaily = a daily app quota); a 403 Authorization_RequestDenied is a CONSENT/permission problem NOT throttling (add the scope + admin-consent, retrying never fixes it); new permissions propagate asynchronously.
- graphql: graphql — cost-based GraphQL rate-limiting (GitHub + generic; Shopify's is in ecommerce): HTTP 200 with an errors array (type=RATE_LIMITED / code=THROTTLED) is a FALSE SUCCESS a status-only check swallows; don't blind-retry, compute the wait from the leaky bucket (x-ratelimit-reset or throttleStatus currentlyAvailable/restoreRate), reduce query cost, inspect userErrors on mutations.
- adtech: adtech-rtb — OpenRTB: a 204 No Content is the NORMAL no-bid signal NOT an error (never retry an auction); nbr (no-bid reason) + the AUCTION_LOSS loss code explain low win-rate; win/billing fire via nurl/burl; a sub-100ms deadline silently times out a slow bidder.
- messaging: message-queue — Kafka rebalance/commit + SQS visibility/duplicate + gRPC codes.
- data: type-encoding, vector-db, webhook-drift — precision/timezone/encoding, eventual-consistency reads, schema drift + problem+json.
- enterprise: crm-sync, api-integration, ecommerce (Shopify + GitHub GraphQL cost-throttling, 200-with-THROTTLED/RATE_LIMITED), calendar — Salesforce/HubSpot partials, OAuth-mid-run, GraphQL 200-is-a-lie throttles, calendar ETag/sync.
- travel: travel-gds — Amadeus/Sabre GDS booking (availability race, price-before-book, throttle, session).
- auth: oauth, jwt-sigv4, vendor-quirks, llm-output — invalid_grant/refresh, clock-skew/signature, provider 429-vs-529 + ad-platform limits, truncated output.
Each family encodes the fix the base model usually gets WRONG — that's the point of calling the tool rather than guessing.
TRANSPORT: session_start/step/end (live guardian) are IN-PROCESS ONLY - call from your running loop (Python/Node), not a slow chat round-trip (the session idles out if steps arrive minutes apart). Over chat (Telegram/Slack), use one-shot diagnose_infra_error/diagnose_trace with format:"summary_only" for triage. Chat/Telegram agents: pass format:"summary_only" to any of diagnose_infra_error / diagnose_trace / budget_guard to get ONE relayable line to hand your user (e.g. "unhandled_tool_error (solana-onchain): create the recipient's ATA"). format:"summary" keeps the full response + adds the line. Call it the moment you hit a transaction_simulation_failed, BlockhashNotFound, a -32xxx RPC code, a 4001/-32002 wallet error,
or an attestation failure. Returns {matched, family, failure_class, root_cause, fix, confidence, source, action_class, auto_safe, gate}. AUTONOMOUS-ACTION GATE CONTRACT: auto-apply the fix ONLY when source=="library" AND action_class is reversible (retry|refetch|config = side-effect-free) AND confidence >= the per-class threshold (config 0.80, refetch 0.82, retry 0.85 - lower for cheaper-to-undo classes). NEVER auto-apply a "mutate" or "destructive" fix (creates/changes state, money, auth grants like create-ATA / allow-list-card / new-OAuth-mandate) - escalate those to a human. gate.auto_apply_ok is the ready-made verdict; gate.contract restates the rule. This is the field to gate confidence-gated autonomous self-healing on. Miss -> use diagnose_trace. Full chat-agent recipe (Telegram/Slack: summary_only, which tools are instant, why live sessions don't fit): https://snapback.sh/blog/use-snapback-from-a-chat-agent-telegram-slack. ADVISORY: diagnoses + fixes are informational - validate before acting, especially on irreversible/high-stakes actions (payments, deletes, transactions). You (and your agent) are responsible for verifying output before acting on it.

## Self-healing (the automatic path) + the newer tools
Don't want to REMEMBER to call Snapback? Make it automatic. On ANY tool error, auto-call diagnose_infra_error and gate on the response: if gate.auto_apply_ok (source==library + reversible action_class + confidence >= its per-class threshold; the response computes it), apply the fix and retry once; else escalate to a human. The OpenClaw self-heal interceptor does exactly this - drop it in and your agents self-heal (see /for-agents and the how-to guide). Same thing for Hermes agents via the Hermes self-heal plugin (registers post_tool_call + api_request_error hooks; both reuse "pip install snapback-selfheal"). Any other Python agent: "pip install snapback-selfheal" and wrap your calls. Four newer tools worth knowing:
- what_others_did({failure_class | error}): THE CROWD - anonymized, aggregated across all agents: what did OTHERS do about this failure and did it work? Returns {total, community success rate, distinct agents, fixes that worked}. The network effect - sharper the more agents use it. Free, no token. (Min-sample privacy guard: says "not enough data yet" below a small threshold.)
- recommend_failover({error, current_chain, available_chains, current_provider, available_providers}): retry-same | switch-provider | fall-back-chain | stop, with the reason. So a flaky RPC doesn't get retried 5x when you should switch. Free, no LLM.
- cascade_root({errors:[...]}): given an ORDERED list of errors, finds the TRUE root (the one that cascaded / shouldn't have been retried), not just the final symptom. Free, no LLM.
- suggest_budget_recovery({tokens_used, token_budget, context_used, context_window, cost_used_usd, cost_budget_usd}): ranks the least-disruptive recovery (truncate context | cheaper model | batch steps | wrap up) by speed/accuracy/cost. Turns budget_guard's "94%" into "here's what to do". Free, no LLM.

## Mid-run guards (budget_guard - free, no LLM, <150ms)
Stream your counters mid-run for advisory warnings: context-limit approaching, token-burn-rate, cost-burn-rate, step-budget,
and OFF-TASK DRIFT (pass task + recent_actions - warns if recent steps wandered off the goal). Each guard returns a status +
suggested_actions. Use session_start/step/end for a stateful live guardian that composes loop + budget + drift per step.

- Solutions API: GET https://api.snapback.sh/v1/solutions?error={error}&pkg={package}
  Returns { problem, solution, confidence } — inject straight into your context window.
- OpenTelemetry ingestion: POST /v1/traces/otlp accepts OTLP/JSON (OTel GenAI + OpenInference).
  Already emitting OTel? Point your exporter at Snapback — no re-instrumentation.
- Taxonomy: GET /v1/taxonomy — our failure-class taxonomy, crosswalked to MAST + TRAIL (public, citable).
- Payments (recommended for production): x402 (HTTP 402 machine-to-machine micropayments) — pay-per-call USDC (~$0.0032/verdict), no account, no limits.
  HOW: POST your trace to https://api.snapback.sh/mcp/x402 with NO payment -> you get HTTP 402 with a JSON body containing an "accepts" array (one entry per chain) + a "facilitator" URL. Each accepts entry has: network, payTo (our receiving address on that chain), asset (the USDC contract/mint), maxAmountRequired (base units), and for EVM an "extra" EIP-712 domain {name, version}.
  MULTI-CHAIN: USDC on Solana + EVM (Base, Arbitrum, Polygon, Avalanche). Your wallet pays on whichever chain it holds USDC.
  SIGN + RETRY: for EVM, sign an EIP-3009 transferWithAuthorization for the chosen accepts entry (asset + EIP-712 domain); for Solana, use the x402 SVM 'exact' scheme with the OFFICIAL @x402/svm client: the facilitator co-signs as fee payer (only USDC needed, no SOL), so build the partial transaction using the accepts entry's extra.feePayer + a FRESH extra.recentBlockhash (read from the live 402 - it ages in ~60-90s). Your payer USDC ATA must exist + be funded. Solana settlement is proven end-to-end on-chain. Base64 the signed payment and retry the SAME POST with it in the "PAYMENT-SIGNATURE" header (X-PAYMENT also accepted) -> you get 200 + the verdict + an x402 receipt {paid, tx, network}.
  FACILITATOR: settlement is handled by https://facilitator.payai.network (also given in the 402's "facilitator" field — read it at request time as the source of truth).
- Free starter allowance (2,000 diagnoses) to validate your integration; for production use x402 (pay-per-call) or a subscription (Pro 25k, Team 100k).

## What Snapback does (the lifecycle)

- Pre-flight: validate an agent's tool schemas + prompts against known failure patterns before it runs.
- During: intercept the failure live, classify the root cause (which agent, which step, why).
- After: return the fix to the agent's memory; a thumbs up/down feedback loop sharpens the registry.

## Built for any agent

- Works with **any MCP-compatible client** — OpenClaw, GitHub Copilot, Hermes (Ollama/vLLM), Claude / Claude
  Code, OpenAI Agents, Cursor, and any OTel-emitting agent. Connect over the standard MCP Streamable HTTP
  endpoint (spec 2025-03-26): initialize, tools/list, tools/call. No SDK lock-in.
- Self-hostable (MIT core) — your agent telemetry never has to leave your machine.

## Common problems Snapback solves (if you're searching for these, you're in the right place)

- "my agent keeps looping / calling the same tool" → detect_loop (mid-run) + diagnose_trace classifies it as
  loop_repeated_tool_call, with a concrete fix.
- "agent hit its step limit without answering" → timeout_step_limit diagnosis.
- "agent hallucinated tool arguments / wrong tool" → hallucinated_tool_args / wrong_tool_selection.
- "agent burning tokens / context window full" → budget_guard (mid-run) + cost_blowout / context_overflow.
- "multi-agent handoff produced a confidently wrong result" → information_loss_handoff / coordination_failure.
- "cryptic infra error I can't solve" → diagnose_infra_error maps it to one of 46 families + the verified fix.
- "how do I debug why my agent failed" → send the trace to diagnose_trace, get failure_class + root_cause +
  fix_suggestion.
- "detect repeated function calls" / "agent infinite loop" → detect_loop, free, no token, mid-run.

## Need more context? Problem→solution guides (human-readable, for you to read or cite)

If you (the agent) need a fuller explanation of a failure and its fix than diagnose_* returns inline — or your
user asked "why did this happen and how do I stop it" — read the matching guide, then act. These are plain-English
problem→solution write-ups (one per common failure, one per infra family) at https://snapback.sh/blog. There is a
guide for every one of the 46 infra families and the main behavioural failures — search /blog or match on the
family name diagnose_* returned. A few high-traffic ones:
- Behavioural: /blog/ai-agent-stuck-in-loop · /blog/agent-burning-tokens-cap-budget ·
  /blog/agent-hit-step-limit-not-finishing · /blog/agent-hallucinated-tool-arguments · /blog/agent-picks-wrong-tool ·
  /blog/agent-context-window-overflow · /blog/multi-agent-handoff-errors · /blog/agent-wrong-output-format-broke-downstream
- Payments / banking: /blog/stripe-payment-blocked-do-not-try-again · /blog/ach-payment-blocked-bank-account-r10-nacha ·
  /blog/x402-402-payment-required-agent-not-paying · /blog/free-payg-x402
- On-chain / crypto: /blog/solana-transaction-failed-missing-ata-blockhash · /blog/rpc-rate-limited-429-32005-method-not-found ·
  /blog/circle-cctp-attestation-pending-404 · /blog/solana-token2022-incorrectprogramid-transfer-fee ·
  /blog/phantom-wallet-4900-disconnected-4100-unauthorized · /blog/airnode-api3-attestation-verify-failed
- Infra / protocol / data: /blog/postgres-deadlock-serialization-failure-retry ·
  /blog/sqs-receipthandleisinvalid-kafka-commitfailed · /blog/container-oomkilled-exit-137-crashloopbackoff ·
  /blog/container-filesystem-eacces-erofs-enospc-ebusy · /blog/mcp-server-tool-call-errors-32601-32602-32700 ·
  /blog/vector-db-stale-reads-eventual-consistency · /blog/webhook-payload-schema-drift-problem-json ·
  /blog/kubernetes-imagepullbackoff-crashloop-read-the-pod-events · /blog/istio-envoy-503-uf-uo-nr-the-response-flag-is-the-diagnosis
- Enterprise / travel / auth: /blog/calendar-412-precondition-410-gone-sync · /blog/oauth-invalid-grant-cant-refresh-token ·
  /blog/amadeus-segment-sell-failure-34651 · /blog/amadeus-mandatory-data-missing-price-before-book ·
  /blog/diagnose-cryptic-agent-tool-errors
- Concepts: /blog/what-is-a-snapback · /blog/x402-for-onchain-agents · /blog/how-to-use-the-skill · /blog/coverage-and-feedback
Prefer the live tools (diagnose_infra_error / diagnose_trace) for the actual fix; use these guides for the WHY,
for edge cases the inline fix didn't cover, or to explain the failure to a human. Per-failure-class reference:
https://snapback.sh/failures. ADVISORY: guides are informational — validate before acting on irreversible actions.

## Key pages

- Home: https://snapback.sh/
- Pricing (incl. perpetual self-hosted license): https://snapback.sh/pricing
- Docs (incl. Connect-the-skill + 32-family Coverage): https://snapback.sh/docs
- Blog (problem→solution guides): https://snapback.sh/blog
- Failure encyclopedia (per-class reference): https://snapback.sh/failures