Your Agent Timed Out. Did It Just Do the Job Twice?

After an AI agent’s write call times out, treat the outcome as unknown until you have evidence. Retry only under the endpoint’s idempotency contract, reconcile the original operation through an authoritative lookup, or stop for review. A missing response does not tell you whether the work happened.

Here is the bug in four lines:

Agent:    Create the ticket.
Service:  Ticket created.
Network:  [response lost]
Agent:    That failed. Create the ticket.

Both tickets can be perfectly valid. Your error dashboard may have very little to say about the second one.

This guide gives you a retry decision card, a local experiment, and a test matrix for the parts a successful demo never exercises. Keep it beside the code that executes your tools.

The conversation has moved past “can it call the API?”

On X, Alex Cloudstar (@alexcloudstar) asked:

“When a webhook fails and you need to retry, how do you handle idempotency?”

— Alex Cloudstar, X

On LinkedIn, Arpit Bhayani put a useful implementation rule plainly:

“The key has to live above the retry loop, not inside it.”

— Arpit Bhayani, LinkedIn

A Hacker News exchange made the missing contract especially concrete. Asked about duplicate requests, the creator of Signbee said its API did not yet solve idempotency and that another POST could create another document. That was the behavior reported in that discussion, not a claim about the service’s current implementation. Original discussion

These discussions are useful leads, not a prevalence study. The engineering rule predates agents: HTTP semantics caution against automatically repeating a non-idempotent request without knowing it is safe to repeat or knowing the original was never applied. RFC 9110, §9.2.2

Agents add another place a repeat can originate. Your SDK retries. Your workflow resumes. Then the model decides to try a different tool. The intended task has stayed the same while the system has found three ways to perform it again.

The agent retry card: retry, reconcile, or stop

Decide from the operation’s contract and saved state, not the HTTP status alone. This is a review aid; implement the actual rules in the execution layer.

The status-code distinctions matter. X’s API documentation, for example, explicitly allows partial errors inside a 200 response and distinguishes rate limits from usage caps. Read the endpoint’s result contract rather than flattening everything into “success” and “try again.” X API response codes and errors

For MCP integrations, inspect the tool result too. The versioned MCP tools specification describes execution errors reported through isError; transport success alone is not the whole result. MCP tools specification, 2025-11-25

One intended action, one durable operation ID

Suppose a support agent has permission to create a follow-up ticket. Before the first write, your application records that intended action and assigns an operation ID. The same ID survives a timeout, a worker restart, and a handoff to another agent.

You also need separate attempt IDs for debugging. Five attempts can belong to one operation. Two separately approved tickets can be two operations even when their text is identical.

That distinction is easy to lose if you generate a fresh UUID inside the retry loop. It is also easy to lose if you hash the payload and assume identical content always means identical intent. Let the application establish the action boundary and persist it before execution.

AWS’s idempotent API design article explains why a caller-provided identifier is useful, why the identifier needs caller context, and why recording it must be coordinated with the mutation. Its discussion of late requests and changed intent is especially relevant to long-running workflows. AWS Builders’ Library

Your internal record might include:

{
  "operation_id": "ticket-intent-042",
  "attempt_id": "attempt-003",
  "tenant_id": "team-example",
  "agent_id": "support-worker",
  "approval_ref": "approval-018",
  "provider": "ticket-service-a",
  "operation": "create_ticket",
  "request_digest": "sha256:<canonical-request-digest>",
  "outcome": "unknown",
  "next_action": "reconcile",
  "automatic_write_allowed": false
}

This is an illustrative internal record, not a standard schema. The digest detects changed inputs; it does not define whether two actions have the same business intent. The approval reference points to actual authorization. None of these fields grant permission by themselves.

Where an idempotency key stops helping

A header is useful only when the receiver implements and documents its behavior. Adding Idempotency-Key to an arbitrary API does not create deduplication.

Stripe is a helpful concrete example. Its documented behavior includes replaying the original status and body, including 500 responses; rejecting changed parameters for an existing key; and allowing key removal after at least 24 hours. Reusing a key after it has been pruned can create a new request. Those are Stripe’s rules, not universal rules for every API. Stripe idempotent requests

A replayed error is a reason to investigate the operation, not to mint a fresh key until something returns green.

Then there is the crash between two systems. Your gateway sends an email, the provider accepts it, and your process dies before saving the response. A local “completed operations” table cannot undo that gap. You need the downstream provider’s deduplication or reconciliation support, or an explicit path for an unresolved outcome. Writing a database row before the call moves the crash window; it does not magically close it.

Also check concurrent execution. Two workers can both observe “nothing saved yet.” The operation needs a durable uniqueness boundary and coordinated execution, not just a cache lookup followed by a write.

Try it locally: lose the reply after the commit

The experiment below creates a note, deliberately raises a timeout after committing it, closes the database connection, and reopens the store. Reusing the operation key returns the existing note. Using a fresh key creates a second one.

Save it as retry_lab.py and run python3 retry_lab.py. It uses the Python standard library and a temporary local database. It makes no network calls.

Scope: the note and deduplication record share one SQLite transaction. This demonstrates a local atomic write, not end-to-end delivery to an external API. It does not test a real network timeout, process termination, concurrent workers, key expiry, or a payment provider.

"""Local teaching simulation: no network, credentials, or external writes.
The effect and deduplication record share one SQLite transaction.
This is not an implementation for external payments, email, or API calls.
"""
import json
import sqlite3
import tempfile
from pathlib import Path


def connect(path):
    db = sqlite3.connect(path)
    db.executescript('''
      CREATE TABLE IF NOT EXISTS notes (
        id INTEGER PRIMARY KEY, tenant TEXT NOT NULL, body TEXT NOT NULL);
      CREATE TABLE IF NOT EXISTS operations (
        tenant TEXT NOT NULL, op_id TEXT NOT NULL,
        payload TEXT NOT NULL, note_id INTEGER NOT NULL,
        PRIMARY KEY (tenant, op_id));
    ''')
    return db


def create_note(db, tenant, op_id, body, lose_reply=False):
    payload = json.dumps({'body': body}, sort_keys=True)
    db.execute('BEGIN IMMEDIATE')
    try:
        row = db.execute(
            'SELECT payload, note_id FROM operations WHERE tenant=? AND op_id=?',
            (tenant, op_id)).fetchone()
        if row:
            if row[0] != payload:
                raise ValueError('same operation ID, different payload')
            note_id = row[1]
        else:
            note_id = db.execute(
                'INSERT INTO notes(tenant, body) VALUES (?, ?)',
                (tenant, body)).lastrowid
            db.execute('INSERT INTO operations VALUES (?, ?, ?, ?)',
                       (tenant, op_id, payload, note_id))
        db.commit()
    except Exception:
        db.rollback()
        raise
    if lose_reply:
        raise TimeoutError('effect committed; reply lost')
    return note_id


def run_lab():
    with tempfile.TemporaryDirectory() as directory:
        path = Path(directory) / 'lab.sqlite'
        db = connect(path)
        try:
            create_note(db, 'team-a', 'intent-1', 'Draft note', lose_reply=True)
        except TimeoutError:
            pass
        db.close()
        db = connect(path)  # reopen the durable store after the lost reply
        first = create_note(db, 'team-a', 'intent-1', 'Draft note')
        assert db.execute('SELECT count(*) FROM notes').fetchone()[0] == 1
        print('PASS: lost reply + reopened connection + same key = one note')
        second = create_note(db, 'team-a', 'intent-2', 'Draft note')
        assert second != first
        assert db.execute('SELECT count(*) FROM notes').fetchone()[0] == 2
        print('PASS: fresh key = second note (unsafe if it was only a retry)')
        try:
            create_note(db, 'team-a', 'intent-1', 'Changed note')
        except ValueError:
            print('PASS: changed payload with the original key is rejected')
        else:
            raise AssertionError('payload conflict was not rejected')
        third = create_note(db, 'team-b', 'intent-1', 'Draft note')
        assert third not in (first, second)
        assert db.execute('SELECT count(*) FROM notes').fetchone()[0] == 3
        print('PASS: tenant scopes do not collide')
        db.close()


if __name__ == '__main__':
    run_lab()

All four assertions passed when we ran this teaching example: same-key recovery, fresh-key duplication, changed-payload rejection, and tenant separation. These are local test results, not production reliability numbers.

Put these cases in your integration tests

The local example teaches the shape of the problem. Your adapter still needs tests against the contract of the actual service, using its sandbox or a controlled stub.

Have one execution policy own the retry budget across your agent, SDK and workflow layers. Otherwise each layer can stay within its own modest limit while their combined attempts grow. Count attempts and elapsed time for the whole intended operation, and record the reason whenever automation stops.

Keep service identity attached to the retry contract

A retry policy belongs to a particular service, account and operation. Keep the official endpoint, contract version, status lookup and support route attached to that service’s maintained identity.

This is where a public .agent record from HeadlessDomains.com can help: it gives developers and agents a consistent place to inspect the service and find its linked documentation through CLI and API workflows. Your implementation still has to enforce the retry policy. A name does not deduplicate writes, extend a provider’s key lifetime, or transfer state between providers.

For the wider setup, see How to Make Your API Agent-Ready. For authority and payment records, see Agent Payments Require Identity, Authorization, and Receipts.

When you document your own agent-facing service, publish its retry behavior alongside its official endpoints. Get started at HeadlessDomains.com when you are ready to give that public service record a persistent name.

At your next review, ask one question: if the work finished and the reply vanished, what would this agent do next?

Research note: This article draws on public X, LinkedIn and Hacker News discussions, checked against technical documentation. Quotes retain their source attribution and are not endorsements. The discussion sample is qualitative; it does not establish how common these failures are.