Distributed Idempotency Keys: The Stripe Architecture

The Network Boundary Problem

In distributed client-server systems, the network is fundamentally unreliable. When a client sends a mutative request (POST /v1/charges), there are three possible points of failure:

1. The request packet never reaches the server (No charge made).
2. The server crashes while processing the charge (Unknown state).
3. The server processes the charge successfully, but the connection drops before the HTTP 200 response reaches the client (Customer charged, but client believes it failed).

If the client retries blindly on step 3, the customer will be charged twice.

---

The Solution: The Idempotency-Key Header

To solve this, modern APIs (spearheaded by Stripe) require the client to attach a unique UUID in the request headers:

POST /v1/charges HTTP/2
Host: api.buildrestapi.com
Idempotency-Key: 7f3b890a-1122-4455-8899-abcdef123456
Content-Type: application/json

{
  "amount": 4900,
  "currency": "usd",
  "customer_id": "cus_9901"
}

---

The 4-State Atomic Lifecycle in Redis

Client POST request (with Idempotency-Key)
                  │
                  ▼
         Check Redis for Key
        ┌─────────┴─────────┐
        │                   │
     Key Exists        Key Not Found
        │                   │
  Is Response Cached?    Acquire Lock via SETNX
  ├── Yes ➔ Return Cache ├── Success ➔ Execute Business Logic
  └── In-flight ➔ 409    └── Failure ➔ Retry after 50ms

1. Atomic Lock Acquisition

The server attempts to acquire a lock using Redis SET with the NX (Not Exists) flag and a short expiration time (e.g. 120 seconds):

SET lock:idempotency:7f3b890a <worker_pid> NX EX 120

2. Handling In-Flight Requests

If SETNX returns 0, another worker is currently processing this identical request. The server should either return 409 Conflict with a Retry-After: 1 header, or hold the socket open and poll Redis until the result is written.

3. Payload Hashing Verification

To prevent attackers from reusing the same idempotency key for different parameters, compute a cryptographic SHA-256 hash of the request payload body and path:

const payloadHash = crypto
  .createHash('sha256')
  .update(`${req.method}:${req.url}:${JSON.stringify(req.body)}`)
  .digest('hex');

If an incoming request arrives with a matching key but a differing payload hash, return 422 Unprocessable Content with an error: “Idempotency key was previously used with different request parameters.”

4. Response Caching

Once the transaction succeeds, the server caches:

• HTTP Status Code (e.g. 201 Created)
• Response Headers (Content-Type, Location)
• Serialized JSON response body
• TTL: 24 to 48 hours.

When subsequent retries arrive within 24 hours, the server serves the cached response instantly without re-executing any business logic!