Stripe Connect escrow for a marketplace: the payout guards that actually hold

CTCGX is a peer-to-peer trading card marketplace I built solo in about three months. Buyers pay into escrow, and the seller is paid out through a Stripe Connect transfer once the card ships and the order clears. The payout path is the part of a marketplace that has to be right, because a mistake there either strands a seller's money or pays it out twice. This is how I built that path, starting with the bug that taught me the most about it.

The errors pointed at the wrong key

Stripe blocked one of my production API keys. Escrow payouts started failing. Dev and staging worked fine, so I could not reproduce the failure locally and had to debug against real transactions in production.

I followed Stripe's unlock instructions, rotated the key, and redeployed. Payouts kept failing. The error messages referenced the new key, so every diagnostic I could run told me the configuration was correct. Nothing I could observe pointed at the actual cause. When the logs and the error text all agree and all of them are wrong, the problem is usually state you cannot see.

Idempotency was working exactly as designed

The original payout had gone out with an idempotency key tied to that transfer. Stripe processed it under the old, blocked key, and it failed. Stripe caches an idempotency key together with its response for 24 hours, so that a retried request returns the original result instead of running twice. My retry logic did the correct thing. Same transfer, same idempotency key. Stripe saw the key, returned the cached failure, and never reached the new API key at all. The error surfaced the new key because that is what my request carried, even though Stripe never used it to make the call.

The bug was not in my code, and it was not in Stripe's caching. Both were working as designed. It was the combination. Correctness worked against me, because the cached state was a failure I wanted to retry past. An idempotency key is shared state with a third party that has its own caching behavior, and the answer was not in my logs. It was invisible state on Stripe's side.

You can read Stripe's own description of this in the idempotent requests documentation. The 24-hour window is stated plainly. I had read it before, and I still did not connect it to a rotated key until I stopped trusting the error text.

Versioning the key, and what the key does not do

The fix was to version the idempotency key, transfer-${orderId}-v${failureCount}. The version is locked in by the user action, hitting retry, rather than by a clock. I considered a Date.now() key and rejected it, because a timestamp makes every attempt a brand-new operation and throws away the protection idempotency is there to provide. I considered a per-hour key and rejected that too, because the hour boundary is arbitrary and still leaves a race inside the window.

One distinction got muddled the first time I explained this, and a careful reader will catch it. The versioned idempotency key is not what stops a double payout. Its only job is deciding whether Stripe replays a cached response. In the retry path its real value is narrow. It guards the moment when two retries fire close enough together that neither has seen the other complete, and it collapses those two into one operation. It protects against a race between near-simultaneous requests, and it does nothing about a duplicate that arrives after the first one has already finished.

Checking against Stripe's own record

The durable protection against paying a seller twice is a check against Stripe's own record of truth. Before creating a transfer, I call transfers.list filtered by the order's transfer_group and skip the create if a transfer already exists. Stripe's model for this, holding funds on a charge and paying out with a separate transfer, is documented under separate charges and transfers.

This check holds because it reads Stripe's state rather than a cache or anything in my process memory. If the order already paid out, the list call sees the existing transfer and no second one is created. The idempotency key cannot do this job. At the instant two requests race, no transfer exists yet for either of them to find, so there is nothing in the list to catch. The two guards cover different moments, and I need both.

Moving the race guard upstream

Rather than lean on Stripe to collapse two concurrent requests, admin retries go through a BullMQ queue that refuses to run two jobs with the same ID. The versioned key feeds a deterministic job ID, so a double-click or a double-fired retry becomes one job before it ever reaches Stripe. The versioning scheme and the queue reinforce each other, because the same value that controls Stripe's replay also deduplicates the job.

One caveat matters here, because it changes how much weight the queue can carry. BullMQ job-ID deduplication is time-bounded. Once a completed job is cleaned up, on whatever removeOnComplete retention I configure, re-adding the same job ID runs fresh. So the queue deduplication is a window, in the same way Stripe's 24-hour cache is a window.

Webhooks are a separate path

Idempotent webhook handling guards a different problem, and it is easy to file it under the payout race by mistake. Stripe delivers webhooks at least once, so duplicate events are expected rather than exceptional. Making the handlers idempotent means a replayed transfer event or charge.succeeded does not drive a second state transition in the order state machine. This has nothing to do with two payout requests racing. It is protection against Stripe's delivery semantics, and it belongs to the state machine, not to the transfer create.

Backoff and the dead-letter queue

Exponential backoff and a dead-letter queue with admin observability are reliability work, not duplication control. They are the reason a transient failure gets another attempt instead of dying quietly, and the reason a terminally failed transfer becomes visible to an admin instead of getting lost. I keep them in a separate category from the deduplication layers on purpose, because it is tempting to count reliability machinery as protection against double payouts when it provides none.

Where the permanent invariant has to live

Count the guards and it looks like five layers stand between an order and a double payout. Look closer and four of them are either windowed or scoped to a different failure. Stripe's idempotency cache lasts 24 hours. The BullMQ job-ID deduplication lasts as long as the completed job is retained. Webhook idempotency protects the state machine rather than the payout race. Backoff and the dead-letter queue protect reliability. The only guard that holds no matter how much time passes is my own database state together with the transfers.list check.

So the hard invariant, that an order pays out once and never again, cannot live in the queue or in Stripe's cache. It has to live in persisted state I control, either a unique constraint or a status column I check inside the same transaction that records the transfer. Five layers is reassuring right up until you notice that the permanent one is a single layer, and that is the layer I would tell anyone building this to be most rigorous about.

Related reading

Architecture

How I test Postgres row-level security for tenant isolation