Idempotency & Exactly-Once Delivery
The problem
When a network is unreliable, a client can never be fully sure whether a request it sent actually succeeded — the response might have been lost even though the operation completed — and naive retries risk applying the same action (charging a card, sending an email) more than once.
Why now
Message queues and event-driven architecture already introduced 'at-least-once' delivery as a common, practical guarantee, which means duplicates are a real possibility, not a hypothetical one. Algebraic structures already named the exact property that fixes this — idempotency, f(f(x)) = f(x) — and this topic is where that abstract algebraic property becomes an engineering requirement.
Mental model
An idempotent operation gives the same result no matter how many times it's applied — 'set balance to $100' is idempotent, 'add $100 to balance' is not, even though both might look like reasonable ways to express the same intent. Exactly-once delivery is expensive and often impossible to guarantee at the network level, so the practical fix is designing operations to be idempotent and delivering at-least-once — retries become safe because repeating them changes nothing.
Requires
- Message Queues & Event-Driven ArchitectureDistributed Systems
The duplicate-delivery problem this topic solves arises directly from the at-least-once guarantees typical of the message queues already covered; you need that realistic delivery guarantee in mind before idempotency's necessity is clear.
- Algebraic StructuresMathematical Thinking
Idempotency is precisely the algebraic property already named there (f(f(x)) = f(x)) — this topic doesn't introduce a new concept, it applies the exact one already defined to a concrete, high-stakes engineering problem.
Used in
- Payment processing APIs (Stripe) requiring idempotency keys on every charge request
- Retry logic in any client calling an unreliable network service
- Message queue consumers designed to safely process a message more than once
- Database upserts (INSERT ... ON CONFLICT) as an idempotent alternative to plain INSERT
Projects
- Design a payment endpoint that accepts an idempotency key, ensuring a retried request with the same key never double-charges, even if the first request's response was lost
- Rewrite a non-idempotent operation ('increment inventory by 1') as an idempotent one ('set inventory to N', or with a deduplication check) and explain the tradeoff
Examples
- Stripe requires an Idempotency-Key header on charge requests specifically so a client's retry after a lost response never results in a double charge
- A message consumer that processes the same 'order shipped' event twice should produce the same end state both times, not send two shipping notifications
Resources
Mastery checklist
- I can define idempotency precisely and give a non-idempotent operation's idempotent equivalent
- I can design an idempotency-key mechanism for a retry-safe API endpoint
- I can explain why exactly-once delivery is hard, and why idempotency plus at-least-once is the practical alternative
- I can identify a non-idempotent operation in real code and fix it