Exactly-Once Delivery Does Not Exist. Your Consumer Has To Be Idempotent.

Somewhere in your queue vendor's documentation there is a page about exactly-once delivery. It may be a premium feature. It is not a feature.

Exactly-once delivery is a proven impossibility in distributed computing, and the result follows from the Two Generals problem and the FLP impossibility result.^[1] This is not an unsolved engineering challenge that a sufficiently funded team will crack. It is in the same category as sorting faster than the comparison bound.

Which is useful to know, because it tells you where the work has to happen, and it is not in the broker.

Why It Is Impossible, Briefly

The Two Generals problem: two parties must agree on a plan by exchanging messages over a channel that can lose them. Any message can be lost, including the acknowledgement, and including the acknowledgement of the acknowledgement. There is no finite exchange that leaves both sides certain. You can make the probability of confusion small; you cannot make it zero.

Map that onto a queue. A consumer processes a message and sends an acknowledgement. The acknowledgement is lost. The broker, having heard nothing, redelivers. From the broker's position, “processed but the ack was lost” and “never processed” are literally indistinguishable, because the only evidence that would distinguish them is the message that went missing.

So the broker must choose. Redeliver, and risk duplicates. Do not redeliver, and risk loss. That choice is the entire taxonomy: at-least-once or at-most-once. There is no third option, which is the same shape of forced choice as the one CAP describes during a partition.

Effectively-Once, Which Is Real

The achievable target is effectively-once: at-least-once delivery from the broker, plus idempotent processing in the consumer, so that a duplicate run changes nothing.^[1]

The duplicate still arrives. It simply stops mattering. And note where the responsibility landed: on your consumer, not on the queue you are paying for. The mature response is not to hunt for a broker that promises exactly-once, it is to make duplicates harmless.^[1]

Which is the same mechanism as idempotency keys on an HTTP endpoint, arriving from a different direction. A message carries an identifier, the consumer records that it processed that identifier in the same transaction as the effect, and a repeat is recognised and skipped. Same problem, same solution, usually built twice by different teams who never noticed.

The Dead Letter Queue Is Not a Safety Net

This reframe is worth the price of admission. A dead letter queue is a diagnostic instrument, and its depth is a leading indicator that your processing objective is already broken.^[1]

The failure mode is treating it as a place failures go so they stop paging you. Messages exhaust their retries, land quietly in the DLQ, nothing alerts, and the queue accumulates work that was supposed to happen and did not. Orders unfulfilled. Emails unsent. Nobody finds out until a customer does.

Alert on DLQ depth, and treat a non-zero depth as an incident rather than a backlog to triage on Thursday. This is exactly the kind of signal that needs to be visible rather than merely recorded, which is the argument in observability: data you technically have but nobody looks at is not a capability.

Retries Need Jitter Here Too

Exponential backoff with jitter, for the same reason it belongs in a rate limiter's Retry-After.^[1] Consumers usually fail for a shared reason: a dependency is down. So they all fail at once and, without randomisation, they all retry at once, recreating the load spike that caused the failures.

Backoff without jitter converts one outage into a series of synchronised ones. The same pattern appears in rate limiting and in circuit breakers, and it is the single most reusable piece of knowledge in this whole area: anything that makes many clients act on the same schedule is a load generator.

The Case a Queue Cannot Fix: The Outbox

One specific problem worth naming, because it catches people who have done everything else right.

You need to save a record and publish an event about it, and both must happen or neither. But your database and your broker are two separate systems, so there is no transaction spanning them. Write the database and then publish, and a crash between them loses the event. Publish and then write, and a crash loses the data while the world has been told about it.

The transactional outbox resolves this: write the event into an outbox table inside the same database transaction as the data change, and have a separate process read that table and publish from it.^[1] Now the commit and the intent to publish are atomic, because they are one write to one system. The publishing step is at-least-once, which is fine, because your consumers are idempotent.

The pattern exists because atomicity across two systems is not available. It is a workaround for a genuine impossibility rather than a nicety, which is why it keeps being reinvented by teams who assumed the queue would handle it.