In places where the internet drops often, a point-of-sale app has one hard rule: the shop must never stop selling. The device confirms a sale immediately and syncs it when the network comes back.
That rule leads straight to the classic bug of offline systems: double counting. The device sends a sale, the server saves it, and the response is lost on the way back. The device doesn't know the sale arrived, so it sends it again. If the server saves it twice, stock and cash are both wrong.
These are my notes on the patterns that make "just send it again" always safe.
1. Confirm into a local outbox
The device keeps a small local database with an outbox table. Confirming a sale writes the document and its local receipt number in one transaction, so there's never a gap or a duplicate on the device. The customer gets a receipt right away. Confirming never waits for the network.
A background loop sends the outbox in order, in small batches, with exponential backoff when the server is unreachable.
2. Let the client name the operation
Each document gets a client-generated UUID at the moment it's created. That ID stays the same through every retry, until the server accepts it. On the server it is the document's identity:
- A unique constraint on the ID, so the database itself refuses a second copy.
- A stored result per ID: not just the data, but the answer the server gave.
- A lock on the ID while it is processed, so two copies that arrive at the same time can't both be saved.
-- simplified: lock on the client ID, then reuse a stored answer
SELECT pg_advisory_xact_lock(hashtextextended($1, 0));
SELECT result FROM processed WHERE client_id = $1;
If an answer already exists, the server returns it again. A lost response, two sync loops racing, a manual retry: they all end the same way, with one document and one answer.
A subtle detail: an ID only means "the same document" if everything else matches. The same ID arriving from a different device or for a different kind of document should be refused, not treated as a duplicate.
3. Two numbers: local first, final on the server
Receipt numbers must be unique and gap-free, which offline devices can't guarantee alone. So each document gets two:
- A local number with a per-device prefix (for example
T1-42), which can't collide with another device. - A final number given only by the server, inside the transaction that saves the document. If that transaction rolls back, the number is released, so there are no gaps.
4. Re-check everything, and never block the queue
Late documents arrive after things have changed, so the server checks everything again instead of trusting the device: stock (a unique item may already be sold), totals (re-calculated on the server) and time (a sale counts for the day it was made, with limits on clock skew and very old documents).
The most important rule: one bad document must never block the rest. Each document is processed in its own transaction. A problem becomes a recorded conflict with a clear reason, shown to a manager to resolve, while everything behind it keeps syncing.
5. Test the failures on purpose
These bugs only appear when something goes wrong, so the tests make things go wrong: two syncs at the same time, a lost response followed by a retry, a conflict in the middle of a batch, a document made in one month and synced in the next, a crash right after the outbox write. Good test names read like requirements, for example "sync twice at the same time makes no doubles".
The short version
- The client names the operation with an ID it keeps through every retry.
- Store the answer, not just the data, and return it again for repeats.
- Lock on the ID, so two copies sent at once can't both be saved.
- Give final numbers inside the saving transaction.
- Record conflicts; never let them block the queue.
The same five ideas apply well beyond point of sale: payments, order APIs, webhooks, mobile apps, anything where a request can be sent twice.