The scariest bug I have shipped in Web3 work was not a smart contract bug. It was an indexer that trusted the chain tip. For weeks it looked perfect. Then a deeper than usual reorg happened, a handful of blocks got rewritten, and our indexer happily kept the old version of events that no longer existed while also writing the new ones. The numbers were wrong and nothing crashed, which is the worst combination. Handling chain reorgs in an indexer is the part of this work that separates a demo from something you can put real money behind.
Why reorgs happen at all
A blockchain is not a clean append-only log at the tip, even though it behaves like one a few blocks back. When two miners or validators produce competing blocks at the same height, the network temporarily disagrees about what the latest block is. The disagreement resolves by one fork winning and the other being abandoned. Any transactions that lived only on the losing fork get unwound. Most reorgs are one or two blocks deep and resolve in seconds. Deeper ones are rare, but rare is not never, and your indexer has to survive the rare case.
The mental shift is to stop thinking of recent blocks as facts and start thinking of them as probably true and getting truer with age. A block ten deep is settled for almost all purposes. A block at the very tip is a guess.
Confirmations buy you safety, latency is the price
The simplest defense is to not index the tip at all. Stay some number of blocks behind head and only process blocks once they have that many confirmations on top of them. Pick the depth based on the chain and the stakes. A few blocks is plenty for low-value display data. For anything that triggers payouts or accounting I want more margin, because the cost of being wrong is real.
The tradeoff is honest and unavoidable. More confirmations means your data lags further behind the chain. Less lag means more exposure to reorgs. There is no setting that gives you both, so the right move is to be explicit about it. I often run two paths: a fast optimistic view at or near the tip that the UI can show with a pending label, and a slower confirmed view that everything financial reads from.
- Decide the confirmation depth per use case, not once for the whole system
- Show users a clear pending state for anything below your confirmed threshold
- Never let payouts, balances, or settlement read from unconfirmed data
Make your writes survive a rollback
Confirmations reduce reorgs but do not eliminate them, so you still need to be able to undo. The cleanest approach I have used is to version every write by the block it came from. When a reorg is detected, you find the last block both your indexer and the chain agree on, delete or revert everything written after it, and replay forward over the new canonical blocks. This only works if your writes are reversible and addressable by block, so design for that from the start rather than bolting it on.
The other property you want is idempotency. Replaying the same block twice must produce the same state as processing it once. If your handler increments a counter, a replay double counts and you are corrupted again. The fix is to make writes set absolute values derived from the event rather than relative deltas applied blindly, or to key each write so a repeat is a no-op overwrite instead of an addition.
Detecting the reorg in the first place
You cannot roll back a reorg you did not notice. The check is cheap: for each new block you process, verify that its parent hash matches the hash of the block you already have at the previous height. If it does not match, a reorg happened somewhere behind you and you walk backward comparing hashes until you find the common ancestor. Store block hashes, not just numbers. An indexer that only tracks block numbers is blind to reorgs by construction, because the numbers look continuous even when the contents changed underneath them.
I learned to log every detected reorg with its depth. Over time that log tells you whether your confirmation depth is actually conservative enough for the chain you are on, instead of you guessing from a blog post. If you see reorgs regularly reaching close to your threshold, your threshold is too shallow.
The quiet failure mode to fear most
Crashes are easy. Crashes page you and you fix them. The dangerous failure is the one where a reorg silently leaves stale data behind and everything keeps running. That is why I treat idempotency and hash-based detection as non-negotiable rather than nice to have. An indexer that loudly falls over on a reorg is annoying. An indexer that serves wrong balances with full confidence is a liability, and by the time someone notices, the bad data has already flowed into reports, dashboards, and possibly transactions made on the strength of it.
Build the rollback path before you need it, test it by deliberately feeding your indexer a forked history, and assume that the deep reorg you think will never happen will happen on the day you are not watching. That assumption has saved me more than any clever optimization ever has.
Building something where this matters?
I am open to senior full-stack, Web3, or AI engineering roles, fully remote and any timezone. If the hard part of your product is fighting you, that is the work I like.
Get in touch →