Atomic Commits in Distributed Systems: TCC, Sagas & The Reality
The Context: "The Zombie Transaction"
It was mid-2024. A customer was transferring $1,000 from their savings account in the "Legacy Core" to their "Investment Wallet" in the modern Azure cloud. The "Legacy" account was successfully debited, but the network failed before the "Wallet" could be credited.
The $1,000 vanished into a "Transaction Vortex". The customer saw a $1,000 loss, the "Wallet" saw nothing, and the database reconciler spent 48 hours manually fixing the mess. This is the Atomic Commit problem. This article explores the architecture of Distributed Transactions and why "Sagas" and "TCC" are the only way to build a reliable global bank.
The Problem Statement
In a monolithic world, COMMIT is a single database command. In a distributed world (Microservices A + B + C), there is no "Global Commit".
- Goal: 100% Consistency (No money is ever "Lost").
- Metric: Zero manual reconciliation.
- Challenge: The "CAP Theorem". You can have Consistency or Availability, but you can't have both during a network partition.
Visual Architecture: The Saga Orchestrator
The Core Logic: Beyond the 2-Phase Commit (2PC)
1. The Saga Pattern (Distributed Transactions)
Since we can't "Lock" three different databases, we use a Saga. A Saga is a sequence of local transactions. Each local transaction has a corresponding Compensating Transaction (the Undo operation).
- Choreography: Each service emits an event that the next service listens to.
- Orchestration: A central "Brain" (Orchestrator) manages the state and tells each service what to do and when to "Compensate".
2. TCC (Try-Confirm-Cancel)
This is a more "Active" version of the Saga.
- Try: Reserve the resources (e.g., "Holding" the money).
- Confirm: Finalize the operation (e.g., "Committing" the hold).
- Cancel: Release the resources if something fails. This is much safer for banking because the money is "Held" immediately, preventing a double-spend during the saga execution.
3. The Idempotency Key
In a Saga, a network timeout might happen during the "Confirm" or "Cancel" phase. The Orchestrator will "Retry" the call.
- Rule: Every service in the saga MUST be Idempotent. If the "Cancel" request is received twice, the second one should do nothing.
The "Baigan" Take: Insights from the Trenches
Look, people love "2PC" (Two-Phase Commit) because it's "Simple" and ACID-compliant. But baigan ka logic hai wo (that's nonsense) when you're running 10,000 TPS across three microservices.
The 'Lock' Avalanche. The biggest mistake I've seen is holding database locks across microservices. If Service A locks a row in Oracle and waits for Service B to respond over a slow network, you've just created a "Distributed Deadlock". Your whole platform will grind to a halt.
- Rule: Use "Status Columns". Change the status to
PENDINGand commit. Release the DB lock. Use a background job to "Move" the state forward. - Rule: "Acceptable Inconsistency" is better than a "System Halt". For 99.9% of transactions, a Saga will finish in < 200ms. For the 0.1% of failures, the "Compensation" will fix it in < 5s. That's better than 100% of users waiting 10s for a 2PC commit.
Performance & Implementation Checklist
- State Machine: Use a framework like Temporal.io or Camunda for your orchestrator. Don't write your own
if-elsestate machine. - Timeout Logic: Every step in a Saga must have a timeout. If Service B doesn't respond in 5 seconds, "Cancel" the Saga.
- Observability: Trace the "Saga ID" across all services. You must be able to see the "Timeline of a Saga" in Jaeger to find where it "Hanged".
- Reconciliation Engine: No Saga is perfect. A background job must scan for "Hanging Sagas" older than 30 minutes and trigger a manual alert or a final "Auto-Cancel".
Conclusion
Atomic Commits in a distributed world are an "Illusion". We must stop trying to make the network "Perfect" and start making our software "Recoverable". By mastering TCC and Sagas, you build a banking platform that is not just "Durable", but also "Resilient" to the chaos of the internet. Commit with intent, compensate with precision, and never lose a dollar again.
Verified for Technical Nexus v1.9.0 | Part of the System Design Vault archive.