ACID Guarantees in a Leaderless World: CameoDB's Transaction Model

The ACID Challenge in Leaderless Systems

Traditional databases achieve ACID guarantees through consensus algorithms like Raft or Paxos. These protocols elect a leader to serialize writes, ensuring consistency across replicas. But consensus comes with costs, leader election latency, quorum requirements, and limited scalability.

CameoDB takes a different approach. Instead of global consensus, it provides ACID guarantees at the storage engine level through redb's transactional model. Each node maintains ACID compliance for local writes, while the distributed layer handles eventual consistency across the cluster. This tradeoff delivers local transactional integrity with global scalability.

Atomicity: All-or-Nothing Writes

Atomicity ensures that a transaction is either fully applied or not applied at all. CameoDB guarantees atomicity through redb's MVCC (multi-version concurrency control) transaction system. Every write operation occurs within a redb transaction, and if any step fails, the entire transaction rolls back.

// Atomic write with rollback on failure
txn = db.begin_write()
result = match execute_write_sequence(txn) {
 Ok(()) => txn.commit(),
 Err(e) => txn.abort(), // Rolls back all changes
}

The seven-step write sequence (sequence ID, WAL, data table, Tantivy index, commit) executes atomically. If the Tantivy index update fails, the redb transaction aborts, leaving the WAL and data table untouched. No partial state, no orphaned writes.

Consistency: Invariants Across Storage

Consistency ensures that the database transitions from one valid state to another. CameoDB maintains consistency through schema validation and cross-engine synchronization. The storage engine enforces type constraints, field requirements, and relationship invariants.

The hybrid architecture maintains consistency between redb and Tantivy. The atomic write sequence ensures that both storage engines reflect the same logical state after each commit. The recovery process replays uncommitted WAL records into Tantivy, guaranteeing eventual consistency after crashes.

Isolation: Concurrent Operation Safety

Isolation ensures that concurrent transactions don't interfere with each other. CameoDB provides isolation through redb's MVCC system. Readers operate on snapshots, seeing a consistent view of the database at a point in time, unaffected by concurrent writes.

The system supports multiple isolation levels:

Read Committed

Readers see only committed data. Uncommitted transaction changes are invisible.

Repeatable Read

Readers see a consistent snapshot throughout the transaction, even if other commits occur.

Writes are serialized through the storage engine's transaction manager. Concurrent writes to the same document are serialized by redb's locking mechanism, preventing lost updates and race conditions.

Durability: Surviving Failures

Durability ensures that committed transactions survive failures. CameoDB provides durability through the write-ahead log (WAL) and configurable fsync behavior. Every write operation is first recorded in the WAL before being applied to the data tables.

// WAL ensures durability before application
wal.insert(seq_id, serialized_op) // Durable
data.insert(id, json_blob) // Applied
writer.add_document(...) // Indexed
txn.commit() // With optional fsync

The WAL survives crashes, power failures, and unclean shutdowns. On recovery, CameoDB replays uncommitted WAL records to restore the database to a consistent state. Configurable fsync allows you to choose between maximum durability (fsync on every commit) or higher throughput (fsync on smart commit intervals).

Distributed Consistency Model

CameoDB's distributed layer provides eventual consistency across the cluster. Writes are routed to the node that owns the shard based on consistent hashing. Each node maintains ACID compliance for its local writes. Changes propagate to other nodes through the gossip-based peer discovery and state synchronization.

This model means that read-your-writes consistency requires using routing keys. If you write a document with a routing key, subsequent reads with the same key route to the same node, ensuring you see your write. Without routing keys, reads may return stale data from other nodes until gossip synchronization completes.

Conflict Resolution

In a leaderless system, concurrent writes to the same document on different nodes can create conflicts. CameoDB uses a conflict-free replicated data type (CRDT) approach for metadata, ensuring eventual consistency without coordination. For user data, the system relies on the application layer to handle conflicts through version vectors or application-specific resolution strategies.

The sequence ID provides a total order for writes within a single node. Combined with timestamps and node IDs, the system can detect and report conflicts to the application layer for resolution.

The Tradeoffs

CameoDB's transaction model makes specific tradeoffs:

Strong Local ACID

• Full ACID on single node
• Fast local transactions
• No leader election overhead
• Predictable latency

Eventual Global Consistency

• Read-your-writes requires routing keys
• Conflicts need application resolution
• Stale reads possible without keys
• Gossip propagation delay

The Takeaway

CameoDB delivers ACID guarantees through redb's transactional storage engine, providing strong consistency at the node level. The distributed layer provides eventual consistency across the cluster, enabling linear scalability without consensus overhead. The result is a database that combines transactional integrity for local operations with global scalability for distributed workloads. Use routing keys for read-your-writes consistency, or embrace eventual consistency for maximum throughput.