Overview
- Architecture: Multiple master nodes, each accepting writes
- Bi-directional: All masters replicate to each other
- Write Availability / Locality: Accept writes in several places (e.g. per region)
- Not True Write Scaling: Every master must still apply every write, so total write capacity doesn’t grow — sharding is what scales writes
- Key Challenge: Requires conflict resolution mechanisms
Architecture Components
Master Nodes
- Write Operations: All masters accept INSERT, UPDATE, DELETE, DDL
- Read Operations: Serve read queries
- Replication: Send and receive changes bidirectionally
- Conflict Resolution: Handle concurrent write conflicts
Replica Nodes (Optional)
- Read-Only: Serve read queries, offload traffic from masters
Topology Types
Ring Topology
- Structure: Masters in circular chain
- Trade-off: Higher latency, ring break affects replication
Star Topology
- Structure: Central hub with spoke masters
- Trade-off: Lower latency, hub failure affects all
Mesh Topology
- Structure: Direct connections between all masters
- Trade-off: Lowest latency, more complex management
Conflict Types
Write-Write Conflicts
- Scenario: Two masters modify same data simultaneously
- Impact: Must determine winning change
Unique Constraint Violations
- Scenario: Two masters insert same unique key
- Impact: Constraint violation on replication
Delete-Update Conflicts
- Scenario: One master deletes, another updates same row
- Impact: Update target doesn’t exist
Schema Conflicts
- Scenario: DDL changes on different masters
- Impact: Schema mismatches, replication breaks
Conflict Resolution Strategies
Last Write Wins (LWW)
- Method: Timestamp-based, most recent wins
- Trade-off: Simple but potential data loss, clock dependency
First Write Wins
- Method: First change to reach node wins
- Trade-off: Non-deterministic across nodes
Version Vectors/Vector Clocks
- Method: Track causal relationships
- Trade-off: Preserves causality but complex, metadata overhead
Application-Level Resolution
- Method: Custom business logic decides winner
- Trade-off: Flexible but requires application changes
CRDTs (Conflict-Free Replicated Data Types)
- Method: Data structures with guaranteed convergence
- Trade-off: No conflicts but limited structure types
Custom Merge Functions
- Method: Intelligent merge of both changes
- Trade-off: Preserves updates but complex design
Advantages
- Write Availability: Multiple nodes accept writes, no single write endpoint (but each node still applies all writes, so capacity doesn’t scale like sharding)
- High Availability: Multiple masters, tolerates failures
- Geographic Distribution: Write locally, low latency
- Read Scalability: Distribute reads across all masters
Disadvantages
- Conflict Complexity: Detection and resolution overhead
- Eventual Consistency: Temporary inconsistency, replication lag
- Operational Overhead: Complex setup, monitoring, debugging
- Performance Cost: Conflict detection, network traffic
- Data Integrity Risks: Potential lost updates, constraint violations
Use Cases
Global Applications
- Example: Social networks, collaboration tools
- Benefit: Low latency writes from any location
High Write Throughput
- Example: IoT ingestion, financial trading
- Benefit: Distribute write load
Always-On Systems
- Example: E-commerce, SaaS platforms
- Benefit: Continue operations during failures
Collaborative Applications
- Example: Google Docs, collaborative editing
- Benefit: Concurrent edits without blocking
Implementation Examples
MySQL Group Replication
- Multi-primary mode, first committer wins
PostgreSQL with BDR
- Row-level conflict detection, configurable strategies
Cassandra
- Masterless architecture, last write wins
CouchDB
- Document version trees, application-level resolution
MongoDB Multi-Region (not multi-master)
- Each replica set has a single primary; multi-region writes come from zone sharding (each shard’s primary placed in its region), not from multiple masters
Monitoring Metrics
- Replication Lag: Delay between masters
- Conflict Rate: Frequency and resolution time
- Write Distribution: Load balance across masters
- Data Consistency: Periodic validation
Best Practices
Design
- Partition Data: Minimize overlapping writes by user/region/tenant
- Idempotent Operations: Safe to retry
- Appropriate Resolution: Match strategy to business needs
Application
- Conflict Awareness: Design for eventual consistency
- Local Writes: Route to nearest master
- Monitoring: Track conflicts and alert
Conflict Prevention
- Data Partitioning: User-based, region-based, tenant-based
- Application Routing: Sticky sessions, locality routing
- Time-Based Isolation: Leverage time zones, schedule operations
Summary
- Pattern: Multiple masters, bidirectional replication
- Strengths: Low local-write latency, high availability, geographic distribution (not write scalability: every master applies every write)
- Weaknesses: Conflict complexity, eventual consistency, operational overhead
- Best For: Global applications, high write throughput, always-on systems
- Success Factor: Proper partitioning and conflict resolution strategy