System Design Fundamentals

Complete guide to designing scalable systems

What is System Design?

The process of defining architecture, components, modules, interfaces, and data for a system to satisfy specified requirements.

Why It Matters:

• Scalability → Handle growing user base
• Reliability → System stays operational
• Performance → Fast response times
• Maintainability → Easy to update and debug

---

System Design Process

1. Requirements Gathering (~5 min)

• Functional requirements (what the system does)
• Non-functional requirements (how well it performs)
• Constraints and assumptions

2. Capacity Planning (a few min, part of requirements)

• Estimate scale (users, data, traffic)
• Calculate storage and bandwidth needs
• Identify bottlenecks

3. High-Level Design (~10 min)

• Draw major components
• Show data flow
• Choose technology stack

4. Detailed Design (~15 min)

• Database schema
• API design
• Algorithms and data structures

5. Scale and Optimize (~10 min)

• Identify bottlenecks
• Add caching, load balancing
• Discuss trade-offs

---

Core Technologies

Databases

Type	Use Case	Examples
SQL	ACID transactions, complex queries	PostgreSQL, MySQL
NoSQL Document	Flexible schema, JSON data	MongoDB, CouchDB
NoSQL Key-Value	Caching, session storage	Redis, DynamoDB
NoSQL Graph	Relationships, social networks	Neo4j, Amazon Neptune
Time-Series	Metrics, IoT data	InfluxDB, TimescaleDB

Caching Strategies

• Browser Cache → Static assets (CSS, JS, images)
• CDN → Geographic distribution
• Application Cache → Redis, Memcached
• Database Cache → Query result caching

Load Balancing

• Round Robin → Equal distribution
• Weighted → Based on server capacity
• Least Connections → Route to least busy server
• Geographic → Route to nearest server

Message Queues

• Apache Kafka → High-throughput streaming
• RabbitMQ → Reliable message delivery
• Amazon SQS → Managed queue service
• Redis Pub/Sub → Real-time notifications

---

Scalability Patterns

Horizontal Scaling

• Add more servers
• Better fault tolerance
• Linear cost increase
• Complexity in coordination

Vertical Scaling

• Upgrade existing hardware
• Simpler to implement
• Single point of failure
• Limited by hardware constraints

Database Scaling

Read Replicas

[Write] → [Master DB] → [Replica 1]
                    └→ [Replica 2]
[Read]  ← [Load Balancer] ← [Replica 3]

Sharding

Users A-M → [Shard 1]
Users N-Z → [Shard 2]

Federation

User Service → [User DB]
Post Service → [Post DB]
Chat Service → [Chat DB]

---

Reliability Patterns

Circuit Breaker

import time
 
class CircuitBreaker:
    def __init__(self, failure_threshold=5, timeout=60):
        self.failure_count = 0
        self.failure_threshold = failure_threshold
        self.timeout = timeout
        self.state = "CLOSED"  # CLOSED, OPEN, HALF_OPEN
    
    def call(self, func, *args, **kwargs):
        if self.state == "OPEN":
            if time.time() - self.last_failure > self.timeout:
                self.state = "HALF_OPEN"
            else:
                raise Exception("Circuit breaker is OPEN")
        
        try:
            result = func(*args, **kwargs)
            if self.state == "HALF_OPEN":
                self.state = "CLOSED"
                self.failure_count = 0
            return result
        except Exception as e:
            self.failure_count += 1
            if self.failure_count >= self.failure_threshold:
                self.state = "OPEN"
                self.last_failure = time.time()
            raise e

Retry with Exponential Backoff

import time
import random
 
def retry_with_backoff(func, max_retries=3, base_delay=1):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise e
            
            delay = base_delay * (2 ** attempt) + random.uniform(0, 1)
            time.sleep(delay)

Bulkhead Pattern

• Isolate critical resources
• Separate thread pools for different operations
• Prevent cascade failures

---

Monitoring and Observability

The Three Pillars

1. Metrics (What is happening?)

- Response time (p50, p95, p99)
- Throughput (requests/second)
- Error rate (4xx, 5xx responses)
- Resource utilization (CPU, memory, disk)

2. Logs (What happened?)

{
  "timestamp": "2024-01-01T10:00:00Z",
  "level": "ERROR",
  "service": "user-service",
  "traceId": "abc123",
  "message": "Failed to authenticate user",
  "userId": 12345,
  "error": "Invalid token"
}

3. Traces (How did it happen?)

Request → API Gateway → User Service → Database
  100ms     10ms          80ms          70ms

Tools

• Metrics: Prometheus + Grafana
• Logs: ELK Stack (Elasticsearch, Logstash, Kibana)
• Traces: Jaeger, Zipkin
• APM: New Relic, Datadog, AppDynamics

---

Common System Design Questions

Social Media (Instagram/Twitter)

• User timeline generation
• News feed algorithm
• Image/video storage and delivery
• Real-time notifications

Chat Systems (WhatsApp/Slack)

• Real-time messaging
• Message delivery guarantees
• Group chat management
• Online presence

E-commerce (Amazon/eBay)

• Product catalog search
• Shopping cart management
• Payment processing
• Inventory management

Video Streaming (YouTube/Netflix)

• Video encoding and storage
• Content delivery network
• Recommendation system
• Live streaming

Ride Sharing (Uber/Lyft)

• Real-time location tracking
• Matching algorithm
• Dynamic pricing
• Route optimization

---

Interview Tips

Time Management

• 5 min: Requirements clarification
• 10 min: High-level design
• 15 min: Detailed design
• 10 min: Scale and optimize
• 5 min: Wrap up and questions

Communication

• Think out loud
• Ask clarifying questions
• Explain trade-offs
• Start simple, then add complexity

Common Mistakes

• Jumping to implementation details
• Not asking about scale requirements
• Ignoring non-functional requirements
• Over-engineering the solution

What Interviewers Look For

• Structured thinking → Systematic approach
• Trade-off analysis → Understanding implications
• Scalability awareness → Design for growth
• Real-world knowledge → Practical experience