Load Balancing
Load balancers distribute incoming traffic across multiple servers to improve availability, reliability, and performance.
Benefits
| Benefit | Description |
|---|---|
| High Availability | If one server fails, others handle traffic |
| Scalability | Add servers to handle more load |
| Performance | Distribute work evenly across servers |
| Flexibility | Perform maintenance without downtime |
| Security | Hide backend server IPs, absorb DDoS |
Layer 4 vs Layer 7 Load Balancing
Layer 4 (Transport Layer)
Routes based on IP address and TCP/UDP port.
Operation:
- Examines source/destination IP and port
- No inspection of packet contents
- Uses NAT to forward packets
Advantages: Very fast, low resource usage Disadvantages: Cannot route based on content (URLs, headers, cookies)
Layer 7 (Application Layer)
Routes based on HTTP content: URL paths, headers, cookies.
Operation:
- Terminates TCP connection
- Inspects HTTP request
- Makes routing decision based on content
- Opens new connection to backend
Advantages: Smart routing, request modification, SSL termination Disadvantages: Higher latency, more resource intensive
Comparison
| Aspect | Layer 4 | Layer 7 |
|---|---|---|
| Speed | Faster | Slower |
| CPU usage | Low | Higher |
| Content inspection | No | Yes |
| URL-based routing | No | Yes |
| SSL termination | No | Yes |
| Request modification | No | Yes |
| WebSocket support | Basic | Full |
| Sticky sessions | IP-based only | Cookie-based |
Load Balancing Algorithms
Round Robin
Distributes requests sequentially across servers.
Advantages: Simple, even distribution when servers are equal Disadvantages: Ignores server capacity and current load
Weighted Round Robin
Assigns weights based on server capacity.
Use case: Heterogeneous server fleet (different CPU/memory)
Least Connections
Routes to server with fewest active connections.
Advantages: Adapts to varying request duration Disadvantages: Does not account for connection weight
Least Response Time
Routes to server with fastest response time.
Advantages: Optimizes for user experience Disadvantages: Requires constant health monitoring
IP Hash
Hashes client IP to consistently route to same server.
Advantages: Session persistence without cookies Disadvantages: Uneven distribution if IPs are clustered
Algorithm Comparison
| Algorithm | Best For | Drawback |
|---|---|---|
| Round Robin | Homogeneous servers, stateless apps | Ignores server load |
| Weighted Round Robin | Mixed server capacities | Manual weight tuning |
| Least Connections | Long-lived connections | Overhead of tracking |
| Least Response Time | Latency-sensitive apps | Monitoring overhead |
| IP Hash | Session persistence | Uneven distribution |
| Random | Simple deployments | Unpredictable |
Session Persistence (Sticky Sessions)
Ensures user requests go to the same server.
Methods
| Method | Mechanism | Advantages | Disadvantages |
|---|---|---|---|
| Cookie-based | LB inserts server ID in cookie | Survives IP changes | Requires cookie support |
| IP-based | Hash client IP | No cookie needed | Breaks with NAT/proxies |
| URL parameter | Server ID in URL | Works everywhere | URL modification |
Recommended approach: Design stateless applications and store sessions in Redis/database.
Health Checks
Load balancers detect unhealthy servers through health checks.
Health Check Types
| Type | Verification | Use Case |
|---|---|---|
| TCP | Port is open | Basic connectivity |
| HTTP | Endpoint returns 200 | Web servers |
| Custom script | Application-specific logic | Complex health criteria |
Configuration Parameters
| Parameter | Typical Value | Description |
|---|---|---|
| Interval | 5-30 seconds | Check frequency |
| Timeout | 2-5 seconds | Max wait for response |
| Unhealthy threshold | 2-3 failures | Failures before removal |
| Healthy threshold | 2-3 successes | Successes before restoration |
High Availability for Load Balancers
Load balancers require redundancy to avoid being a single point of failure.
Active-Passive (Failover)
Operation:
- Primary handles all traffic
- Secondary monitors via heartbeat
- On failure, secondary takes over VIP
Active-Active
Operation:
- Both load balancers handle traffic
- DNS distributes across LBs
- More capacity, but complex state sync
Load Balancer Technologies
| Technology | Type | Use Case |
|---|---|---|
| Nginx | Software L7 | Web traffic, reverse proxy |
| HAProxy | Software L4/L7 | High performance, TCP/HTTP |
| AWS ALB | Managed L7 | AWS applications |
| AWS NLB | Managed L4 | Ultra-low latency |
| Cloudflare | CDN + LB | Global distribution |
| F5 BIG-IP | Hardware | Enterprise, high throughput |
Common Patterns
SSL Termination
Benefits:
- Backend servers freed from SSL overhead
- Centralized certificate management
- Enables HTTP inspection for L7 routing
Global Load Balancing (GSLB)
Use case: Route users to nearest data center for lower latency.