C10K Problem

How do you handle 10,000 concurrent connections on a single server?

Coined by Dan Kegel in 1999 when servers struggled to go beyond ~10k clients.

Why Was It a Problem?

Old approach — one thread per connection:

Client 1     -- Thread 1     (2MB stack)
Client 2     -- Thread 2     (2MB stack)
...
Client 10000 -- Thread 10000 (2MB stack)

10,000 × 2MB = 20GB RAM just for threads
+ massive context switching overhead

OS spends more time switching between threads than actually doing work.

Solutions

1. Multi-threading with Thread Pool

• Fixed pool of threads (e.g. 100 threads).
• Threads pick up connections from queue.
• Better but still blocking I/O per thread.

2. Non-blocking I/O + Event Loop

• Single thread, never blocks.
• Uses epoll to watch all sockets.
• Handles whoever is ready.
• Node.js, Nginx use this.

3. Async I/O

• Kernel handles I/O completely.
• Notifies via callback when done.
• Thread free the entire time.

How Nginx Solved It vs Apache

Apache (old):

One process/thread per connection
1000 connections = 1000 threads = slow, high memory

Nginx (new):

Event-driven, single/few threads
1000 connections = handled by epoll in one thread = fast, low memory

Nginx was specifically built to solve C10K.

C10K → C10M

Once C10K was solved, the new challenge became C10M — 10 million concurrent connections.

• Kernel bypass (DPDK) — skip OS kernel entirely, handle packets in userspace.
• CPU pinning — dedicate cores to network processing.
• Used in high frequency trading, CDNs.

Summary

Approach	Concurrency	Memory	Complexity
Thread per connection	Low	Very High	Low
Thread pool	Medium	High	Medium
Event loop (epoll)	Very High	Low	High
Async I/O	Very High	Very Low	Very High

The C10K problem is essentially why Node.js and Nginx exist.