Build a multi-threaded log writer: A Sample System Design Solution

The system design question

Build a logging system that lots of threads in one program can write to. Every entry has to end up in the file complete and in the order it was written, and it needs to keep up as you add threads. This one's a single-machine design.

Agree on the scope (5 minutes)

Say up front what you're covering.

We'll design	We'll skip
Many threads writing at once	Sending logs to other machines
Keeping entries whole and in order	Searching logs
Getting entries onto disk	Log formats
Starting a new file when one gets big	Deleting old logs

Then agree on what "good" means:

Correct: entries never get mixed together, and they stay in the order they were written.
Fast for writers: logging shouldn't slow down the threads doing real work.
Safe: agree with your interviewer how much can be lost if the program crashes.
Scales: adding threads shouldn't make things fall apart.

Rough numbers (5 minutes)

Made up, but reasonable: 32 threads writing 100,000 entries a second in total, about 200 bytes each.

100K
entries per second: ~20 MB
written per second: ~200 MB/s
a disk can write in order: ~1K
times a second you can force data onto disk

Question	Rough math	What it means
Can the disk keep up?	20 MB/s against about 200 MB/s	Yes, if we write in big, ordered chunks
Can we force every entry to disk?	100K entries vs about 1K forced writes a second	No, we have to group entries together
How much is in memory at once?	Say 50 ms of entries	About 1 MB, which is nothing

The overall design (10 minutes)

Threads just drop entries on a queue. One writer thread owns the file.

A thread calls log() with its entry.
The entry goes on an in-memory queue. Its place in the queue is its place in the file.
One background writer thread takes entries off the queue in batches.
It adds each batch to a buffer and writes the buffer to the file in one go.
Every few milliseconds, it forces what's written onto the disk.

What the apps need to do

Action	Who does it	What's sent
Log an entry	Any thread	The message, which gets a timestamp and sequence number
Flush	The program	Waits until everything logged so far is on disk
Close	The program	Writes everything left, then stops the writer

What we store and where

Data	Where it goes	Why
Waiting entries	An in-memory queue with a size limit	Hands work from many threads to one
The current chunk	A buffer in memory	One big write is much cheaper than lots of small ones
Log files	Disk	Start a new file when the current one gets big

Step 1: Start simple: one lock (5 minutes)

The simplest correct version wraps every write in a lock: only one thread writes at a time, so entries never get mixed. Say this first. It's correct, and it gives you something to improve.

The problem is that every thread waits on the lock and on the disk. As you add threads, they spend more and more time just waiting.

Step 2: One writer and a queue (15 minutes)

Instead of every thread writing to the file, threads put their entry on a queue and go back to work. One writer thread is the only thing that ever touches the file. That's the same idea as a message queue, just inside one program.

What does 'in order' mean with lots of threads? The cleanest answer is the order entries were added to the queue. Give each entry a sequence number as it goes in, and order in the file always matches.

Adding to the queue still needs a little locking, but it only lasts a moment, so threads barely wait.

When the queue fills up

If threads log faster than the disk can write, the queue grows. Give it a size limit and pick a rule: make threads wait (nothing lost, but slower), or drop low-priority entries (fast, but you lose some). Usually, wait for errors and drop debug logs.

Step 3: Batching and getting onto disk (10 minutes)

The writer takes everything waiting on the queue at once, writes it as one chunk, and repeats. Under heavy load, batches get bigger and it gets more efficient.

Writing to a file doesn't mean it's on disk yet. The operating system holds it in memory for a while. Forcing it to disk is slow, so do it every 10 milliseconds or so, not for every entry. That's a trade-off: if the program crashes, you lose up to the last 10 milliseconds. Check with your interviewer what's acceptable, and offer a 'wait until this is safe' option for important entries.

When a file reaches a size limit, the writer closes it and starts a new one. Only the writer touches files, so no other thread has to know.

Choices you should be able to defend

We chose	Instead of	Because	We'd reconsider if
One writer thread and a queue	A lock around the file	Threads barely wait, and order is simple	There were only ever a couple of threads
Forcing to disk every few milliseconds	Forcing every entry to disk	It's about 100 times faster	Losing any entry in a crash was unacceptable
Making threads wait when the queue's full	Dropping entries	Nothing important gets lost	Logging must never slow the program down
One shared file	A file per thread, merged later	Order is already correct	Even the queue became a bottleneck

What could go wrong (5 minutes)

What happens	What we do about it
The program crashes	Up to the last few milliseconds of entries can be lost, and important ones can wait to be safe
The disk fills up	Stop writing debug logs, raise an alert, and keep the most important entries
The disk is slow for a moment	The queue absorbs it. If it fills, the full-queue rule kicks in
The writer thread dies	Restart it, and have log() report an error instead of hanging forever
Someone logs a giant entry	Cap the size of one entry so it can't block everyone else

Follow-up questions (5 minutes)

How would you send logs to a central place?

Have the writer also hand batches to a shipper that sends them over the network, with its own queue so a slow network doesn't slow the file writer.

Could you avoid the queue lock entirely?

Give each thread its own small buffer and number entries with one shared counter, then have the writer merge buffers in number order.

What about multiple processes?

Each process writes its own file, and a separate tool merges them by timestamp when needed.

How the 60 minutes add up

Minutes	What you're doing
0–5	Agree on what you're building
5–10	Rough numbers
10–20	Draw the overall design
20–25	One lock
25–40	One writer and a queue
40–50	Batching and getting onto disk
50–55	What could go wrong
55–60	Follow-up questions

System design: build a multi-threaded log writer