The system design question
Design a system that takes live prices from stock exchanges and streams them to thousands of users. Each user only wants the tickers on their watchlist, and updates should arrive within milliseconds.
Agree on the scope (5 minutes)
Say up front what you're covering.
| We'll design | We'll skip |
|---|---|
| Taking in exchange feeds | Placing trades |
| Putting prices in one format | Historical charts |
| Sending each user their tickers | Price alerts |
| Showing a price straight away on subscribe | News |
Then agree on what "good" means:
- Fast: a price change should reach screens within a few milliseconds inside our network.
- In order: a user should never see an older price after a newer one.
- Complete: no missed updates, or at least we notice and fix them.
- Fair: one slow user's connection can't slow down everyone else.
Rough numbers (5 minutes)
Made up, but reasonable: 100,000 tickers, about 1 million price updates a second at market open, and 300,000 users online with about 50 tickers each.
- 1M
- price updates per second at the open
- 300K
- users connected
- 15M
- user-to-ticker subscriptions
- 100K+
- watchers on the most popular tickers
| Question | Rough math | What it means |
|---|---|---|
| How many messages go out? | Each update × everyone watching that ticker | Far more than come in, so sending is the expensive part |
| How much is one popular update? | 1 update × 100,000 watchers | One price change can turn into 100,000 messages |
| How fast can a screen use updates? | People can't read 50 changes a second | We can send only the latest price a few times a second |
The overall design (10 minutes)
- Feed handlers connect to each exchange and convert prices into one format.
- They publish each price to a stream, split up by ticker so each ticker's prices stay in order.
- Users' screens keep an open connection to a fan-out server and tell it which tickers they want.
- Each fan-out server only listens for the tickers its own users want.
- When a price arrives, the server sends it to each of its users watching that ticker.
What the apps need to do
| Action | Who does it | What's sent |
|---|---|---|
| Subscribe | User's screen | A list of tickers to start watching |
| Unsubscribe | User's screen | Tickers to stop watching |
| Price update | Our servers | Ticker, price, time and a sequence number |
| Snapshot | User's screen | Tickers, and we return their latest prices |
What we store and where
| Data | Where it goes | Why |
|---|---|---|
| Latest price per ticker | In memory on every fan-out server | New subscribers get a price straight away |
| Who's watching what | In memory on each fan-out server | Only that server needs it |
| The price stream | A message log, split by ticker | Keeps order and lets servers catch up |
| Watchlists | Database | Saved so users get them back after logging in |
Step 1: Taking prices in, in order (10 minutes)
Every exchange formats its data differently, so feed handlers convert everything once, close to where it comes in. Exchanges number their messages, and if a number's missing, the handler asks the exchange to resend it.
Run two feed handlers per exchange. If one dies the other keeps going, and the sequence numbers let us drop the duplicates.
The handlers publish into a stream split up by ticker, like a message queue. Because a ticker's prices always land in the same partition, they stay in order.
Step 2: Sending prices out (15 minutes)
Each fan-out server keeps a simple map from ticker to the users watching it. It only listens to the partitions for tickers its users care about, so no server handles everything.
Every server also keeps the latest price for each ticker in memory. When someone subscribes, they get it immediately, without waiting for the next change. That's basically a cache.
Screens keep a connection open, so prices are pushed the moment they arrive. Polling would add delay and a lot of wasted requests.
Slow connections
Some users are on slow networks. If we queue every update for them, their queue grows forever. So for each slow user, keep only the latest price per ticker and send that when they're ready. They see fewer updates, but never old ones.
Step 3: Very popular tickers (5 minutes)
A big name might have 100,000 watchers. Spread those users across many fan-out servers, so each server sends to a few thousand of them instead of one server sending to everyone.
Assign tickers to stream partitions with sharding, and give the busiest tickers more room so they don't crowd out quieter ones.
Choices you should be able to defend
| We chose | Instead of | Because | We'd reconsider if |
|---|---|---|---|
| Pushing over an open connection | Screens polling for prices | It's much faster and wastes far less | Clients were on networks that block long connections |
| Sending only the latest price to slow users | Sending every single tick | Their queue can't grow forever, and they see current prices | Users needed every tick, like for trading tools |
| A shared message log | Feed handlers sending straight to servers | Servers can join, leave and catch up easily | We needed the lowest possible latency, and the extra hop cost too much |
| Splitting by ticker | Splitting by exchange | Keeps each ticker in order | One exchange's feed needed special handling |
What could go wrong (5 minutes)
| What happens | What we do about it |
|---|---|
| A feed handler crashes | Its twin keeps going, and sequence numbers drop duplicates |
| A message from the exchange is missing | Ask the exchange to resend, and hold that ticker until it arrives |
| A fan-out server dies | Screens reconnect to another server and get a snapshot of the latest prices |
| A user's connection is slow | Send only their latest prices so they don't fall further behind |
| An exchange feed stops | Mark its prices as stale on screen, so nobody trusts an old number |
Follow-up questions (5 minutes)
How would you add price alerts?
A separate job reads the price stream and checks each update against users' alert levels for that ticker. Store which alerts already fired so they don't fire twice.
Not everyone can see every exchange. How do you handle that?
Check what each user is allowed to see when they subscribe, and refuse tickers they're not paying for. That's access control.
How would you show historical charts?
Have another consumer write prices into storage built for time ranges, completely separate from the live path.
How the 60 minutes add up
| Minutes | What you're doing |
|---|---|
| 0–5 | Agree on what you're building |
| 5–10 | Rough numbers |
| 10–20 | Draw the overall design |
| 20–30 | Taking prices in, in order |
| 30–45 | Sending prices out |
| 45–50 | Very popular tickers |
| 50–55 | What could go wrong |
| 55–60 | Follow-up questions |
Key concepts used here
- Message queues and event streamsKeep services from depending on each other, and handle traffic spikes.
- Caching strategiesMake reads faster, and figure out what to do when data changes.
- Sharding and consistent hashingSplit data across machines and keep it balanced.
- Stream processing and windowingCount things in data that never stops coming in.
- Authentication and access controlWork out who someone is and what they're allowed to do.
- Real-time deliveryPush updates to users the moment they happen, with WebSockets and friends.
Further reading
We wrote this sample answer ourselves.
Practice Loop is independent and isn't affiliated with or endorsed by Bloomberg.