Stock price alerts: A Sample System Design Solution

The system design question

We get a stream of stock prices with the symbol, current price and previous close. Users subscribe to stocks. When a price goes up or down more than 5% from the close, send them a push notification. Design it.

Agree on the scope (5 minutes)

Say up front what you're covering.

We'll design	We'll skip
Reading the price stream and spotting a 5% move	Where the price feed comes from
Storing who follows which stock	Charts and the rest of the app
Fanning one move out to every follower	Custom thresholds per user, until the follow-ups
Sending pushes without duplicates, and retrying failures	Email and SMS

Then agree on what "good" means:

Quick: a push should land within a few seconds of the move. A stale alert is close to useless.
Once per move: a user gets one alert when a stock crosses 5%, not one for every price tick after that.
Survives bursts: a market-wide drop can trigger thousands of stocks in the same second.
Doesn't lose alerts: if a send fails, we retry it. If a server dies, the alert still goes out.

Rough numbers (5 minutes)

These are assumptions to size things: 10,000 symbols, each ticking about once a second while the market's open, 50 million users, and each user following about 20 stocks.

~10K
price updates per second: ~1B
subscriptions in total: ~5M
followers on a really popular stock: ~50M
pushes in a bad market minute

Question	Rough math	What it means
How many price updates?	10,000 symbols × 1 a second	About 10K a second. One machine could check those, but we'll split it anyway
How many subscriptions?	50 million users × 20 stocks	About a billion rows. Small rows, but too many for one machine
How many followers on a big stock?	Say 10% of users follow the top few	About 5 million people to notify for one stock crossing
What about a crash day?	Most of 50 million users have a stock that crosses	Tens of millions of pushes in a few minutes. That's the real load

The overall design (10 minutes)

A detector turns ticks into a few crossing events, and workers turn each event into pushes.

Prices arrive on a stream, split by symbol, so every tick for one stock lands on the same detector.
The detector keeps a little state per symbol: the previous close and whether it's already alerted today. When a price first crosses 5% up or down, it writes one event to an alert stream.
Fan-out workers read each event, page through that stock's followers in chunks, and drop one send job per chunk onto a push queue.
Push senders take jobs off the queue, check they haven't already sent that alert to that user, and call the phone push services.
Failed sends go back on the queue with a delay. After a few tries, they go to a dead-letter queue so we can look at them.

What the apps need to do

Action	Who does it	What's sent
Follow a stock	Mobile app	User ID and symbol
Unfollow a stock	Mobile app	User ID and symbol
List what I follow	Mobile app	User ID, and we return their symbols
Register a device	Mobile app	User ID and the phone's push token, so we know where to send

What we store and where

Data	Where it goes	Why
Subscriptions, by symbol	A key-value or wide-column store, keyed by symbol then user	Fan-out reads every follower of one symbol, in pages
Subscriptions, by user	Same store, a second table keyed by user	The app's 'what I follow' screen
Per-symbol alert state	In the detector's memory, saved every few seconds	Close price and whether we've alerted, read on every tick
Sent alerts	A fast key-value store with a one-day expiry	Lets senders skip an alert they've already sent
Device push tokens	A normal database, keyed by user	Where to send each user's pushes

Step 1: Spotting a move without spamming (8 minutes)

The check itself is simple: compare the price to the previous close and see if it's more than 5% away. The catch is that a stock sitting at 5.1% will tick hundreds of times. If we alert on every tick, people turn notifications off forever.

So the detector alerts on the crossing, not on being past the line. It keeps a small record per symbol: the close and which way, if any, it's already alerted today. A tick past 5% up only makes an event if we haven't sent an up alert yet. That's stream processing with a bit of state per key.

Splitting the stream by symbol means one detector owns each stock, so two machines can't both decide it crossed. The state is tiny, so we save it every few seconds. If a detector restarts, it reloads that and picks up where it left off.

Bouncing around the line: once we've alerted up, wait until it falls back under, say, 4% before we'd alert up again. Or just alert once per direction per day. Ask which they want.
A new day: reset the state when the close changes, which happens once a day per symbol.

Step 2: Fanning out to millions of followers (12 minutes)

One event for a popular stock means 5 million pushes. Doing that in one loop on one machine would take minutes, and if it crashed halfway we'd have no idea where it stopped.

Instead, break it up. A fan-out worker reads followers from the subscriptions table in pages of, say, 1,000, and drops one job per page on a queue. Each job says: this alert, these 1,000 users. Lots of push senders pull jobs in parallel. A big stock becomes 5,000 small jobs, and we add senders to go faster.

Storing subscriptions keyed by symbol makes those pages cheap: it's one range read in the order they're stored. Split that table by symbol with sharding, and keep a second copy keyed by user for the app's own screens.

A crash day

When the whole market drops, thousands of symbols cross in the same minute. The queue soaks up the spike, and senders work through it as fast as the push services let them. Let users get one combined push, like "5 of your stocks moved more than 5%", if they'd otherwise get a pile of them. That cuts the load and it's nicer to get.

Step 3: No duplicates, and retrying failures (10 minutes)

Queues usually deliver at least once. A sender can crash after sending but before marking the job done, and the job comes back. So assume every job can run twice, and make sending idempotent.

Give each alert an ID made from the symbol, the day and the direction. Before sending to a user, the sender sets a key for alert ID plus user ID, only if it doesn't exist yet. If it was already there, skip. The keys expire after a day, so the store stays small.

Push services fail for normal reasons. A timeout or a busy error goes back on the queue with a growing delay. An invalid token means the user deleted the app, so drop it and remove the token. After a few failed tries, park it on a dead-letter queue and alert someone if that queue starts growing.

Old alerts: if a retry's more than a few minutes late, drop it. A 5% alert from an hour ago isn't worth sending.

Choices you should be able to defend

We chose	Instead of	Because	We'd reconsider if
Alert on the crossing	Alert whenever the price is past 5%	One push per move instead of hundreds	Users wanted updates as it kept moving, like every extra 5%
Split the stream by symbol	Any detector checks any tick	Only one place decides a stock crossed, so no double events	One symbol got way more ticks than one machine can handle, which is unlikely
Fan out through a queue in pages	Send to every follower in one loop	Parallel, restartable, and spikes just wait in line	Everyone followed a handful of stocks with few followers each
A sent-alerts check before each push	Trust the queue to deliver once	Duplicates are what users notice and hate	The extra lookup became the bottleneck, then batch it per page

What could go wrong (5 minutes)

What happens	What we do about it
A detector crashes	Another one takes its symbols, reloads the saved state and replays a few seconds of ticks
A sender dies mid-job	The job comes back from the queue, and the sent-alerts check skips anyone already done
The push service is slow or down	Jobs retry with growing delays, and anything too old gets dropped
The price feed sends a bad price	Ignore a tick that jumps far from the last one until a second tick agrees with it
Everything crosses at once	The queue fills, senders go flat out, and we combine alerts per user
The subscriptions store is slow	Fan-out slows down but nothing's lost, since the event sits on the alert stream until it's done

Follow-up questions (5 minutes)

What if users can pick their own threshold?

Store the threshold on each subscription and group them into a few buckets, like 3%, 5% and 10%. The detector alerts per bucket, and fan-out only reads followers in that bucket.

How would you add email too?

Fan-out drops jobs onto a separate email queue with its own senders. The same alert ID works for the duplicate check, with the channel added to the key.

What about quiet hours?

Senders check the user's settings, and if it's quiet hours, they either drop the alert or save it for a morning summary.

How the 60 minutes add up

Minutes	What you're doing
0–5	Agree on what you're building
5–10	Rough numbers
10–20	Draw the overall design
20–28	Spotting a move without spamming
28–40	Fanning out to millions of followers
40–50	No duplicates, and retrying failures
50–55	What could go wrong
55–60	Follow-up questions

System design: stock price alerts