The system design question
Design a web application where users of a financial data company can sign up, reset their password, request information about companies, and raise IT help requests. Each feature should deploy independently, and the data is highly sensitive.
Agree on the scope (5 minutes)
Say up front what you're covering.
| We'll design | We'll skip |
|---|---|
| Signing up and logging in | Where the company data comes from |
| Resetting a password | Billing and contracts |
| Requesting company information | The UI design |
| Raising and tracking IT tickets | The IT team's internal tools |
| How each feature ships on its own |
Then agree on what "good" means:
- Secure: only the right person sees their own data, and a stolen database dump is useless to whoever took it.
- Independent: the ticket team can ship on a Tuesday without touching sign-up or waiting on anyone.
- Contained: if one feature breaks or gets attacked, the others keep working.
- Auditable: we can say who looked at what, and when, months later.
Rough numbers (5 minutes)
Made up, but reasonable: about 300,000 client users, 50,000 of them active on a given day, around 100,000 company info requests a day and a couple of thousand IT tickets. Audit logs kept for 7 years.
- ~300K
- client users
- ~50K
- active each day
- ~200
- requests per second at a peak
- 7 years
- of audit logs kept
| Question | Rough math | What it means |
|---|---|---|
| How busy is it? | 50K users × about 100 requests a day ÷ 86,400 seconds | About 60 a second on average, maybe 200 at a peak. A few small servers per feature |
| How big is the user table? | 300K users × about 2 KB | Under 1 GB. It fits in one database easily |
| How big are audit logs? | 50K users × 100 events × 500 bytes × 365 days × 7 years | About 6 TB, so store it somewhere cheap and append-only |
The overall design (10 minutes)
- The browser talks only to an API gateway over HTTPS. The gateway checks the login token, applies rate limits, and routes the request.
- Behind it are separate services: an auth service for sign-up, login and password reset, a company info service, and a ticket service. Each has its own database and its own deploy pipeline.
- The auth service hands out short-lived tokens. The other services check the token themselves, so they don't call auth on every request.
- Slow work, like sending emails or pulling a big company report, goes onto a queue and gets picked up by workers.
- Every service writes who did what to a shared, append-only audit log.
What the apps need to do
| Action | Who does it | What's sent |
|---|---|---|
| Sign up | New user | Email, name, company, and an invite code from their account manager |
| Log in | User | Email, password, then a one-time code from their second factor |
| Request a password reset | User | Email. Always replies 'check your inbox', whether or not the account exists |
| Set a new password | User | The reset token from the email and the new password |
| Request company info | User | Company ID and what they want, like filings or financials |
| Open or update a ticket | User | Subject, description, attachments |
What we store and where
| Data | Where it goes | Why |
|---|---|---|
| Users and password hashes | Auth service's relational database | Small, needs strict uniqueness on email, and nobody else can read it |
| Reset tokens | Auth database, stored hashed with a 15 minute expiry | So a leaked table can't be used to reset anyone's password |
| Company info requests | Info service's relational database | Simple rows with a status, tied to a user ID |
| Popular company data | A cache in front of the info service | Lots of people ask about the same big companies |
| Tickets and comments | Ticket service's database, attachments in encrypted object storage | Files don't belong in the database |
| Audit events | Append-only log storage, kept 7 years | Nobody can edit or delete them, including us |
Step 1: Sign-up, login and password reset (10 minutes)
Since this is a paid product for businesses, don't let just anyone sign up. The account manager creates an invite, and the user signs up with that invite code. That cuts out most fake accounts before they start.
Store passwords with a slow hashing algorithm made for passwords, like bcrypt or Argon2. Slow on purpose means someone who steals the table can only guess a handful of passwords a second. Require a second factor, like an authenticator app or a hardware key, for everyone. This is standard authentication practice, and with financial data it's not optional.
Login hands back a short-lived access token, around 15 minutes, plus a longer refresh token stored in a secure cookie the page's JavaScript can't read. Short tokens mean a stolen one isn't worth much for long.
Password reset
The user enters their email. We always say 'if that account exists, we sent a link', so attackers can't use the form to find out who's a customer. Then we email a random token that works once and expires in 15 minutes. We store only a hash of it, the same way we treat passwords.
When they set the new password, kill all their existing sessions and email them that it changed. Rate limit the reset form per email and per IP so nobody can spam it.
Step 2: Letting each feature deploy on its own (10 minutes)
Split by feature: auth, company info, and tickets are separate services, each owned by one team, with its own code, its own database and its own pipeline. The rule is that no service reads another service's database. If tickets needs a user's name, it asks auth through an API or keeps its own copy.
The gateway is the only shared piece, and it just routes by path, like everything under /tickets goes to the ticket service. Keep it boring so changing it is rare.
To ship safely, each service versions its API and only adds fields, never removes or renames them in place. New versions roll out to a small slice of traffic first, and roll back automatically if errors jump.
The frontend can be split the same way: one shell page that loads each feature's screens separately, so the ticket team can ship UI changes too. In an interview, mention it, but it's fine to say one shared frontend with feature flags is simpler and probably enough to start.
Step 3: Keeping sensitive data locked down (10 minutes)
Encrypt everything in transit, including between our own services, and encrypt every database and file store at rest with keys held in a key management service, not in config files.
Each service gets its own database credentials, and only it can read its data. Inside each service, check that the user is allowed to see this specific thing, like 'this ticket belongs to your company'. Checking only that someone is logged in is the most common way these apps leak data.
Ticket attachments are the riskiest input, since anyone can upload anything. Scan them for malware, store them outside the web servers, and serve them through short-lived signed links.
Every read and write of sensitive data goes to the audit log: who, what, when, from where. Write it to storage that can't be changed after the fact, so even an admin can't cover their tracks.
- Don't log passwords, tokens or full documents in normal app logs.
- Admin access for engineers goes through a separate login, needs approval, and is time-limited.
- Cache only data that's the same for everyone, like public company facts, never one user's private data in a shared cache.
Choices you should be able to defend
| We chose | Instead of | Because | We'd reconsider if |
|---|---|---|---|
| A service per feature | One app with modules | Teams can deploy independently, which the question asks for | We had one small team, where a single app is easier to run |
| Invite-only sign-up | Open sign-up with email checks | Paying clients are known ahead of time, so fakes are mostly gone | We added a free tier for the public |
| Short-lived tokens checked by each service | Calling auth on every request | Auth isn't a bottleneck or a single point of failure | We needed to revoke access instantly, in which case keep a small revoked list |
| Relational databases | A document store | The data's small and structured, and we want strict rules like unique emails | Tickets got really free-form, which a document store handles fine too |
What could go wrong (5 minutes)
| What happens | What we do about it |
|---|---|
| The auth service goes down | People already logged in keep working until their token expires. New logins wait |
| The ticket service has a bad deploy | Only tickets break. The rollout catches the error spike and rolls back |
| Someone tries thousands of passwords | Rate limits per account and per IP, then lock the account and alert. The second factor stops them anyway |
| The email provider is down | Reset and verification emails wait on the queue and go out when it's back |
| A database backup leaks | Data is encrypted with keys that weren't in the backup, and passwords are slow hashes |
Follow-up questions (5 minutes)
How would you add single sign-on for big clients?
Let a client company connect its own login system, so their users log in through their employer. The auth service trusts that and still hands out our own tokens.
How do you remove someone who leaves a client company?
The client admin disables them, auth kills their refresh tokens, and access ends when the current short token expires a few minutes later.
How would you handle a big jump in company info requests?
Cache popular, non-private company data and put slow report generation on the queue, so the web servers stay quick.
How the 60 minutes add up
| Minutes | What you're doing |
|---|---|
| 0–5 | Agree on what you're building |
| 5–10 | Rough numbers |
| 10–20 | Draw the overall design |
| 20–30 | Sign-up, login and password reset |
| 30–40 | Letting each feature deploy on its own |
| 40–50 | Keeping sensitive data locked down |
| 50–55 | What could go wrong |
| 55–60 | Follow-up questions |
Key concepts used here
- Authentication and access controlWork out who someone is and what they're allowed to do.
- Rate limitingStop too much traffic with token buckets and sliding windows.
- Relational vs non-relational databasesPick a database based on how you'll actually use it.
- Caching strategiesMake reads faster, and figure out what to do when data changes.
- Message queues and event streamsKeep services from depending on each other, and handle traffic spikes.
- Load balancing and replicationSpread traffic across servers and keep copies so one failure isn't an outage.
Further reading
We wrote this sample answer ourselves.
Practice Loop is independent and isn't affiliated with or endorsed by Bloomberg.