System design: design a music streaming app

Design a music streaming service like Spotify. Users can search songs, artists and albums, make playlists and like songs. Playback should start almost immediately for millions of listeners at once, and the service should basically never be down.

Say up front what you're covering.

Then agree on what "good" means:

• Starts fast: a song starts playing within about 200 ms of pressing play.
• Always up: playback keeps working even if a database or a whole region is having a bad day.
• Search is quick: results show up as you type, in under about 100 ms.
• Your stuff is safe: a playlist edit or a like is never lost once you've seen it saved.

These are assumptions: 100 million listeners a day, each listening about an hour, songs about 3.5 minutes long, 100 million songs in the catalog, and each song stored at three quality levels, about 15 MB in all.

~1.5 PB

of audio files

~4M

people listening at once, on average

~650 Gbps

of audio going out, on average

~20K

songs starting per second

1. Every song is stored once in object storage, at a few quality levels, cut into small chunks of a few seconds each.
2. When you press play, the app asks the playback service for the song. It checks you're allowed to play it and returns short-lived links to the chunks on a CDN.
3. The app fetches chunks from the nearest CDN server. Popular songs are already sitting there, so most plays never reach our storage.
4. Search goes to a search index built from the catalog. Playlists and likes go to their own databases, split by user or playlist.
5. Every play is sent to an event stream, which feeds play counts, charts and anything else that can happen later.

What the apps need to do

What we store and where

Our servers should never send audio. The playback service only answers 'can this person play this song, and where are the chunks?', which is a small, fast lookup. The bytes come from a CDN, which is just a big cache spread around the world.

Songs are cut into chunks of a few seconds, at a few quality levels. To start fast, the app asks for the first chunk at a lower quality, starts playing as soon as it lands, then switches up once it knows how fast the connection is. You hear sound after one small download, not a whole file.

Popularity is very lopsided. A small set of songs gets most of the plays, so those are almost always already on the CDN server near you. Old or rare songs miss the CDN and come from storage, which is slower. A middle layer of cache between the CDN and storage stops those misses from all hitting storage at once.

The app can also hide the wait. While a song plays, it fetches the start of the next song in the playlist or album, so skipping feels instant. It keeps recently played songs on the phone too.

We don't search the catalog database. It's built for 'get song 123', not 'everything that starts with "tay"'. So we keep a separate search index built from the catalog, with entries for song, artist and album names.

The catalog only changes when new music is added, which is mostly a weekly batch from labels. Each change goes on a queue and indexers update the index from it. Being a few minutes behind is fine.

Search runs as you type, so matching has to work on the start of words and forgive small typos. Ranking matters more than matching: lots of things match 'love', so sort by how often each result gets played, which we already count from play events.

The index for 100 million songs is a few hundred GB, so it's split into shards, with copies in every region. Popular searches repeat a lot, so cache the results of the most common ones for a minute or so.

Likes are a long list per person, read all at once when you open your library. A key-value database split by user ID fits that well, and it's the usual relational vs non-relational call. Splitting by user is plain sharding, and everyone's library is roughly the same size.

Playlists are split by playlist ID, since a popular playlist is read by millions of people who aren't its owner. Those hot ones get cached. Every edit carries a request ID, so if the app retries after a timeout, the same song isn't added twice. That's idempotency.

Shared playlists can get two edits at once. Give each playlist a version number, and reject an edit made against an old version. The app reloads and tries again. Edits are rare, so that almost never happens.

• Caching strategiesMake reads faster, and figure out what to do when data changes.
• Message queues and event streamsKeep services from depending on each other, and handle traffic spikes.
• Relational vs non-relational databasesPick a database based on how you'll actually use it.
• Sharding and consistent hashingSplit data across machines and keep it balanced.
• Idempotency and retriesMake sure a retried request doesn't happen twice.
• Consistency and the CAP theoremWhat happens when copies of your data can't agree.
• Search and inverted indexesFind text fast, rank what comes back, and keep the index fresh.
• Blob storage and CDNsStore big files cheaply and serve them from close to your users.

We wrote this sample answer ourselves.

Practice Loop is independent and isn't affiliated with or endorsed by Amazon.