Blob storage and CDNs

Images, videos, audio and backups don't belong in a database. Blob storage, like Amazon S3 or Google Cloud Storage, holds files of any size cheaply and very reliably. A CDN keeps copies of popular files on servers around the world, so users download them from somewhere close.

To upload, the client asks your server for a signed URL that allows one upload for a few minutes. The client sends the file straight to storage, so your servers never handle gigabytes of video. Big files are uploaded in parts, so a dropped connection only means retrying one part.

Your database only stores the file's metadata: who owns it, its storage key, its size and its status. A background job can then make thumbnails or other versions once the upload finishes.

To serve it, you hand out a CDN URL. The first request in a region fetches from storage, and later ones are served from the CDN's copy. For private files, the URL is signed and expires.

• Keep big files out of your database and off your app servers.
• Say how uploads work, including big files and failed uploads.
• Mention a CDN for anything many users download.

Uploading and playing videos

Users upload videos up to 2 GB, and each video can be watched by millions of people around the world.

1. The app asks your server to start an upload. The server creates a video record with status 'uploading' and returns signed URLs for uploading the file in 10 MB parts.
2. The app uploads parts straight to blob storage, a few at a time. If one fails, only that part is retried. Your servers never see the video bytes.
3. When all parts are in, storage tells your system the upload is complete, and the video is queued for processing.
4. Workers convert it into several qualities, like 360p, 720p and 1080p, cut into short segments, so players can switch quality as the connection changes.
5. The finished files go into storage under names that include the video ID and version, and the record's status becomes 'ready'.
6. Players fetch segments through a CDN. The first viewer in a region pulls from storage, and the next million get the CDN's copy.
7. Storage costs are now about the files, not the views. The CDN bill grows with views, which is why popular files sitting at the edge matters so much.

Things interviewers tend to ask once you've brought this up.

How do you keep private files private?

Don't make the storage public. Hand out signed URLs that expire after a few minutes, and only to users who are allowed to see the file.

How do you avoid storing the same file twice?

Hash the file's contents and use the hash as its name. If that name already exists, point the new record at it instead of storing it again.

What if a user starts an upload and never finishes?

The record stays 'uploading'. A cleanup job deletes records and parts older than a day, and storage can be set to delete unfinished uploads automatically.

How do you reduce storage costs over time?

Move old, rarely-viewed files to cheaper, slower storage tiers automatically. Most files are hot for a short while and then barely touched.

• Storing files in the database.
• Sending uploads through your app servers.
• Not mentioning a CDN for content lots of people view.
• Overwriting files in place and wondering why users see old versions.
• Forgetting large uploads fail halfway and need to resume.

• Caching strategiesMake reads faster, and figure out what to do when data changes.
• Relational vs non-relational databasesPick a database based on how you'll actually use it.
• Load balancing and replicationSpread traffic across servers and keep copies so one failure isn't an outage.