System Design: Designing Instagram (Photo Sharing at Scale)

Instagram is a massive photo-sharing platform where users can follow others, post photos, and view a personalized feed. The core challenges are efficiently storing billions of images and generating a feed that aggregates content from thousands of followed accounts.

1. Core Requirements

• Post Photos: Users can upload images.
• Follow: Users can follow other users.
• News Feed: Users see a chronological or algorithmic feed of photos from people they follow.
• Search: Searching for photos/users by hashtags or usernames.

2. Image Storage Architecture

Storing billions of high-resolution images is expensive and requires a specialized approach.

• Object Store: Images are stored in an object store like Amazon S3 or Google Cloud Storage.
• CDN: Use a CDN (CloudFront/Akamai) to cache images at edge locations globally to ensure fast loading times.
• Metadata Store: Store image URLs, owner IDs, and upload timestamps in a distributed NoSQL database like Cassandra or DynamoDB.

3. Feed Generation (The Core Challenge)

Generating a feed for a user who follows 1,000 people, each posting frequently, is a compute-intensive task.

Option A: Pull Model (Fan-out on Load)

The system fetches the latest photos from all followed accounts at the time the user opens the app.

• Problem: As the number of followers grows, the query becomes extremely slow.

Option B: Push Model (Fan-out on Write)

When a user uploads a photo, it is immediately "pushed" into the feed cache of all their followers.

• Optimization: Pre-compute the feed and store it in Redis. When a user logs in, they simply read their pre-computed feed.

4. Handling Celebrity "Thundering Herds"

If a user with 100 million followers (like Cristiano Ronaldo) posts, pushing that photo to 100 million caches will crash the system.

• The Solution: Use a Hybrid Model. Push for regular users. For celebrities, don't push. Instead, when a follower of a celebrity pulls their feed, the system "merges" the celebrity's latest post into the feed on-the-fly.

5. Database Sharding

To scale the metadata store:

• Shard by UserID: All data for a user (profile, photo metadata) is stored on the same shard.
• Problem: Some users are much more active than others (Hot Shards).
• Refinement: Use Consistent Hashing to distribute users across shards evenly.

6. Real-time Interactions (Likes and Comments)

Likes and comments are high-frequency, low-latency updates.

• Counter Optimization: Use Redis to keep track of like/comment counts. Periodic background jobs flush these counts to the main database to ensure durability.

Summary

Designing Instagram is about balancing the extreme storage requirements of media with the complex query requirements of a social graph. By using a hybrid feed generation model and a globally distributed CDN, you can build a platform that serves billions of high-quality photos with sub-second latency.