For engineers working on social media, celebrities can be like natural disasters — swinging into town without warning and wrecking any vital infrastructure in sight. As Instagram revealed in a blog post yesterday, the picture sharing service was occasionally brought to its knees by posts from the likes of Justin Bieber and Kim Kardashian. Fans would mob new pictures with likes, slowing down Instagram's servers in the process. In an interesting article at Wired, Instagram's co-founder Mike Krieger explains that Bieber in particular was the cause of so many of these "early scaling issues," that he ended up memorizing the singer's database ID (6860189) so he could recognize these bugs on sight.
The problem for Instagram's engineers, was that the memory cache in the company's data centers — the speedy part of the servers tasked with delivering the most popular content — couldn't keep up with the number of likes on popular posts. Instagram's servers used to create an individual entry in their database for each person who liked a photo, adding up all these entries to give the total.
But with very popular photos, adding up all these numbers again and again took too much time. The solution was to "denormalize" the counters, which in this case meant creating a new database field for each photo to store its running total of likes. So, when Bieber drops his latest selfie and the likes go wild, instead of the servers having to add up all these hearts anytime someone loaded the picture, they instead made a single request for the running total.* No more overload.
Now, Instagram has more than 400 million users uploading 80 million photos and videos a day, and has recently moved into Facebook's servers to handle all the extra traffic. Perhaps one day there'll be someone bigger than Bieber whose popularity will crumble those systems, but really, there's only so much celebrity the world can handle — let alone those poor servers.
*This is a simplified explanation, but for more details check out Instagram's blog.
Verge Video: The ultimate selfie-stick