Mint Digital

Responses range from the modest to the outright delusional with looks of confusion and disbelief skirting the perimeter. Fortunately, we have had more than a bit of experience in this area, particularly when the site in question is linked to a television property. In the most recent case, Sexperience, the website supporting Channel 4's popular The Sex Education Show Vs Pornography, we had numbers from the first season as guidelines.

However the subject matter is more than just a catalyst for traffic, it's more like a match to a can of gasoline. We had expected high numbers, but 750–900K+ page views per day was double what we served last season. Amazingly, the 3+ terabytes of content (primarily video) served daily was TEN TIMES previous levels.

On top of these totals, the traffic was more spikey than we expected. Compared to last season, viewers were substantially more prone to visit the site immediately after a promotional shout-out.

At the end of the day, Channel 4 and industry magazine Broadcast deemed Sexperience a resounding success.

How did Mint deal with these traffic spikes?

Well, it was not a cakewalk. Our original configuration did need some tweaks. But our experience allowed us to use the proper tools to diagnose the problems, identify the pain points and get ready for even more traffic during the next show.

In a nutshell, we relied on the following:

1. Server Virtualization — We have to scale up rapidly and separate concerns. Ordering hardware and building out physical servers like the old days (one year ago) takes too much time.
2. Analytics — New Relic, Five Runs, Munin provide the data we need to know where to add capacity or focus on optimizing performance. Without it, you are shooting in the dark.
3. Monitoring — Monit has long been part of the stack and it is indispensable in these high load situations.
4. Page caching and nginx — Extensive page caching with the remarkable performance of the nginx web server is key.
5. Load testing — httperf or ab should be standard so when you see problems, you can replicate and solve instead of heading back into the next show not knowing whether the servers will melt down.
6. Care with Sweepers — It is important to assess how dynamic a site really needs to be. Do you need it to change for every action? If you do, and you get a traffic spike like Mount Everest, you might not get much benefit from your caching. Perhaps an update once a minute is enough.
7. Clear communication about traffic with the client — There was a time when we talked hardware with clients when starting a project... how many web servers, how many CPUs, how much memory, etc. No more. A client cares how many customers/viewers they can handle and that is the language we speak now (a dialect called Briarpatch).