Mint Digital

Mint Digital

Sex Sells. Mint Delivers.

Posted in by Christopher Wilson

07 April, 2009

Please download the latest Flash player

"How much success?"

Throughout the history of our client engagements, there is always a slightly uncomfortable moment when this question rears its ugly head.

"How much traffic do you think the site will do?"

Responses range from the modest to the outright delusional with looks of confusion and disbelief skirting the perimeter. Fortunately, we have had more than a bit of experience in this area, particularly when the site in question is linked to a television property. In the most recent case, Sexperience, the website supporting Channel 4's popular The Sex Education Show Vs Pornography, we had numbers from the first season as guidelines.

However the subject matter is more than just a catalyst for traffic, it's more like a match to a can of gasoline. We had expected high numbers, but 750–900K+ page views per day was double what we served last season. Amazingly, the 3+ terabytes of content (primarily video) served daily was TEN TIMES previous levels.

On top of these totals, the traffic was more spikey than we expected. Compared to last season, viewers were substantially more prone to visit the site immediately after a promotional shout-out.

At the end of the day, Channel 4 and industry magazine Broadcast deemed Sexperience a resounding success.

How did Mint deal with these traffic spikes?

Well, it was not a cakewalk. Our original configuration did need some tweaks. But our experience allowed us to use the proper tools to diagnose the problems, identify the pain points and get ready for even more traffic during the next show.

In a nutshell, we relied on the following:

  1. Server Virtualization — We have to scale up rapidly and separate concerns. Ordering hardware and building out physical servers like the old days (one year ago) takes too much time.
  2. Analytics — New Relic, Five Runs, Munin provide the data we need to know where to add capacity or focus on optimizing performance. Without it, you are shooting in the dark.
  3. Monitoring — Monit has long been part of the stack and it is indispensable in these high load situations.
  4. Page caching and nginx — Extensive page caching with the remarkable performance of the nginx web server is key.
  5. Load testing — httperf or ab should be standard so when you see problems, you can replicate and solve instead of heading back into the next show not knowing whether the servers will melt down.
  6. Care with Sweepers — It is important to assess how dynamic a site really needs to be. Do you need it to change for every action? If you do, and you get a traffic spike like Mount Everest, you might not get much benefit from your caching. Perhaps an update once a minute is enough.
  7. Clear communication about traffic with the client — There was a time when we talked hardware with clients when starting a project... how many web servers, how many CPUs, how much memory, etc. No more. A client cares how many customers/viewers they can handle and that is the language we speak now (a dialect called Briarpatch).

Comments

  1. LiaitrarkNepe

    09 March, 2010

    Hello,

    We want to inform you about the illegal german carding forum "http://carders.cc". You can buy there stolen creditcards, drugs, weapons, child porn and all other things what you want.

    Our local police stations seems to be too stupid to stop them. Please write abuses to the Domainregistrar, to the hoster and so an, please help us!

    Here are some screenshots which proofs my words! http://yfrog.com/5bdirtymoneyp http://yfrog.com/1fdrugs2p http://yfrog.com/1ndrugs1p http://yfrog.com/2odrugs3p http://yfrog.com/jbfraudp http://yfrog.com/esfraud2p

    Here is again the link to this crimeforum: http://www.carders.cc Please help us to stop them, spread this news everywhere!

    We need to stop the crime in this world, fight with us for a better world without internet crime!

    Thank you!

Add your comment

  1. Or sign in using these: