Lori's recent Dear Slashdot blog post was not only interesting and entertaining, but it was also very popular. So much so that it got picked up by, and posted on Slashdot. When you're one of the people responsible for the site hosting such a suddenly famous post, this is both fantastic, and a little bit scary.

I'm always happy to see people writing compelling things that will draw attention to the other awesome stuff going on on DevCentral. Lori is fantastic at that, and this recent post is a prime example. By getting pushed out to a major geek news site like /. she's bringing in a huge influx of visitors that would likely have never seen DevCentral otherwise. Some will leave after reading her article, sure, but others will stay. They'll be interested by what she wrote and go digging to see what else is out there. They'll find the killer tech goodness that is DC and maybe they'll stick around.  This is the awesome side of Slashdot fame.  There is, however, a darker side.

The term "slashdotted" has been simultaneously feared and strived for by many in the tech industry for years.  Wikipedia explains what it means:

"The Slashdot effect, also known as slashdotting, is the phenomenon of a popular website linking to a smaller site, causing the smaller site to slow down or even temporarily close due to the increased traffic. The name stems from the huge influx of web traffic that results from the technology news site Slashdot linking to websites. The effect has been associated with other websites or metablogs such as Fark, Drudge Report and Digg, leading to terms such as being Farked or Drudged and the Digg effect. Typically, less robust sites are unable to cope with the huge increase in traffic and become unavailable – common causes are lack of sufficient data bandwidth, servers that fail to cope with the high number of requests, and traffic quotas. Sites that are maintained on shared hosting services often fail when confronted with the Slashdot effect."

I've seen many, many small-ish sites shut down completely for days on end thanks to a good slashdotting.  If someone had asked me if I would be concerned about DevCentral going down, or at least having major performance issues, thanks to be slashdotted, I'd be a liar if I said I no.  Sure, we have fantastic BIG-IPs and WAMs and other F5 gear in front of the site.  We have optimization iRules and all sorts of cool things to keep us up and running. I've seen what these things can do in test, and talked about theoretical limits, and I'm very confident in the infrastructure we're running. Putting all of that to test with a huge traffic spike against the production systems, though? That's always a little un-nerving.

Well, it was yesterday, anyway. Since then, Slashdot picked up Lori's post, and we have indeed seen a huge spike in site traffic. The number of hits has gone up dramatically, we've seen scores of new users coming in, and because of that the application servers trying to serve content for DevCentral have been ... bored. I know. It doesn't add up, right?  A huge spike in hits and overall site traffic, a direct Slashdot link and ... nothing.  I'm honestly not sure the servers even knew something had happened.  Why?  Web Accelerator, that's why.croppedGraph

We've semi-recently deployed Web Accelerator in front of DevCentral.  The system does all sorts of performance tuning for the site now. It makes things faster in general, as reports from all around the world have confirmed.  One very, very key feature though, at least in this case, is caching. Thanks to the powerful, advanced caching features of Web Accelerator (WA), roughly 85% of the traffic during the spiked period of being pummeled from Slashdot was served from the active cache.

That's ... amazing. Sure, I know caching isn't a new technology or anything. I know it's been around for a long time and many people are doing it very well, in many ways.  I've read about it, implemented it, coded for it, etc.  I'm just saying that to have it make such a hugely profound impact on the servers that I know and love, and to realize that without the WA boxes in front of us our app servers very well may have felt some severe pain, makes it hit a lot closer to home.

As you can see in the graph to the right (generated by WA, by the way), we were experiencing pretty normal site traffic until the Slashdot article hit.  Once it did, we saw a very substantial jump in site traffic.  To be clear, the green bars in the image are the requests being served out of the cache. The light blue bars above them are new requests to the application servers.  Notice that the blue bars didn't grow much at all, even with a gigantic increase in pages being served? Thank you, Web Accelerator, for filtering out the bad side of being slashdotted, and letting us enjoy only the good.

Also, a huge thanks to the IT guys here at F5 that helped us get this solution in place to make things better for the community. It was one of them that pointed out what a great job WA was doing through the traffic spike, and I really appreciate them being so on top of things.

#Colin