Digging His Way to Web Success

By Alex Handy

July 7, 2008 —  (Page 1 of 3)
When TV talk show host and eventual Digg founder Kevin Rose asked systems admin and TechTV intern Ron Gorodetzky if he'd help to build a news site, the pair weren't expecting to be more popular than baseball. But shortly after Digg.com first launched in late 2004, the site quickly became a bona fide hit. Today, Digg is the 33rd most popular site in the U.S., ahead of CNET.com, the Washington Post and even MLB.com.

With over 26 million unique visitors a month, what started as a lark for Gorodetzky has matured into a full-time, enterprise-style systems management challenge. Gorodetzky, who turns 27 in August, said that Digg has evolved organically over the past four years, something the Digg team has worked both for and against over the years as the hacks turned into solid solutions.

While the current architecture does scale, Gorodetzky said that the goal is to move the Digg systems into a much more architected design.

“Ultimately what you want to do is go to a sharded architecture,” said Gorodetzky. “The first 1,000 users go on this set of servers, the next thousand on this set. You'll find that's an appropriate way to go, but right now we haven't done that yet. The way we've achieved some of the benefits of that is by doing the normal master/slave replications, and sending some types of traffic to only certain slaves and others to other slaves. We'll have queries related to comments going to some specific slaves, even though any slave could answer the query.”

Wage Slave
That slavery is also a heavy focus of optimization at Digg. The site is based on a LAMP stack, with  MySQL and PHP doing most of the heavy lifting, said Gorodetzky. Since he's been there right from the start, Gorodetzky had to work through the typical growing pains, particularly those related to MySQL replication.

“The first pain point we hit was just database stuff. The first thing you'll notice is when you start to grow these queries, the database can't commit as much time to committing a certain query as it used to,” said Gorodetzky. “You'll find the normal things that work, suddenly don't. You'll find that, one day, you'll see a spike in your graphs telling you that something's going slower. Once you do that, you get to the point where the database part is as fast as it can be, you cache things. You scale out your Web server so you have more resources there, generally caching and doing less work per request.”

Related Search Term(s): Database administration, server management, Digg

Share this link: http://www.sysmannews.com/link/32490