I just participated in a great webminar titled “Avoiding the Fail Whale“, where Robert Scoble moderated a group of guests to talk about scalability issues and planning.
The guests, which included Paul Bucheit, Founder of Friendfeed (and who also did a lot of the development work for Gmail), Dorion Carroll, VP of engineering at Technorati and Nat Brown, CTO of iLike, touched on many aspects of how they were able to scale their applications across hundreds of servers.
Some of the things that I found really interesting:
- Application design before actual implementation is critical to scale
- They are not fans of queuing, but rather prefer to use MemcacheDB, which to be honest, I had never heard of;
- Google has always preferred doing things custom and internally;
- Code maintenance is done using Subversion and tags;
- Monitoring is done using Nagios, and –
- When asked which database engine they prefer using, they unanimously said “MySql”.
A couple of additional tips mentioned by Paul were to “always keep thing as simple as possible” and to “partition the data in the smalles chunks possible”.
Here’s the chat log: and the video is coming soon.
It’s a very interesting space that Scoble and company are exploring; I do think we need more of these, as the economy and air fare becomes more and more out of reach for entrepreneurs.
One thing I found personally encouraging is that I use all of the technologies mentioned above, but learned about MemcacheDB for the first time, and this is what makes attending these webinars so valuable. I could very well be unto the next big solution for my scalability issues.
But most important of all, I connected with a dozen or so peers with whom I lookforward to continue the conversation offline.
Update: There’s a great thread still going on about scalability in Robert’s blog
{ 3 comments }
