Maybe you didn't notice, but Gmail went down for about 100 minutes Tuesday,
depriving millions of the Gfaithful of their beloved e-mail service.
[ Google has its hands full with a variety of issues these days, including
Internet anonymity: "Skanks for nothing: Google must identify 'anonymous'
blogger" | Stay up to date on Robert X. Cringely's musings and observations with
InfoWorld's Notes from the Underground newsletter. ]
The gnashing of teeth and the renting of garments could be heard clear across
Twitterville. Like these representative twits -- err, tweets:
Words and phrases like “apocalypse” and “digital terrorism” really did come to
mind when I realized the #gfail.
gmail is down? im having an anxiety attack. i cant function without it.
No Gmail, Facebook is twitchy and Twitter is slow. My life may end just now…
Actually, as several Tweeters pointed out, Google's POP and IMAP servers were
working just fine. It was the Web interface that did a face plant. But to the
Twitterati who couldn't quite grasp that concept, it seemed the world had ended,
at least temporarily.
Turns out the problem was fairly prosaic, though at the scale Google operates,
even a hangnail can look like a fatal condition. The Google team took a few
Gmail Web servers offline for routine maintenance, the servers that remained
online got too slow and shut themselves down, and the resulting traffic
overwhelmed the machines still left standing.
In a post to The Official Gmail Blog, "Site Reliability Czar" Ben Treynor
writes:
The Gmail engineering team was alerted to the failures within seconds (we take
monitoring very seriously). After establishing that the core problem was
insufficient available capacity, the team brought a LOT of additional request
routers online (flexible capacity is one of the advantages of Google's
architecture), distributed the traffic across the request routers, and the Gmail
web interface came back online.