How a search engine looks like

In the words of technophilosopher René Berger, "It's becoming impossible not to visit with Google daily". But when you do, what do you visit exactly? In other words: have you ever wondered how a search engine really looks like? It actually looks like this:

The picture is from today's New York Times, and it shows Google's new data center in The Dalles, Oregon, which is under construction. It is only one of several G data centers, but John Markoff and Saul Hansell in their article speculate that it may "soon be one of the world's most powerful supercomputers". Google is known as a search engine, they write, but it is actually "foremost an effort to build a network of supercomputers (...) that can process more data, faster and cheaper than its rivals". "Google has found that for search engines, every millisecond longer it takes to give users their results leads to lower satisfaction. So the speed of light ends up being a constraint".

The choice of The Dalles as the location of the data center has apparently to do with the availability of low-cost electricity (the grey structures protruding from the two football-field-sized buildings on the left are cooling towers) and easy access to data networks.

Markoff and Hansell have some interesting data: In March 2001, when Google was serving about 70 million pages a day, its computing system had about 8'000 servers; by 2003 that number had grown to 100'000. Today "the best guess is that Google has more than 450'000 servers spread over at least 25 locations around the world". For comparison, Microsoft's Internet activities currently use some 200'000 servers.

UPDATE 5 July 06 - In a recent Fortune article David Kirkpatrick puts the number of Google servers around the world at one million and confirms that Microsoft is also investing billions in infrastructure. Kirkpatrick quotes Microsoft's Ray Ozzie: "Just think about where there are windmills, dams, and other natural power sources around the world: that's where you're going to see server farms".

UPDATE 10 August 06 - Server farms, where the information age happens.

Comments

[cri]

So good fun to click here again. To get this interesting football fresco questions so close by the Google new datacenter field-sized-energy-cost-saving... when almost yesterday Italian team won 2-0 :-) Ok, I support Italian team but each time I get here I do not now which kind of new info I'll find... and that is the real fun...tks for these delicious non-football-but-a-little-linked-to discovering :-)

[Dave Bennet]

I have often wondered how on earth Youtube stores such a staggering daily amount of data! Now google owns them, I suppose world domination is next.