/ 2 August 2009

Rivals strive to topple Google in quest for the ‘perfect search’

Inside the Googleplex — the nickname for Google’s Californian headquarters — stands a flat screen on which rotates an image of the Earth.

As the globe slowly turns, bright lights shoot out like fireworks, illustrating where in the world Google searches are coming from. At the same time, a live feed cascades down one side of the screen with a random sample of search queries, everything from sports scores to Hollywood gossip.

It is a stark illustration of what lies behind one of the most powerful companies in the world and of a new industry, barely a decade old, that now lies at the heart of our internet-driven way of life: search.

Search measures what everyone in the world wants. For good or ill, all human desires — whether for a new car, a holiday, breaking news, fighting injustice or pornography — are expressed in search.

Every time your fingers flick across a keyboard and type words into a search engine, you are telling the internet what you want. Google has built a multibillion-dollar corporate empire on the back of that simple fact, as advertisers flock to the huge database of intentions to try to meet the needs expressed there.

It has given Google enormous power as it seeks to put all the world’s information at your disposal, making it easily accessible and better able to fulfil your wants. But Google is no longer alone. Last week’s announcement of a tie-up between Yahoo! and Microsoft will perhaps see the first real challenge to the giant of our unfolding information age. For the first time, meaningful competition could enter the world of search engines and, like the battling railway barons of a previous era, the biggest corporate battle of our time will be under way. Microsoft’s Bing search engine will take on Google head to head.

It is not hard to see why. This year it is expected that more than one trillion searches will be made online. In five years, that could be three trillion or more. That store of data is of astonishing value to corporations and governments. “It is scary. One does not want to think about it too closely,” said Professor David Yoffie of the Harvard Business School. It is often said that information is power. If that is true, the information now collected on the internet — and accessed by our searches — is the most powerful thing on Earth. No wonder the struggle over it is about to be waged so fiercely.

In just 10 years, search engines, especially Google, have changed our way of life. Back in 1999 Google was barely a year old as a company. The internet was a place of mostly dial-up accounts and clunky speeds. Now it is a world of fast-speed searches, connected all over the world, often free of charge, and Google has embarked on a mission to put as much of the world’s information as possible online and make it searchable. Google is scanning the world’s books and photographing the planet, sometimes down to your very own street corner. You can now search videos, maps and images, not only words. And the latest phones allow you access to all that information from almost anywhere in the world at high speed. No wonder Google has become a verb.

In a recent post on the company blog, Marissa Mayer, one of Google’s top employees, mused openly about where search was heading. It was a revealing insight into what the near future may hold and just how far Google, and other search engines, may end up taking the quest for search.

Even now, Mayer thought, accessing all that information was not speedy or convenient enough. She said it was frustrating at times not to be able to search for things without heading for a computer or even typing things into a handheld device. “How about a wearable device that does searches in the background, based on the words it picks up from conversations and then flashes relevant facts?” she said. If that sounds far-fetched, be warned: the idle musings of Google executives have a habit of coming true.

The money being ploughed into improving and developing search is enormous. So is the reward. Google generates about $20-billion a year in search-related revenue. Aside from Microsoft-Yahoo!, a host of other smaller search engines are nipping at its heels or focusing on different areas of the web to get a slice of that pie. One is Wolfram Alpha, which employs a staff of 250 to cull data from government and other public databases. Others, such as Answers.com, Mahalo, and Aardvark, aim to provide specific answers to search queries, not just lists of websites. Artificial intelligence is increasingly a growing part of the search engine world, expertly examining your online patterns to predict your tastes, desires and future needs. Rich Kahn is working to fit AI into his firm’s new search engine, eZanga, which should be launched by the end of next year. “Will we be the next Google? Every entrepreneur is aiming for that,” Kahn said.

From Silicon Valley to Bangalore, computer science whizz kids and the money men who back them are engaged in this quest for “perfect search”. They are writing complex computer codes that can dig down into the so-called Deep Web and scour the most obscure corners of the internet in the blink of an electronic eye. They are called things like Kosmix and DeepPeep and are designed to search the parts of the web that an ordinary Google search misses.

But it is also possible that the concept of search may be moving forward so fast that it will outstrip search engines themselves. The explosive growth of social networking services, such as Twitter and Facebook, is taking the concept of search into unknown areas. Both began as ways for friends and acquaintances to share information and news about themselves but then they developed a critical mass. Suddenly Facebook and Twitter became tools that could be searched by their millions of users. Search on Google for a recipe and you will get a relatively random selection. Ask Facebook or Twitter for a recipe and you will get a choice often aimed specifically at you.

Nor is it just for trivia. In the recent political turmoil in Iran, Twitter became a vital tool for organising and releasing information about the violent post-election crackdown. If you wanted the latest news from Iran, it was Twitter that you turned to as well. Twitter had showed itself perfectly able to channel people’s most profound desires: in this case, political freedom of expression. You can plan a revolution with Twitter.

No one saw that coming. Just as no one saw Google coming. It is impossible to predict a future even 10 years ahead. “In the field of search, there is not a 10-year view. It would require a crystal ball,” said Mike McVeigh, head of the search division at digital marketing agency Zeta Interactive.

What is going on is revolutionary in the purest sense of the word: it upends all established orders in ways no one could foresee. For example, search is a crime-solver. Several murderers have been caught by their Google trace after police looked at their search histories and saw patterns hinting at the crime to come, such as searching for weapons or tips on disposing of bodies. Only two entities knew that the killer wanted to murder someone: the killer and Google.

The power of search goes far beyond such cases. By using Google Trends — which analyses Google searches — you can see what is on the minds of people all over the world. It has become a vital tool for seeing how popular things are and spotting needs and desires as they come into being. For corporations, it is a tool of great power, offering a way of tapping into every single customer in the world with access to a computer.

But the real power of the internet and search has been to break down gate-keepers. Putting all the information in the world into a form where it can be accessed and searched by anyone has destroyed industries. Just ask journalists, who suddenly find that anyone can listen to a press conference, scan a court case or email a president. Or the music industry, which found the internet had allowed bands to get their music out to the world virtually free. Or publishing. The growth of Amazon and the Kindle electronic book reader is poised to allow anyone to write and sell a book online. It is a democratising of power to the individual, fuelled by the simple fact of allowing people to search for whatever they want without a middle man.

There is a potentially dark side to all of this too. All those searches — effectively revealing knowledge about yourself — are logged and of interest to others. We are heading into a world that was previously the remit of science fiction dystopias: of massive computer networks parsing our every moment. Just take the murder example cited above. It is no longer too far-fetched to imagine a world where a husband with a violent history that has been caught in an online record, who has recently searched for a divorce lawyer and then starts to search for poisoning tips, may be brought to the attention of the police by a computer program monitoring his online activities. It could just save his wife’s life.

But is it really that empowering to have all that information so readily available? Perhaps information does not always equal knowledge. Literacy could collapse in the face of keyboards and searching Wikipedia instead of writing things down from books. Many parents despair at the language and grammar used for SMSing on mobile phones.

In a world where all information is available at the click of a mouse, how much will we bother to store in our minds? “We are heading into a different world,” said Jorge Pinto, a professor at New York’s Pace University. Perhaps not even Google knows exactly what the world will look like. Yet. – guardian.co.uk