Internet giant Yahoo is adopting a new system for indexing web pages that will charge businesses to include material currently unlisted in its online search engine, marking the first volley in a duel with its former ally Google.
Sunnyvale-based Yahoo is touting the approach, scheduled to be announced on Tuesday, as a practical way to assure its search engine includes more of the so-called ”deep web” — the billions of pages that aren’t found during periodic crawls of the internet.
The method, often called ”paid inclusion,” will also help Yahoo’s search engine keep better tabs on the most current material on a web page, company officials said.
Search engine analysts generally applauded the move, saying it could open a rich new vein of content that’s lacking from all internet search engines.
The fees required to participate in the programme are likely to raise worries about Yahoo creating an online caste system dividing the haves and have nots of the internet.
To ease those concerns, Yahoo isn’t charging non-profit websites to add unlisted links to its search engine. The non-profit sites initially participating in the new indexing system include National Public Radio and the Library of Congress.
Yahoo’s index will continue to include websites that don’t pay the fees, but there’s no guarantee on how frequently those destinations will be visited or how extensively the content will be analysed, said Tim Cardogan, Yahoo’s vice-president of search.
The fees won’t buy websites a higher ranking in Yahoo’s non-commercial search results, Cardogan said.
The fees under Yahoo’s ”Content Acquisition Programme” will be based on the size of the participating websites, how many unlisted links are submitted and how frequently the links are clicked on by the users of Yahoo’s search engine.
Yahoo is counting on the programme to give it an advantage over Google as it vies to supplant its rival as the web’s most popular search engine. Yahoo licensed Google’s search engine for more than three-and-a-half years, but started to cut ties with its former partner two weeks ago, vowing to introduce better ways to explore the web.
Mountain View-based Google has built the web’s largest search engine index, spanning 4,28-billion pages without charging fees to be included. Yahoo says its index contains ”several billion” web pages, but won’t provide specifics.
In an interview on Monday, Google co-founder Larry Page called Yahoo’s new system ”a pretty bad thing to do. There are plenty of profits to go around in search engines to find ways to improve the user experience without charging fees to do it”.
Like Google and other major search engines, Yahoo has long been using its search results page to display text-based ads that are tied to search requests. But these advertising listings are labelled as ”sponsored results” and separated from the results generated through algorithmic formulas that are designed to provide an objective analysis.
Yahoo’s new system runs the risk of blurring the lines between the advertising and editorial sides of its search engine, said Chris Sherman, editor of Search Day, a newsletter published by SearchEngineWatch. ”There’s definitely going to be a gray area,” Sherman said.
Forrester Research analyst Charlene Li said the distinctions might not matter to many web surfers. ”Most people don’t care whether something (in a search engine) is paid for or not. They just care whether its relevant. This could be an instance where people go to Google when they are looking for general information and go to Yahoo when they are shopping for something.”
News of Yahoo’s system rankled Gary Ruskin, executive director of Commercial Alert, a consumer group that that has criticised search engines for co-mingling their advertising and editorial results.
”The bottom line is that this is just going to be another way for businesses and the wealthy to buy search engine results so they get the material they want in front of the eyes of search engine users,” Ruskin said. – Sapa-AP