March 28, 2024

Ping: An Engine’s Tall Order: Streamline the Search

In 0.18 seconds, Google led me to 1.9 million pieces of advice, both good and suspect. Drink plenty of water and try to tough it out, but go to the doctor if it doesn’t go away — that seemed to be the consensus. But also among the search results were how-to articles like one saying that if I sipped tea brewed from ground celery seeds or corn silk, the stone would pass within hours. My laughter didn’t reduce the pain.

Ultimately, with the help of several quarts of water, the stone passed. But I learned one thing from the Google search results. While you get them very rapidly, they may not be all that useful and dependable.

It isn’t hard to see why. A considerable amount of human effort is spent gaming Google results. Practitioners of the art call it search engine optimization, or S.E.O., and it is used to move a retailer’s Web pages or a news organization’s articles to the top of the search results page. Web pages are created specifically to fool Google’s search algorithm in order to get a higher ranking.

Over on Mechanical Turk, an online job clearinghouse run by Amazon for employers looking for people to do small tasks for small amounts of money, a significant number of the listings are for people to aid in either some form of S.E.O. or spam generation.

Google says it tweaks its page-ranking algorithm regularly to fight the S.E.O. experts, who frantically experiment to find a way to gain back the advantage, and the cat-and-mouse game continues.

The magnitude of this tit-for-tat battle is stunning. Consider that by Google researchers’ own estimates, there are now more than a trillion Web pages. “There are not that many things in the world,” says Rich Skrenta, a Silicon Valley entrepreneur and programmer who, in a series of business ventures, has been trying to instill some order into the vast clutter of the Web.

Mr. Skrenta has recently embarked on the daunting mission of developing a search engine that will compete with Google and its main competitor, Bing, Microsoft’s less-used but no less confounding entry in the field. “The Web has become wrecked and links aren’t useful anymore,” Mr. Skrenta says. “It’s a hall of mirrors.”

One of the great innovations of Google, and the source of its wealth, was auctioning off search terms. But Mr. Skrenta contends that when Google assigned an economic value to links on a Web page, many pages filled with links that served only as bait for Google’s search engine. That’s an important way to make money because that search engine drives so much of the Internet’s traffic. Hundreds of billions of those kinds of pages were created with only one purpose, and that is to fool Google. A result is less useful searches for information.

It may sound like a stretch to take on an entrenched competitor that even a giant like Microsoft, after spending hundreds of millions of dollars, has barely been able to budge. But Mr. Skrenta has found investors to give him $25 million to try.

Google keeps making its search engine faster and easier — I had to type only the letters “kidn” to get information on kidney stone treatments — and the company notes that the billions of searches each day and its 66 percent market share prove that consumers find it useful. A long list of challengers who have fallen seem to prove that point — Alta Vista, Yahoo, Ask Jeeves, Cuil, Kosmix, SearchMe and Wikisearch, to name only a few.

Still, Mr. Skrenta, who sold his first company to Netscape and then was a co-founder of Topix, which aggregates local news, is staking his reputation and his investors’ money on a search engine called Blekko.com. Mr. Skrenta pitched his investors with the notion that there is still money to be made in search because of the high price that the two big competitors get for search terms and advertising. If Blekko could get even a small part of that revenue, the investors would reap a healthy return on their money.

His idea is to concentrate the search. Only a relatively small number of the Web’s total pages are visited — in the tens of millions rather than in the hundreds of billions. In his view, it should be possible to simplify a search engine so it could satisfy a vast majority of searchers.

Blekko uses a search algorithm like Google’s or Bing’s but also gets humans, mostly volunteers, to identify the sites they know, trust and visit most often and to put those at the top of the search results.

“The best site may not have the best S.E.O.,” Mr. Skrenta says.

It is a Wikipedia model — or Huffington Post model — applied to search. Some people apparently will work for no pay if they are convinced that their efforts will help or influence others. Experts who care enough about a topic edit the results. For instance, editors trawling the health results may give a higher ranking to the Web pages written by medical experts at the Cleveland Clinic and the Mayo Clinic than those generated for eHow by writers getting paid a few dollars per piece.

Using Blekko takes a little more effort. It works the way Google or Bing does, but if you want cleaner search results you must type in a slash mark and a category. The company calls them slashtags. Typing “/conservative” after “taxes,” for instance, would give you sites written from the right; “/liberal” gives you the other side.

Blekko also sorts results for financial advice or sports. And it has some rather esoteric experts who have edited results for “gluten free” and “material safety data sheets,” a category containing information on the properties of myriad substances. If someone tries to game the results, an expert presumably would block the efforts.

Not that there is a lot of gaming of Blekko right now. Only 750,000 searches were made in April, about the number made on Google in 22 seconds. But for Blekko, it’s a big number — 30 percent more than were made in March.

So far, the S.E.O. experts haven’t taken aim at Blekko, but when they do start trying to manipulate its search results, Mr. Skrenta will know he has arrived. As he puts it, “They say that’s a problem you want to have.”

Article source: http://feeds.nytimes.com/click.phdo?i=4dee4c920c0af26255e038859f15905e