Google News Search Leaps Ahead

Google has dramatically enhanced its news search service, serving up a portal of real-time news drawn from more than 4,000 sources worldwide.

Until recently, Google's news search has been competent, but less useful than other news-aggregating services such as AllTheWeb's News Search and Yahoo's Full coverage. The new enhancements establish Google as one of the premier news finding and filtering destinations on the web.

Like Yahoo's Full Coverage, Google News Search now looks like a portal, with links to the top headlines organized into categories such as Top Stories, World, Business, Sports and so on. Each category has its area on the News Search home page, with headlines, descriptions and links for the top two or three stories.

"The page looks very different than the average Google page," said Marissa Mayer, Google product manager. That's because it's packed with headlines, descriptions, thumbnail photos and dozens of links to the sources of the articles online.

Unlike Yahoo Full Coverage, however, Google News Search isn't assembled by human editors who select and format the news. Google's process is fully automated. News stories are chosen and the page is updated without human intervention. Google crawls news sources constantly, and uses real-time ranking algorithms to determine which stories are the most important at the moment -- in theory highlighting the sources with the "best" coverage of news events.

Each top story is presented with a headline linked directly to the source. Beneath the headline is a short description, name of the source, and the time when the article was last crawled, ranging from a few minutes to several hours ago.

Beneath the main headline and description are two full headlines from other sources, followed by four or five links to stories with only the name of the publication indicated. Finally, there are links to "related" stories from other sources.

This design makes it easy to quickly scan the headlines while having the option of reading multiple accounts of a story from different news sources -- from literally thousands of sources, for some stories.

Each major category has a link at the top of its respective section that allows you to scan news just within the category. Tabs on the upper left of each page also allow you focus in on Top Stories, World, U.S., Business, Sci/Tech, Sports, Entertainment and Health categories.

Unlike many news aggregators that simply "scrape" headlines and links from news sites, Google's news crawler indexes the full text of articles. This approach offers several unique benefits.

For example, full text indexing allows true searching, rather than just browsing of headlines. Creating a full text index of news also allows Google to cluster related news stories, around what Mayer calls a "centroid" of keywords. "A cluster is defined by a centroid of keywords, and all the articles have some of those key words in them," she said.

The process uses artificial intelligence in addition to traditional information retrieval techniques to match keywords with stories. Mayer says this approach to identifying related articles means that the relative importance of each article is "baked in," which is how the top sources for each story are selected.

Other factors used in calculating the relevance of top and related stories include how recently articles were published, and the reputation of the source. When you actually do a search, these factors are also applied in addition to keyword analysis to determine how closely particular stories match your query.

On search results pages, a link allows you to override the default ranking by relevance and order results by date -- a feature that's particularly helpful for monitoring breaking news.

Google's decision to index the full text of news sources rather than simply scraping headlines posed a major challenge for implementing the new service. The vast diversity and typically cluttered design of most online news formats is more difficult to crawl and index than many other types of web sites. "Article extraction has proven to be one of the most difficult aspects of the project," said Mayer.

Google crawls its 4,000 sources of news continuously and in real time. According to Mayer, the crawler continuously computes what's likely to change on each news source, and when the change is likely to occur. To expedite the discovery of new stories, the crawler tends to hit hub or major section pages frequently, to see what new links are there.

While the news sources are crawled constantly and individual news stories are updated continuously, the entire set of displayed stories is "auto generated" every 15 minutes. A message in the upper right corner of the main news page indicates when it was last generated.

Google's updated news search is an exceptionally powerful tool for web users. It's still in beta, so there are still a few rough edges, but all told it's one of the best news browse and search portals currently operating on the web.

Google News Search
http://news.google.com

News Search Engines
http://searchenginewatch.com/links/news.html

Hitwise Data: Google Rules

Google Receives 68 Percent of U.S. Searches in May 2008

Search leader continues record growth - up 5 percent year-over-year;

Google accounted for 87 percent of searches in UK

NEW YORK, NY – June 10, 2008 – Google accounted for 68.29 percent of all U.S. searches in the four weeks ending May 31, 2008, Hitwise announced today. Yahoo! Search, MSN Search and Ask.com each received 19.95, 5.89 and 4.23 percent respectively. The remaining 41 search engines in the Hitwise Search Engine Analysis Tool accounted for 1.63 percent of U.S. searches.

Percentage of U.S. Searches Among Leading Search Engine Providers

Domain	May-08	Apr-08	May-07
www.google.com	68.29%	67.90%	65.13%
search.yahoo.com	19.95%	20.28%	20.89%
search.msn.com	5.89% *	6.26% *	7.61% *
www.ask.com	4.23%	4.17%	3.92%

Note: Data is based on four week rolling periods (ending 5/31/ 2007, 4/26/08, 5/26/2007 from the Hitwise sample of 10 million U.S. Internet users. * - includes executed searches on Live.com and MSN Search but does not include searches on Club.Live.com.

Source Hitwise

In the U.K. market, Google search properties (Google.co.uk and Google.com) accounted for 87 percent of all UK searches in May 2008 representing a 12 percent increase compared to May 2007. Yahoo! search properties accounted for 4.09 percent of UK searches in May 2008, a 2 percent increase compared to April 2008. MSN search properties accounted for 3.72 percent and Ask search properties accounted for 3.07 percent of searches. MSN increased two percent compared to April 2008 and Ask increased 6 percent.

Percentage of U.K. Searches Among Leading Search Engine Providers

Domain	May-08	Apr.-08	May-07
Google Properties	87.30%	87.69%	78.28%
Yahoo! Properties	4.09%	4.01%	8.58%
Microsoft Properties	3.72%	3.65%	5.46%
Ask Properties	3.07%	2.89%	4.96%

Note: Data is based on UK Internet usage over the four week rolling periods (ending 5/31/ 2007, 4/26/08, 5/26/2007) from the Hitwise sample of 8.4 million UK Internet users. Note that the percentages for the search properties include the .uk and .com domains.

Source: Hitwise UK

Google an Increasing Source of Traffic to Key U.S. Industries
Search engines continue to be the primary way Internet users navigate to key industry categories. Comparing May 2008 to May 2007, the Travel, News and Media, Entertainment, Business and Finance, Sports, Online Video and Social Networking categories showed double digit increases in their share of traffic coming directly from search engines.

U.S. Category Upstream Traffic from Search Engines and Google - May 2008

Category	Percent of Category Traffic from Search Engines, May-08	Percentage Change in Share of Traffic From, Search Engines, May-08 - May-07	Percent of Category Traffic from Google, May-08	Percent Change in Share of Traffic From Google, May-08 - May-07
Health and Medical	45.76%	3%	30.86%	5%
Travel	34.81%	11%	24.26%	21%
Shopping and Classifieds	25.48%	2%	16.84%	8%
News and Media	21.70%	7%	14.53%	10%
Entertainment	24.33%	17%	15.76%	22%
Business and Finance	18.15%	14%	11.73%	22%
Sports	13.09%	17%	8.81%	24%
Online Video*	29.94%	37%	20.78%	52%
Social Networking*	16.50%	18%	9.98%	21%

All figures are based on U.S. data from the Hitwise sample of 10 million Internet users. * denotes custom category

Source: Hitwise

About Hitwise
Hitwise is the leading online competitive intelligence service. Only Hitwise provides its 1,400 clients around the world with daily insights on how their customers interact with a broad range of competitive websites, and how their competitors use different tactics to attract online customers.

Since 1997, Hitwise has pioneered a unique, network-based approach to Internet measurement. Through relationships with ISPs around the world, Hitwise’s patented methodology anonymously captures the online usage, search and conversion behavior of 25 million Internet users. This unprecedented volume of Internet usage data is seamlessly integrated into an easy to use, web-based service, designed to help marketers better plan, implement and report on a range of online marketing programs.