The CLEVER project was a research project in
Web search
Web most often refers to:
* Spider web, a silken structure created by the animal
* World Wide Web or the Web, an Internet-based hypertext system
Web, WEB, or the Web may also refer to:
Computing
* WEB, a literate programming system created by ...
led by
Jon Kleinberg
Jon Michael Kleinberg (born 1971) is an American computer scientist and the Tisch University Professor of Computer Science and Information Science at Cornell University known for his work in algorithms and networks. He is a recipient of the Nevanl ...
at
IBM's
Almaden Research Center
IBM Research is the research and development division for IBM, an American multinational information technology company headquartered in Armonk, New York, with operations in over 170 countries. IBM Research is the largest industrial research org ...
. Techniques developed in CLEVER included various forms of
link analysis
In network theory, link analysis is a data-analysis technique used to evaluate relationships (connections) between nodes. Relationships may be identified among various types of nodes (objects), including organizations, people and transactions. Lin ...
, including the
HITS algorithm Hyperlink-Induced Topic Search (HITS; also known as hubs and authorities) is a link analysis algorithm that rates Web pages, developed by Jon Kleinberg. The idea behind Hubs and Authorities stemmed from a particular insight into the creation of we ...
.
Features
The CLEVER search engine incorporates several
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
s that make use of the
Web's
hyperlink structure for discovering high-quality information. It can be exceedingly difficult to locate resources on the World Wide Web that are both high-quality and relevant to a user's informational needs. Traditional automated search methods for locating information on the Web are easily overwhelmed by low-quality and unrelated content. Second generation search engines have to have effective methods for focusing on the most authoritative documents. The rich structure implicit in hyperlinks among Web documents offers a simple, and effective, means to deal with many of these problems.
Members of the Clever project have come up with a mathematical algorithm that views the Net as simply web pages pointing at each other. It also takes into account the notion of hubs, which point to quality content and link information together, and the idea of authority pages, which are often written by specialists in certain fields.
Bill Cody, senior manager of exploratory data management research at
IBM's Almaden Research Center, said: "Web searches provide a lot of information, some good, some bad. But the people providing good hubs usually point to authority pages and authority pages generally know of good hubs. The algorithm enables us to find them and so provide users with quality information rather than the regular list of irrelevant web pages."
He added that the algorithm had also been used to find Internet based communities using the same principle of finding links between like and like.
It was used to discover 300,000 communities worldwide, only four per cent of which turned out to be spurious. About two thirds of these still existed, he claimed, with about half now appearing on
Yahoo
Yahoo! (, styled yahoo''!'' in its logo) is an American web services provider. It is headquartered in Sunnyvale, California and operated by the namesake company Yahoo Inc., which is 90% owned by investment funds managed by Apollo Global Manage ...
as mature communities.
Cody said that the tool could be used for targeted advertising purposes or for enabling users to find out more information about incipient communities, but declined to say whether IBM had plans to turn Clever into a commercial product or not.
References
External links
Clever project home page
{{DEFAULTSORT:Clever Project
IBM software