The World Wide Web Wanderer, also simply called The Wanderer, was a
Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offici ...
-based
web crawler
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing (''web spid ...
that was first deployed in June 1993 to measure the size of the
World Wide Web
The World Wide Web (WWW), commonly known as the Web, is an information system enabling documents and other web resources to be accessed over the Internet.
Documents and downloadable media are made available to the network through web se ...
. The Wanderer was developed at the
Massachusetts Institute of Technology
The Massachusetts Institute of Technology (MIT) is a private land-grant research university in Cambridge, Massachusetts. Established in 1861, MIT has played a key role in the development of modern technology and science, and is one of the ...
by Matthew Gray, who also created back in 1993 one of the 100 first web servers in history: www.mit.edu, as of 2022, he has spent more than 15 years as a software engineer at
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
. The crawler was used to generate an index called the ''Wandex'' later in 1993. While ''the Wanderer'' was probably the first
web robot
An Internet bot, web robot, robot or simply bot, is a software application that runs automated tasks (scripts) over the Internet, usually with the intent to imitate human activity on the Internet, such as messaging, on a large scale. An Internet b ...
, and, with its index, clearly had the potential to become a general-purpose
WWW search engine, the author does not make this claim and elsewhere
Brian LaMacchia's PhD thesis, section 1.2.3
/ref> it is stated that this was not its purpose. The Wanderer charted the growth of the web until late 1995.
References
{{reflist
External links
Growth of the Web Report
Defunct internet search engines