The World Wide Web Wanderer, also simply called The Wanderer, was a

Perl Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offic ...

-based

web crawler A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing (''web s ...

that was first deployed in June 1993 to measure the size of the

World Wide Web The World Wide Web (WWW), commonly known as the Web, is an information system enabling documents and other web resources to be accessed over the Internet. Documents and downloadable media are made available to the network through web ...

. The Wanderer was developed at the

Massachusetts Institute of Technology The Massachusetts Institute of Technology (MIT) is a private land-grant research university in Cambridge, Massachusetts. Established in 1861, MIT has played a key role in the development of modern technology and science, and is one of th ...

by Matthew Gray, who also created back in 1993 one of the 100 first web servers in history: www.mit.edu, as of 2022, he has spent more than 15 years as a software engineer at

Google Google LLC () is an American Multinational corporation, multinational technology company focusing on Search Engine, search engine technology, online advertising, cloud computing, software, computer software, quantum computing, e-commerce, ar ...

. The crawler was used to generate an index called the ''Wandex'' later in 1993. While ''the Wanderer'' was probably the first

web robot An Internet bot, web robot, robot or simply bot, is a software application that runs automated tasks (scripts) over the Internet, usually with the intent to imitate human activity on the Internet, such as messaging, on a large scale. An Internet b ...

, and, with its index, clearly had the potential to become a general-purpose WWW search engine, the author does not make this claim and elsewhereBrian LaMacchia's PhD thesis, section 1.2.3
/ref> it is stated that this was not its purpose. The Wanderer charted the growth of the web until late 1995.

References

{{reflist

External links

Growth of the Web Report
Defunct internet search engines