![Demystifying Search Engine Optimization -viz notes- -wcyvr (8084222814)](https://upload.wikimedia.org/wikipedia/commons/e/ea/Demystifying_Search_Engine_Optimization_-viz_notes-_-wcyvr_%288084222814%29.jpg)
Search engine optimization (SEO) is the process of improving the quality and quantity of
website traffic
Web traffic is the data sent and received by visitors to a website. Since the mid-1990s, web traffic has been the largest portion of Internet traffic. Sites monitor the incoming and outgoing traffic to see which parts or pages of their site are ...
to a
website
A website (also written as a web site) is a collection of web pages and related content that is identified by a common domain name and published on at least one web server. Examples of notable websites are Google Search, Google, Facebook, Amaz ...
or a
web page from
search engine
A search engine is a software system designed to carry out web searches. They search the World Wide Web in a systematic way for particular information specified in a textual web search query. The search results are generally presented in a ...
s. SEO targets unpaid traffic (known as "natural" or "
organic" results) rather than direct traffic or
paid traffic. Unpaid traffic may originate from different kinds of searches, including
image search
An image retrieval system is a computer system used for browsing, searching and retrieving images from a large database of digital images. Most traditional and common methods of image retrieval utilize some method of adding metadata such as captio ...
,
video search A video search engine is a web-based search engine which Web crawler, crawls the web for video content. Some video search engines parse externally hosted content while others allow content to be uploaded and hosted on their own servers. Some engines ...
,
academic search
Academic Search is a monthly indexing service. It was first published in 1997 by EBSCO Publishing in Ipswich, Massachusetts. Its academic focus is international universities, covering social science, education, psychology, and other subjects. Pu ...
,
news search, and industry-specific
vertical search
A vertical search engine is distinct from a general web search engine, in that it focuses on a specific segment of online content. They are also called specialty or topical search engines. The vertical content area may be based on topicality, media ...
engines.
As an
Internet marketing
The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices. It is a '' network of networks'' that consists of private, pub ...
strategy, SEO considers how search engines work, the computer-programmed
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algorithms are used as specificat ...
s that dictate search engine behavior, what people search for, the actual search terms or
keywords typed into search engines, and which search engines are preferred by their targeted audience. SEO is performed because a website will receive more visitors from a search engine when websites rank higher on the
search engine results page
Search Engine Results Pages (SERP) are the pages displayed by search engines in response to a query by a user. The main component of the SERP is the listing of results that are returned by the search engine in response to a keyword query. The p ...
(SERP). These visitors can then potentially be converted into customers.
History
Webmaster
A webmaster is a person responsible for maintaining one or more websites. The title may refer to web architects, web developers, site authors, website administrators, website owners, website coordinators, or website publishers.
The duties of ...
s and content providers began optimizing websites for search engines in the mid-1990s, as the first search engines were cataloging the early
Web. Initially, all webmasters only needed to submit the address of a page, or
URL, to the various engines, which would send a
web crawler
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing (''web spid ...
to ''crawl'' that page, extract links to other pages from it, and return information found on the page to be
indexed. The process involves a search engine spider downloading a page and storing it on the search engine's own server. A second program, known as an
indexer, extracts information about the page, such as the words it contains, where they are located, and any weight for specific words, as well as all links the page contains. All of this information is then placed into a scheduler for crawling at a later date.
Website owners recognized the value of a high
ranking
A ranking is a relationship between a set of items such that, for any two items, the first is either "ranked higher than", "ranked lower than" or "ranked equal to" the second.
In mathematics, this is known as a weak order or total preorder of o ...
and visibility in search engine results, creating an opportunity for both
white hat
White hat, white hats, or white-hat may refer to:
Art, entertainment, and media
* White hat, a way of thinking in Edward de Bono's book ''Six Thinking Hats''
* White hat, part of black and white hat symbolism in film
Other uses
* White hat (compu ...
and
black hat
Black hat, blackhats, or black-hat refers to:
Arts, entertainment, and media
* Black hat (computer security), a hacker who violates computer security for little reason beyond maliciousness or for personal gain
* Black hat, part of black and whit ...
SEO practitioners. According to industry analyst
Danny Sullivan
Daniel John Sullivan III (born March 9, 1950), better known as Danny Sullivan, is an American former racing driver. He earned 17 wins in the CART Indy Car World Series, including the 1985 Indianapolis 500. Sullivan won the 1988 CART Champions ...
, the phrase "search engine optimization" probably came into use in 1997. Sullivan credits Bruce Clay as one of the first people to popularize the term.
Early versions of search
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algorithms are used as specificat ...
s relied on webmaster-provided information such as the keyword
meta tag
Meta elements are tags used in HTML and XHTML documents to provide structured metadata about a Web page.
They are part of a web page's head section. Multiple Meta elements with different attributes can be used on the same page. Meta elements can ...
or index files in engines like
ALIWEB. Meta tags provide a guide to each page's content. Using metadata to index pages was found to be less than reliable, however, because the webmaster's choice of keywords in the meta tag could potentially be an inaccurate representation of the site's actual content. Flawed data in meta tags, such as those that were not accurate, complete, or falsely attributes, created the potential for pages to be mischaracterized in irrelevant searches. Web content providers also manipulated some attributes within the
HTML
The HyperText Markup Language or HTML is the standard markup language for documents designed to be displayed in a web browser. It can be assisted by technologies such as Cascading Style Sheets (CSS) and scripting languages such as JavaSc ...
source of a page in an attempt to rank well in search engines. By 1997, search engine designers recognized that webmasters were making efforts to rank well in their search engine and that some webmasters were even
manipulating their rankings in search results by stuffing pages with excessive or irrelevant keywords. Early search engines, such as
Altavista
AltaVista was a Web search engine established in 1995. It became one of the most-used early search engines, but lost ground to Google and was purchased by Yahoo! in 2003, which retained the brand, but based all AltaVista searches on its own sear ...
and
Infoseek
Infoseek (also known as the "big yellow") was an American internet search engine founded in 1994 by Steve Kirsch.
Infoseek was originally operated by the Infoseek Corporation, headquartered in Sunnyvale, California. Infoseek was bought by The Wa ...
, adjusted their algorithms to prevent webmasters from manipulating rankings.
By heavily relying on factors such as
keyword density, which were exclusively within a webmaster's control, early search engines suffered from abuse and ranking manipulation. To provide better results to their users, search engines had to adapt to ensure their
results pages showed the most relevant search results, rather than unrelated pages stuffed with numerous keywords by unscrupulous webmasters. This meant moving away from heavy reliance on term density to a more holistic process for scoring semantic signals.
Since the success and popularity of a search engine are determined by its ability to produce the most relevant results to any given search, poor quality or irrelevant search results could lead users to find other search sources. Search engines responded by developing more complex ranking algorithms, taking into account additional factors that were more difficult for webmasters to manipulate.
Companies that employ overly aggressive techniques can get their client websites banned from the search results. In 2005, the ''
Wall Street Journal
''The Wall Street Journal'' is an American business-focused, international daily newspaper based in New York City, with international editions also available in Chinese and Japanese. The ''Journal'', along with its Asian editions, is published ...
'' reported on a company,
Traffic Power, which allegedly used high-risk techniques and failed to disclose those risks to its clients. ''
Wired
''Wired'' (stylized as ''WIRED'') is a monthly American magazine, published in print and online editions, that focuses on how emerging technologies affect culture, the economy, and politics. Owned by Condé Nast, it is headquartered in San ...
'' magazine reported that the same company sued blogger and SEO Aaron Wall for writing about the ban.
Google's
Matt Cutts
Matthew Cutts (born 1972 or 1973) is an American software engineer. Cutts is the former Administrator of the United States Digital Service. He was first appointed as acting administrator, to later be confirmed as full administrator in October 20 ...
later confirmed that Google did in fact ban Traffic Power and some of its clients.
Some search engines have also reached out to the SEO industry and are frequent sponsors and guests at SEO conferences, webchats, and seminars. Major search engines provide information and guidelines to help with website optimization.
Google has a
Sitemaps
The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site. It allows webmasters to include additional information about ea ...
program to help webmasters learn if Google is having any problems indexing their website and also provides data on Google traffic to the website.
Bing Webmaster Tools
Bing Webmaster Tools (previously the ''Bing Webmaster Center'') is a free service as part of Microsoft's Bing search engine which allows webmasters to add their websites to the Bing index crawler, see their site's performance in Bing (clicks, impre ...
provides a way for webmasters to submit a sitemap and web feeds, allows users to determine the "crawl rate," and track the web pages index status.
In 2015, it was reported that
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
was developing and promoting mobile search as a key feature within future products. In response, many brands began to take a different approach to their Internet marketing strategies.
Relationship with Google
In 1998, two graduate students at
Stanford University
Stanford University, officially Leland Stanford Junior University, is a private research university in Stanford, California. The campus occupies , among the largest in the United States, and enrolls over 17,000 students. Stanford is consider ...
,
Larry Page
Lawrence Edward Page (born March 26, 1973) is an American business magnate, computer scientist and internet entrepreneur. He is best known for co-founding Google with Sergey Brin.
Page was the chief executive officer of Google from 1997 unti ...
and
Sergey Brin
Sergey Mikhailovich Brin (russian: link=no, Сергей Михайлович Брин; born August 21, 1973) is an American business magnate, computer scientist, and internet entrepreneur, who co-founded Google with Larry Page. Brin was th ...
, developed "Backrub," a search engine that relied on a mathematical algorithm to rate the prominence of web pages. The number calculated by the algorithm,
PageRank
PageRank (PR) is an algorithm used by Google Search to rank webpages, web pages in their search engine results. It is named after both the term "web page" and co-founder Larry Page. PageRank is a way of measuring the importance of website pages. A ...
, is a function of the quantity and strength of
inbound links.
PageRank estimates the likelihood that a given page will be reached by a web user who randomly surfs the web and follows links from one page to another. In effect, this means that some links are stronger than others, as a higher PageRank page is more likely to be reached by the random web surfer.
Page and Brin founded Google in 1998. Google attracted a loyal following among the growing number of
Internet
The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices. It is a '' network of networks'' that consists of private, pub ...
users, who liked its simple design.
Off-page factors (such as PageRank and hyperlink analysis) were considered as well as on-page factors (such as keyword frequency,
meta tags
Meta elements are tags used in HTML and XHTML documents to provide structured metadata about a Web page.
They are part of a web page's head section. Multiple Meta elements with different attributes can be used on the same page. Meta elements can ...
, headings, links and site structure) to enable Google to avoid the kind of manipulation seen in search engines that only considered on-page factors for their rankings. Although PageRank was more difficult to
game
A game is a structured form of play (activity), play, usually undertaken for enjoyment, entertainment or fun, and sometimes used as an educational tool. Many games are also considered to be work (such as professional players of spectator s ...
, webmasters had already developed link-building tools and schemes to influence the
Inktomi
Inktomi Corporation was a company that provided software for Internet service providers (ISPs). It was incorporated in Delaware and headquartered in Foster City, California, United States. Customers included Microsoft, HotBot, Amazon.com, eBay, ...
search engine, and these methods proved similarly applicable to gaming PageRank. Many sites focus on exchanging, buying, and selling links, often on a massive scale. Some of these schemes, or
link farm
On the World Wide Web, a link farm is any group of websites that all hyperlink to other sites in the group for the purpose of increasing SEO rankings. In graph theoretic terms, a link farm is a clique. Although some link farms can be created ...
s, involved the creation of thousands of sites for the sole purpose of
link spamming.
By 2004, search engines had incorporated a wide range of undisclosed factors in their ranking algorithms to reduce the impact of link manipulation. In June 2007, ''The New York Times Saul Hansell stated Google ranks sites using more than 200 different signals.
The leading search engines, Google,
Bing
Bing most often refers to:
* Bing Crosby (1903–1977), American singer
* Microsoft Bing, a web search engine
Bing may also refer to:
Food and drink
* Bing (bread), a Chinese flatbread
* Bing (soft drink), a UK brand
* Bing cherry, a varie ...
, and
Yahoo
Yahoo! (, styled yahoo''!'' in its logo) is an American web services provider. It is headquartered in Sunnyvale, California and operated by the namesake company Yahoo! Inc. (2017–present), Yahoo Inc., which is 90% owned by investment funds ma ...
, do not disclose the algorithms they use to rank pages. Some SEO practitioners have studied different approaches to search engine optimization and have shared their personal opinions. Patents related to search engines can provide information to better understand search engines. In 2005, Google began personalizing search results for each user. Depending on their history of previous searches, Google crafted results for logged in users.
In 2007, Google announced a campaign against paid links that transfer PageRank. On June 15, 2009, Google disclosed that they had taken measures to mitigate the effects of PageRank sculpting by use of the
nofollow
nofollow is a setting on a web page hyperlink that directs search engines not to use the link for page ranking calculations. It is specified in the page as a type of link relation; that is: <a rel="nofollow" ...>. Because search engines ...
attribute on links.
Matt Cutts
Matthew Cutts (born 1972 or 1973) is an American software engineer. Cutts is the former Administrator of the United States Digital Service. He was first appointed as acting administrator, to later be confirmed as full administrator in October 20 ...
, a well-known software engineer at Google, announced that Google Bot would no longer treat any no follow links, in the same way, to prevent SEO service providers from using nofollow for PageRank sculpting. As a result of this change, the usage of nofollow led to evaporation of PageRank. In order to avoid the above, SEO engineers developed alternative techniques that replace nofollowed tags with obfuscated
JavaScript
JavaScript (), often abbreviated as JS, is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS. As of 2022, 98% of Website, websites use JavaScript on the Client (computing), client side ...
and thus permit PageRank sculpting. Additionally, several solutions have been suggested that include the usage of
iframes,
Flash
Flash, flashes, or FLASH may refer to:
Arts, entertainment, and media
Fictional aliases
* Flash (DC Comics character), several DC Comics superheroes with super speed:
** Flash (Barry Allen)
** Flash (Jay Garrick)
** Wally West, the first Kid F ...
, and JavaScript.
In December 2009, Google announced it would be using the web search history of all its users in order to populate search results. On June 8, 2010 a new web indexing system called
Google Caffeine was announced. Designed to allow users to find news results, forum posts, and other content much sooner after publishing than before, Google Caffeine was a change to the way Google updated its index in order to make things show up quicker on Google than before. According to Carrie Grimes, the software engineer who announced Caffeine for Google, "Caffeine provides 50 percent fresher results for web searches than our last index..."
Google Instant
Google Search (also known simply as Google) is a search engine provided by Google. Handling more than 3.5 billion searches per day, it has a 92% share of the global search engine market. It is also the most-visited website in the world.
The ...
, real-time-search, was introduced in late 2010 in an attempt to make search results more timely and relevant. Historically site administrators have spent months or even years optimizing a website to increase search rankings. With the growth in popularity of social media sites and blogs, the leading engines made changes to their algorithms to allow fresh content to rank quickly within the search results.
In February 2011, Google announced the
Panda
The giant panda (''Ailuropoda melanoleuca''), also known as the panda bear (or simply the panda), is a bear species endemic to China. It is characterised by its bold black-and-white coat and rotund body. The name "giant panda" is sometimes use ...
update, which penalizes websites containing content duplicated from other websites and sources. Historically websites have copied content from one another and benefited in search engine rankings by engaging in this practice. However, Google implemented a new system that punishes sites whose content is not unique. The 2012
Google Penguin
Google Penguin was a codename for a Google algorithm update that was first announced on April 24, 2012. The update was aimed at decreasing search engine rankings of websites that violate Google's Webmaster Guidelines by using now declared Grey Ha ...
attempted to penalize websites that used manipulative techniques to improve their rankings on the search engine. Although Google Penguin has been presented as an algorithm aimed at fighting web spam, it really focuses on spammy links by gauging the quality of the sites the links are coming from. The 2013
Google Hummingbird
Hummingbird is the codename given to a significant algorithm change in Google Search in 2013. Its name was derived from the speed and accuracy of the hummingbird. The change was announced on September 26, 2013, having already been in use for a m ...
update featured an algorithm change designed to improve Google's natural language processing and semantic understanding of web pages. Hummingbird's language processing system falls under the newly recognized term of "
conversational search
An intelligent virtual assistant (IVA) or intelligent personal assistant (IPA) is a software agent that can perform tasks or services for an individual based on commands or questions. The term "chatbot" is sometimes used to refer to virtual ...
," where the system pays more attention to each word in the query in order to better match the pages to the meaning of the query rather than a few words. With regards to the changes made to search engine optimization, for content publishers and writers, Hummingbird is intended to resolve issues by getting rid of irrelevant content and spam, allowing Google to produce high-quality content and rely on them to be 'trusted' authors.
In October 2019, Google announced they would start applying
BERT
Bert or BERT may refer to:
Persons, characters, or animals known as Bert
*Bert (name), commonly an abbreviated forename and sometimes a surname
*Bert, a character in the poem "Bert the Wombat" by The Wiggles; from their 1992 album Here Comes a Son ...
models for English language search queries in the US. Bidirectional Encoder Representations from Transformers (BERT) was another attempt by Google to improve their natural language processing, but this time in order to better understand the search queries of their users. In terms of search engine optimization, BERT intended to connect users more easily to relevant content and increase the quality of traffic coming to websites that are ranking in the Search Engine Results Page.
Methods
Getting indexed
![PageRanks-Example](https://upload.wikimedia.org/wikipedia/commons/f/fb/PageRanks-Example.svg)
The leading search engines, such as Google, Bing, and Yahoo!, use
crawlers to find pages for their algorithmic search results. Pages that are linked from other search engine-indexed pages do not need to be submitted because they are found automatically. The
Yahoo! Directory
The Yahoo! Directory was a web directory which at one time rivaled DMOZ in size. The directory was Yahoo!'s first offering and started in 1994 under the name Jerry and David's Guide to the World Wide Web. When Yahoo! changed its main results to c ...
and
DMOZ
DMOZ (from ''directory.mozilla.org'', an earlier domain name, stylized in lowercase in its logo) was a multilingual open-content directory of World Wide Web links. The site and community who maintained it were also known as the Open Directory ...
, two major directories which closed in 2014 and 2017 respectively, both required manual submission and human editorial review. Google offers
Google Search Console
Google Search Console is a web service by Google which allows webmasters to check indexing status, search queries, crawling errors and optimize visibility of their websites.
Until 20 May 2015, the service was called Google Webmaster Tools. In ...
, for which an XML
Sitemap
A sitemap is a list of pages of a web site within a domain.
There are three primary kinds of sitemap:
* Sitemaps used during the planning of a website by its designers.
* Human-visible listings, typically hierarchical, of the pages on a site.
* St ...
feed can be created and submitted for free to ensure that all pages are found, especially pages that are not discoverable by automatically following links in addition to their URL submission console. Yahoo! formerly operated a paid submission service that guaranteed to crawl for a
cost per click
Pay-per-click (PPC) is an internet advertising model used to drive traffic to websites, in which an advertiser pays a publisher (typically a search engine, website owner, or a network of websites) when the ad is clicked.
Pay-per-click is usually ...
; however, this practice was discontinued in 2009.
Search engine
A search engine is a software system designed to carry out web searches. They search the World Wide Web in a systematic way for particular information specified in a textual web search query. The search results are generally presented in a ...
crawlers may look at a number of different factors when
crawling a site. Not every page is indexed by search engines. The distance of pages from the root directory of a site may also be a factor in whether or not pages get crawled.
Today, most people are searching on Google using a mobile device. In November 2016, Google announced a major change to the way crawling websites and started to make their index mobile-first, which means the mobile version of a given website becomes the starting point for what Google includes in their index. In May 2019, Google updated the rendering engine of their crawler to be the latest version of Chromium (74 at the time of the announcement). Google indicated that they would regularly update the
Chromium
Chromium is a chemical element with the symbol Cr and atomic number 24. It is the first element in group 6. It is a steely-grey, lustrous, hard, and brittle transition metal.
Chromium metal is valued for its high corrosion resistance and hardne ...
rendering engine to the latest version. In December 2019, Google began updating the User-Agent string of their crawler to reflect the latest Chrome version used by their rendering service. The delay was to allow webmasters time to update their code that responded to particular bot User-Agent strings. Google ran evaluations and felt confident the impact would be minor.
Preventing crawling
To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard
robots.txt
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to indicate to visiting web crawlers and other web robots which portions of the site they are allowed to visit.
Th ...
file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a
meta tag
Meta elements are tags used in HTML and XHTML documents to provide structured metadata about a Web page.
They are part of a web page's head section. Multiple Meta elements with different attributes can be used on the same page. Meta elements can ...
specific to robots (usually
). When a search engine visits a site, the robots.txt located in the
root directory
In a computer file system, and primarily used in the Unix and Unix-like operating systems, the root directory is the first or top-most directory in a hierarchy. It can be likened to the trunk of a tree, as the starting point where all branches ...
is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish to crawl. Pages typically prevented from being crawled include login-specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam. In 2020, Google
sunsetted the standard (and open-sourced their code) and now treats it as a hint not a directive. To adequately ensure that pages are not indexed, a page-level robot's meta tag should be included.
Increasing prominence
A variety of methods can increase the prominence of a webpage within the search results.
Cross linking
In chemistry and biology a cross-link is a bond or a short sequence of bonds that links one polymer chain to another. These links may take the form of covalent bonds or ionic bonds and the polymers can be either synthetic polymers or natural ...
between pages of the same website to provide more links to important pages may improve its visibility. Page design makes users trust a site and want to stay once they find it. When people bounce off a site, it counts against the site and affects its credibility.
Writing content that includes frequently searched keyword phrases so as to be relevant to a wide variety of search queries will tend to increase traffic. Updating content so as to keep search engines crawling back frequently can give additional weight to a site. Adding relevant keywords to a web page's metadata, including the
title tag and meta description, will tend to improve the relevancy of a site's search listings, thus increasing traffic.
URL canonicalization
A canonical link element is an HTML element that helps webmasters prevent duplicate content issues in search engine optimization by specifying the "canonical" or "preferred" version of a web page. It is described in RFC 6596, which went live in Apr ...
of web pages accessible via multiple URLs, using the
canonical link element
A canonical link element is an HTML element that helps webmasters prevent duplicate content issues in search engine optimization by specifying the "canonical" or "preferred" version of a web page. It is described in RFC 6596, which went live in Apr ...
or via
301 redirects can help make sure links to different versions of the URL all count towards the page's link popularity score. These are known as incoming links, which point to the URL and can count towards the page link's popularity score, impacting the credibility of a website.
Also, in recent times Google is giving more priority to the below elements for SERP (Search Engine Ranking Position).
* HTTPS version (Secure Site)
* Page Speed
* Structured Data
* Mobile Compatibility
* AMP (Accelerated Mobile Pages)
* BERT
White hat versus black hat techniques
SEO techniques can be classified into two broad categories: techniques that search engine companies recommend as part of good design ("white hat"), and those techniques of which search engines do not approve ("black hat"). Search engines attempt to minimize the effect of the latter, among them
spamdexing
Spamdexing (also known as search engine spam, search engine poisoning, black-hat search engine optimization, search spam or web spam) is the deliberate manipulation of search engine indexes. It involves a number of methods, such as link building ...
. Industry commentators have classified these methods and the practitioners who employ them as either
white hat
White hat, white hats, or white-hat may refer to:
Art, entertainment, and media
* White hat, a way of thinking in Edward de Bono's book ''Six Thinking Hats''
* White hat, part of black and white hat symbolism in film
Other uses
* White hat (compu ...
SEO or
black hat
Black hat, blackhats, or black-hat refers to:
Arts, entertainment, and media
* Black hat (computer security), a hacker who violates computer security for little reason beyond maliciousness or for personal gain
* Black hat, part of black and whit ...
SEO. White hats tend to produce results that last a long time, whereas black hats anticipate that their sites may eventually be banned either temporarily or permanently once the search engines discover what they are doing.
An SEO technique is considered a white hat if it conforms to the search engines' guidelines and involves no deception. As the search engine guidelines
are not written as a series of rules or commandments, this is an important distinction to note. White hat SEO is not just about following guidelines but is about ensuring that the content a search engine indexes and subsequently ranks is the same content a user will see. White hat advice is generally summed up as creating content for users, not for search engines, and then making that content easily accessible to the online "spider" algorithms, rather than attempting to trick the algorithm from its intended purpose. White hat SEO is in many ways similar to web development that promotes accessibility, although the two are not identical.
Black hat SEO
Spamdexing (also known as search engine spam, search engine poisoning, black-hat search engine optimization, search spam or web spam) is the deliberate manipulation of search engine indexes. It involves a number of methods, such as link building ...
attempts to improve rankings in ways that are disapproved of by the search engines or involve deception. One black hat technique uses hidden text, either as text colored similar to the background, in an invisible
div
Div or DIV may refer to:
Science and technology
* Division (mathematics), the mathematical operation that is the inverse of multiplication
* Span and div, HTML tags that implement generic elements
* div, a C mathematical function
* Divergence, ...
, or positioned off-screen. Another method gives a different page depending on whether the page is being requested by a human visitor or a search engine, a technique known as
cloaking
Cloaking is a search engine optimization (SEO) technique in which the content presented to the search engine spider is different from that presented to the user's browser. This is done by delivering content based on the IP addresses or the User ...
. Another category sometimes used is
grey hat SEO. This is in between the black hat and white hat approaches, where the methods employed avoid the site being penalized but do not act in producing the best content for users. Grey hat SEO is entirely focused on improving search engine rankings.
Search engines may penalize sites they discover using black or grey hat methods, either by reducing their rankings or eliminating their listings from their databases altogether. Such penalties can be applied either automatically by the search engines' algorithms or by a manual site review. One example was the February 2006 Google removal of both
BMW Germany and
Ricoh
is a Japanese multinational imaging and electronics company (law), company. It was founded by the now-defunct commercial division of the Riken, Institute of Physical and Chemical Research (Riken) known as the ''Riken concern (business), Concer ...
Germany for the use of deceptive practices.
Both companies, however, quickly apologized, fixed the offending pages, and were restored to Google's search engine results page.
As marketing strategy
SEO is not an appropriate strategy for every website, and other Internet marketing strategies can be more effective, such as paid advertising through pay-per-click
(PPC) campaigns, depending on the site operator's goals.
Search engine marketing (SEM) is the practice of designing, running, and optimizing search engine ad campaigns. Its difference from SEO is most simply depicted as the difference between paid and unpaid priority ranking in search results. SEM focuses on prominence more so than relevance; website developers should regard SEM with the utmost importance with consideration to visibility as most navigate to the primary listings of their search. A successful Internet marketing campaign may also depend upon building high-quality web pages to engage and persuade internet users, setting up
analytics
Analytics is the systematic computational analysis of data or statistics. It is used for the discovery, interpretation, and communication of meaningful patterns in data. It also entails applying data patterns toward effective decision-making. It ...
programs to enable site owners to measure results, and improving a site's
conversion rate
In electronic commerce, conversion marketing is marketing with the intention of increasing ''conversions—''that is, site visitors who are paying customers.
Measures
Conversion marketing attempts to solve low online conversions through optim ...
. In November 2015, Google released a full 160-page version of its Search Quality Rating Guidelines to the public, which revealed a shift in their focus towards "usefulness" and
mobile local search
Mobile local search is a technology that lets people search for local things using mobile equipment such as mobile phones, PDAs, and other mobile devices. Mobile local search satisfies the need to offer a mobile subscriber spontaneous access to n ...
. In recent years the mobile market has exploded, overtaking the use of desktops, as shown in by
StatCounter
StatCounter is a web traffic analysis website started in 1999. Access to basic services is free to use and advanced services can cost between and US$119 a month. StatCounter is based in Dublin, Ireland. The statistics from StatCounter are used ...
in October 2016, where they analyzed 2.5 million websites and found that 51.3% of the pages were loaded by a mobile device. Google has been one of the companies that are utilizing the popularity of mobile usage by encouraging websites to use their
Google Search Console
Google Search Console is a web service by Google which allows webmasters to check indexing status, search queries, crawling errors and optimize visibility of their websites.
Until 20 May 2015, the service was called Google Webmaster Tools. In ...
, the Mobile-Friendly Test, which allows companies to measure up their website to the search engine results and determine how user-friendly their websites are. The closer the keywords are together their ranking will improve based on key terms.
SEO may generate an adequate
return on investment
Return on investment (ROI) or return on costs (ROC) is a ratio between net income (over a period) and investment (costs resulting from an investment of some resources at a point in time). A high ROI means the investment's gains compare favourably ...
. However, search engines are not paid for organic search traffic, their algorithms change, and there are no guarantees of continued referrals. Due to this lack of guarantee and uncertainty, a business that relies heavily on search engine traffic can suffer major losses if the search engines stop sending visitors. Search engines can change their algorithms, impacting a website's search engine ranking, possibly resulting in a serious loss of traffic. According to Google's CEO, Eric Schmidt, in 2010, Google made over 500 algorithm changes – almost 1.5 per day. It is considered a wise business practice for website operators to liberate themselves from dependence on search engine traffic. In addition to accessibility in terms of web crawlers (addressed above), user
web accessibility
Web accessibility, or eAccessibility,European CommissionCommunication from the Commission to the Council, the European Parliament and the , European Economic and Social Committee and the Committee of the Regions: eAccessibility, EC(2005)1095 ...
has become increasingly important for SEO.
International markets
Optimization techniques are highly tuned to the dominant search engines in the target market.
The search engines' market shares vary from market to market, as does competition.
In 2003,
Danny Sullivan
Daniel John Sullivan III (born March 9, 1950), better known as Danny Sullivan, is an American former racing driver. He earned 17 wins in the CART Indy Car World Series, including the 1985 Indianapolis 500. Sullivan won the 1988 CART Champions ...
stated that Google represented about 75% of all searches. In markets outside the United States, Google's share is often larger, and Google remains the dominant search engine worldwide as of 2007. As of 2006, Google had an 85–90% market share in Germany.
While there were hundreds of SEO firms in the US at that time, there were only about five in Germany.
As of June 2008, the market share of Google in the UK was close to 90% according to
Hitwise
Hitwise is a division of Connexity, that measures behavior across desktop, tablet and smartphone devices. Hitwise provides marketers with online insights to help them understand, track and grow market share. Using Hitwise, marketers can define ...
. That market share is achieved in a number of countries.
As of 2009, there are only a few large markets where Google is not the leading search engine. In most cases, when Google is not leading in a given market, it is lagging behind a local player. The most notable example markets are China, Japan, South Korea, Russia, and the Czech Republic, where respectively
Baidu
Baidu, Inc. ( ; , meaning "hundred times") is a Chinese multinational technology company specializing in Internet-related services and products and artificial intelligence (AI), headquartered in Beijing's Haidian District. It is one of the la ...
,
Yahoo! Japan
is a Japanese internet company originally formed as a joint venture between the American internet company Yahoo! (later divested by Verizon into Altaba) and the Japanese company SoftBank. It is headquartered at Kioi Tower in the Tokyo Garden Terr ...
,
Naver
Naver (Hangul: 네이버) is a South Korean online platform operated by the Naver Corporation. It was launched in 1999 as the first web portal in South Korea to develop and use its own search engine. It was also the world's first operator to in ...
,
Yandex
Yandex LLC (russian: link=no, Яндекс, p=ˈjandəks) is a Russian multinational technology company providing Internet-related products and services, including an Internet search engine, information services, e-commerce, transportation, maps ...
and
Seznam are market leaders.
Successful search optimization for international markets may require professional
translation
Translation is the communication of the Meaning (linguistic), meaning of a #Source and target languages, source-language text by means of an Dynamic and formal equivalence, equivalent #Source and target languages, target-language text. The ...
of web pages, registration of a domain name with a
top level domain
A top-level domain (TLD) is one of the domains at the highest level in the hierarchical Domain Name System of the Internet after the root domain. The top-level domain names are installed in the root zone of the name space. For all domains in ...
in the target market, and
web hosting
A web hosting service is a type of Internet hosting service that hosts websites for clients, i.e. it offers the facilities required for them to create and maintain a site and makes it accessible on the World Wide Web. Companies providing web h ...
that provides a local
IP address
An Internet Protocol address (IP address) is a numerical label such as that is connected to a computer network that uses the Internet Protocol for communication.. Updated by . An IP address serves two main functions: network interface ident ...
. Otherwise, the fundamental elements of search optimization are essentially the same, regardless of language.
Legal precedents
On October 17, 2002, SearchKing filed suit in the
United States District Court
The United States district courts are the trial courts of the United States federal judiciary, U.S. federal judiciary. There is one district court for each United States federal judicial district, federal judicial district, which each cover o ...
, Western District of Oklahoma, against the search engine Google. SearchKing's claim was that Google's tactics to prevent spamdexing constituted a
tortious interference
Tortious interference, also known as intentional interference with contractual relations, in the common law of torts, occurs when one person intentionally damages someone else's contractual or business relationships with a third party, causing e ...
with contractual relations. On May 27, 2003, the court granted Google's motion to dismiss the complaint because SearchKing "failed to state a claim upon which relief may be granted."
In March 2006, KinderStart filed a lawsuit against Google over search engine rankings. KinderStart's website was removed from Google's index prior to the lawsuit, and the amount of traffic to the site dropped by 70%. On March 16, 2007, the
(
San Jose Division) dismissed KinderStart's complaint without leave to amend and partially granted Google's motion for
Rule 11 sanctions against KinderStart's attorney, requiring him to pay part of Google's legal expenses.
See also
*
Blog network
On the World Wide Web, a link farm is any group of websites that all hyperlink to other sites in the group for the purpose of increasing SEO rankings. In graph theoretic terms, a link farm is a clique. Although some link farms can be created ...
*
Competitor backlinking Competitor backlinking is a search engine optimization strategy that involves analyzing the backlinks of competing websites within a vertical search. The outcome of this activity is designed to increase organic search engine rankings and to gain an ...
*
List of search engines
Search engines, including web search engines, selection-based search engines, metasearch engines, desktop search tools, and web portals and vertical market websites have a search facility for online databases.
By content/topic
General ...
*
Search engine marketing
Search engine marketing (SEM) is a form of Internet marketing that involves the promotion of websites by increasing their visibility in search engine results pages (SERPs) primarily through paid advertising. SEM may incorporate search engine opti ...
*
Search neutrality
Search neutrality is a principle that search engines should have no editorial policies other than that their results be comprehensive, impartial and based solely on relevance. This means that when a user types in a search engine query, the engine ...
, the opposite of search manipulation
*
User intent
User intent, otherwise known as query intent or search intent, is the identification and categorization of what a user online intended or wanted to find when they typed their search terms into an online web search engine for the purpose of search ...
*
Website promotion
Website promotion is the continuing process used by webmasters to improve content and increase exposure of a website to bring more visitors. Many techniques such as search engine optimization and search engine submission are used to increase a si ...
References
External links
Webmaster Guidelinesfrom Google
Google Search Quality Evaluators Guidelines (PDF)from Yahoo!
Webmaster Guidelinesfrom
Microsoft Bing
Microsoft Bing (commonly known as Bing) is a web search engine owned and operated by Microsoft. The service has its origins in Microsoft's previous search engines: MSN Search, Windows Live Search and later Live Search. Bing provides a variety ...
The Dirty Little Secrets of Searchin
The New York Times
''The New York Times'' (''the Times'', ''NYT'', or the Gray Lady) is a daily newspaper based in New York City with a worldwide readership reported in 2020 to comprise a declining 840,000 paid print subscribers, and a growing 6 million paid ...
(February 12, 2011)
{{Authority control
Digital marketing
Web analytics
Internet terminology
Online advertising
Promotion and marketing communications