Website monitoring is the process of testing and verifying that end-users can interact with a

website A website (also written as a web site) is a collection of web pages and related content that is identified by a common domain name and published on at least one web server. Examples of notable websites are Google Search, Google, Facebook, Amaz ...

web application A web application (or web app) is application software that is accessed using a web browser. Web applications are delivered on the World Wide Web to users with an active network connection. History In earlier computing models like client-serve ...

as expected. Website monitoring are often used by businesses to ensure website

uptime Uptime is a measure of system reliability, expressed as the percentage of time a machine, typically a computer, has been working and available. Uptime is the opposite of downtime. It is often used as a measure of computer operating system reliab ...

, performance, and functionality is as expected. Website monitoring companies provide organizations the ability to consistently monitor a website or server function and observe how it responds. The monitoring is often conducted from several locations around the world to a specific website, or server, to detect issues related to general Internet latency, and network hop issues, and to prevent false positives caused by local or inter-connect problems. Monitoring companies generally report on these tests in a variety of reports, charts, and graphs. When an error is detected monitoring services send out alerts via email, SMS, phone,

SNMP Simple Network Management Protocol (SNMP) is an Internet Standard protocol for collecting and organizing information about managed devices on IP networks and for modifying that information to change device behaviour. Devices that typically ...

trap, a pager that may include diagnostic information, such as a network

traceroute In computing, traceroute and tracert are computer network diagnostic commands for displaying possible routes (paths) and measuring transit delays of packets across an Internet Protocol (IP) network. The history of the route is recorded as th ...

, code capture of a web page's HTML file, a screenshot of a webpage, and even a video of a website failing. These diagnostics allow

network administrator A network administrator is a person designated in an organization whose responsibility includes maintaining computer infrastructures with emphasis on local area networks (LANs) up to wide area networks (WANs). Responsibilities may vary between org ...

s and

webmaster A webmaster is a person responsible for maintaining one or more websites. The title may refer to web architects, web developers, site authors, website administrators, website owners, website coordinators, or website publishers. The duties of ...

s to correct issues faster. Monitoring gathers extensive data on website performance, such as load times, server response times, and page element performance that is often analyzed and used to further optimize website performance.

Purpose

Monitoring is essential to ensure that a website is available to users,

downtime The term downtime is used to refer to periods when a system is unavailable. The unavailability is the proportion of a time-span that a system is unavailable or offline. This is usually a result of the system failing to function because of an u ...

is minimized, and performance can be optimized. Users that rely on a website or an application for work or pleasure will get frustrated or even stop using the application if it is not reliably available. Monitoring can cover many things that an application needs to function, like network connectivity,

Domain Name System The Domain Name System (DNS) is a hierarchical and distributed naming system for computers, services, and other resources in the Internet or other Internet Protocol (IP) networks. It associates various information with domain names assigned to ...

records, database connectivity, bandwidth, and computer resources like free

RAM Ram, ram, or RAM may refer to: Animals * A male sheep * Ram cichlid, a freshwater tropical fish People * Ram (given name) * Ram (surname) * Ram (director) (Ramsubramaniam), an Indian Tamil film director * RAM (musician) (born 1974), Dutch * ...

, CPU load, disk space, events, etc. Commonly measured metrics are

response time Response time may refer to: *The time lag between an electronic input and the output signal which depends upon the value of passive components used. * Responsiveness, how quickly an interactive system responds to user input * Response time (biolog ...

and

availability In reliability engineering, the term availability has the following meanings: * The degree to which a system, subsystem or equipment is in a specified operable and committable state at the start of a mission, when the mission is called for at ...

(or

), but consistency and reliability metrics are gaining popularity. Measuring a website's availability and reliability under various amounts of traffic is often referred to as

load testing Load testing is the process of putting demand on a structure or system and measuring its response. Software load testing The term ''load testing'' is used in different ways in the professional software testing community. ''Load testing'' gen ...

. Website monitoring also helps benchmark the website against the performance of a competitor to help determine how well a site is performing. Website speed is also used as a metric for

search engine A search engine is a software system designed to carry out web searches. They search the World Wide Web in a systematic way for particular information specified in a textual web search query. The search results are generally presented in a ...

rankings. Website monitoring can be used to hold web hosting providers accountable for their service-level agreements. Most web hosts offer a 99.9% uptime guarantee and when uptime is less than that, individuals can be refunded for the excessive downtime. Note that not all hosts will refund individuals for excessive downtime so one must become familiar with the terms of service of their host. Most paid website monitoring services will also offer security features such as

virus A virus is a submicroscopic infectious agent that replicates only inside the living cells of an organism. Viruses infect all life forms, from animals and plants to microorganisms, including bacteria and archaea. Since Dmitri Ivanovsky's 1 ...

and malware scanning which is of growing importance as websites become more complicated and integral to the business.

Internal vs. external

Website monitoring can be done from both ''inside'' and ''outside'' of a corporate

firewall Firewall may refer to: * Firewall (computing), a technological barrier designed to prevent unauthorized or unwanted communications between computer networks or hosts * Firewall (construction), a barrier inside a building, designed to limit the spr ...

. Traditional

network management Network management is the process of administering and managing computer networks. Services provided by this discipline include fault analysis, performance management, provisioning of networks and maintaining quality of service. Network managem ...

solutions focus on ''inside the firewall monitoring'', whereas ''external performance monitoring'' will test and monitor performance issues across the

Internet backbone The Internet backbone may be defined by the principal data routes between large, strategically interconnected computer networks and core routers of the Internet. These data routes are hosted by commercial, government, academic and other high-ca ...

and in some cases all the way to the

end-user In product development, an end user (sometimes end-user) is a person who ultimately uses or is intended to ultimately use a product. The end user stands in contrast to users who support or maintain the product, such as sysops, system administrat ...

. Third-party website performance monitoring solutions can monitor internal (behind the firewall), external (customer-facing), or cloud-based Web applications. Inside the firewall, monitoring is done by special hardware appliances which can help you determine if your internal applications’ sluggish performance is caused by: the design of applications, internal infrastructure, internal applications or connections to any public Internet. External performance monitoring is also known as end-user monitoring or end-to-end performance monitoring.

Real user monitoring Real user monitoring (RUM) is a passive monitoring technology that records all user interaction with a website or client interacting with a server or cloud-based application. Monitoring actual user interaction with a website or an application is i ...

measures the performance and availability experienced by actual users, diagnoses individual incidents, and tracks the impact of a change.

Measures of website availability

Types of protocol

A website monitoring service can check other internet protocols besides

HTTP The Hypertext Transfer Protocol (HTTP) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, ...

pages and

HTTPS Hypertext Transfer Protocol Secure (HTTPS) is an extension of the Hypertext Transfer Protocol (HTTP). It is used for secure communication over a computer network, and is widely used on the Internet. In HTTPS, the communication protocol is enc ...

such as

FTP The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and data ...

SMTP The Simple Mail Transfer Protocol (SMTP) is an Internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typical ...

POP3 In computing, the Post Office Protocol (POP) is an application-layer Internet standard protocol used by e-mail clients to retrieve e-mail from a mail server. POP version 3 (POP3) is the version in common use, and along with IMAP the most common p ...

, ActiveSync,

IMAP In computing, the Internet Message Access Protocol (IMAP) is an Internet standard protocol used by email clients to retrieve email messages from a mail server over a TCP/IP connection. IMAP is defined by . IMAP was designed with the goal of per ...

DNS The Domain Name System (DNS) is a hierarchical and distributed naming system for computers, services, and other resources in the Internet or other Internet Protocol (IP) networks. It associates various information with domain names assigned to ...

SSH The Secure Shell Protocol (SSH) is a cryptographic network protocol for operating network services securely over an unsecured network. Its most notable applications are remote login and command-line execution. SSH applications are based on ...

Telnet Telnet is an application protocol used on the Internet or local area network to provide a bidirectional interactive text-oriented communication facility using a virtual terminal connection. User data is interspersed in-band with Telnet control i ...

, SSL, TCP,

PING Ping may refer to: Arts and entertainment Fictional characters * Ping, a domesticated Chinese duck in the illustrated book '' The Story about Ping'', first published in 1933 * Ping, a minor character in ''Seinfeld'', an NBC sitcom * Ping, a c ...

, UDP, SOAP, Domain Name Expiry, SSL Certificate Expiry and a range of ports. Monitoring frequency occurs at intervals of once every 4-hours to every 15-seconds. Typically, most website monitoring services test a server, or application, between a once-per-hour per once-per-minute. Advanced monitoring services capture browser interactions with websites using macro recorders, or browser add-ons such as

Selenium Selenium is a chemical element with the symbol Se and atomic number 34. It is a nonmetal (more rarely considered a metalloid) with properties that are intermediate between the elements above and below in the periodic table, sulfur and tellurium, ...

iMacros iMacros is a browser based application for macro recording, editing and playback for web automation and testing. It is provided as a standalone application and extensions for the Mozilla Firefox, Google Chrome, and Internet Explorer web browsers. ...

. These services test a website by running a web browser through a typical website transaction (such as a shopping cart) or a custom scenario, in order to check for user experience issues, performance problems, and availability errors. Browser-driven monitoring services detect not only network and server issues, but also webpage object issues (such as slow loading

JavaScript JavaScript (), often abbreviated as JS, is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS. As of 2022, 98% of Website, websites use JavaScript on the Client (computing), client side ...

, or third-party hosted page elements). Implementation of time performance monitoring for the

Apache HTTP Server The Apache HTTP Server ( ) is a free and open-source cross-platform web server software, released under the terms of Apache License 2.0. Apache is developed and maintained by an open community of developers under the auspices of the Apache So ...

is the mod_arm4 module.

Types of monitoring

Users of website monitoring (typically network administrators, webmasters, web operations personnel) may monitor a single page of a website, but can also monitor a complete

business process A business process, business method or business function is a collection of related, structured activities or tasks by people or equipment in which a specific sequence produces a service or product (serves a particular business goal) for a parti ...

(often referred to as multi-step transactions).

Servers monitoring from around the globe

Website monitoring services usually have a number of servers around the globe –

South America South America is a continent entirely in the Western Hemisphere and mostly in the Southern Hemisphere, with a relatively small portion in the Northern Hemisphere at the northern tip of the continent. It can also be described as the southe ...

Africa Africa is the world's second-largest and second-most populous continent, after Asia in both cases. At about 30.3 million km2 (11.7 million square miles) including adjacent islands, it covers 6% of Earth's total surface area ...

North America North America is a continent in the Northern Hemisphere and almost entirely within the Western Hemisphere. It is bordered to the north by the Arctic Ocean, to the east by the Atlantic Ocean, to the southeast by South America and the Car ...

, Europe, Africa, Asia, Australia and other locations. By having multiple servers in different geographic locations, a monitoring service can determine if a

web server A web server is computer software and underlying hardware that accepts requests via HTTP (the network protocol created to distribute web content) or its secure variant HTTPS. A user agent, commonly a web browser or web crawler, initiate ...

is available across continents over the Internet. Some vendors claim that the more locations the better picture of your website availability while others say that three globally distributed stations are sufficient and more stations do not give more information.

Types

There are two main types of website monitoring *

Synthetic monitoring In software design, web design, and electronic product design, synthetic monitoring (also known as ''active monitoring or proactive monitoring'') is a monitoring technique that is done by using a simulation or scripted recordings of transactions ...

is also known as active monitoring, and * Passive monitoring is also known as real monitoring.

Levels of Website Monitoring

There are different levels of website monitoring, the more complex your website, the more complex your monitoring needs: *Level 1 Uptime Monitoring – Availability of a Critical Page *Level 2 Transaction Monitoring – Availability of a Critical Process *Level 3 Performance Monitoring – Performance of a Critical Page *Level 4 Synthetic Monitoring – Performance of a Critical Process *Level 5 Customer Journey Monitoring – Level 1 to 4 plus Security Information

Notification options: alerts

As the information brought by website monitoring services is in most cases urgent and may be of crucial importance, various notification methods, often known as "alerts" are used:

e-mail Electronic mail (email or e-mail) is a method of exchanging messages ("mail") between people using electronic devices. Email was thus conceived as the electronic ( digital) version of, or counterpart to, mail, at a time when "mail" meant ...

, IM, regular and cell phones, SMS, fax, pagers,

Skype Skype () is a proprietary telecommunications application operated by Skype Technologies, a division of Microsoft, best known for VoIP-based videotelephony, videoconferencing and voice calls. It also has instant messaging, file transfer, deb ...

RSS Feed RSS ( RDF Site Summary or Really Simple Syndication) is a web feed that allows users and applications to access updates to websites in a standardized, computer-readable format. Subscribing to RSS feeds can allow a user to keep track of many d ...

SNMP trap Simple Network Management Protocol (SNMP) is an Internet Standard protocol for collecting and organizing information about managed devices on IP networks and for modifying that information to change device behaviour. Devices that typically ...

, URL notifications, etc.

Website monitoring services

The Website monitoring market is very competitive. There are 150+ active service providers and more than 100 are documented to have gone out of business. Most of the providers offer a free plan with low-frequency monitoring.

References

# Amazon Downtime Cost $66,000
Forbes
# Speed Affects Website Usage

Website management Web server management software Website monitoring software