Prometheus (software)

Prometheus is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting. The project is written in Go and licensed under the Apache 2 License, with source code available on GitHub, and is a graduated project of the Cloud Native Computing Foundation, along with Kubernetes and Envoy.

History

Prometheus was developed at SoundCloud starting in 2012, when the company discovered that its existing metrics and monitoring solutions (using StatsD and Graphite) were not sufficient for their needs. Specifically, they identified needs that Prometheus was built to meet including: a multi-dimensional data model, operational simplicity, scalable data collection, and a powerful query language, all in a single tool. The project was open-source from the beginning and began to be used by Boxever and Docker users as well, despite not being explicitly announced. Prometheus was inspired by the monitoring tool Borgmon used at Google.

By 2013, Prometheus was introduced for production monitoring at SoundCloud. The official public announcement was made in January 2015.

In May 2016, the Cloud Native Computing Foundation accepted Prometheus as its second incubated project, after Kubernetes. The blog post announcing this stated that the tool was in use at many companies including DigitalOcean, Ericsson, CoreOS, Weaveworks, Red Hat, and Google.

Prometheus 1.0 was released in July 2016. Subsequent versions were released through 2016 and 2017, leading to Prometheus 2.0 in November 2017.

In August 2018, the Cloud Native Computing Foundation announced that the Prometheus project had graduated.

Architecture

A typical monitoring platform with Prometheus is composed of multiple tools:
* Multiple ''exporters'' typically run on the monitored host to export local metrics.
* Prometheus to centralize and store the metrics.
* ''Alertmanager'' to trigger alerts based on those metrics.
* ''Grafana'' to produce dashboards.
* ''PromQL'' is the query language used to create dashboards and alerts.

Data storage format

Prometheus data is stored in the form of metrics, with each metric having a name that is used for referencing and querying it. Each metric can be drilled down by an arbitrary number of key=value pairs (labels). Labels can include information on the data source (which server the data is coming from) and other application-specific breakdown information such as the HTTP status code (for metrics related to HTTP responses), query method (GET versus POST), endpoint, etc. The ability to specify an arbitrary list of labels and to query based on these in real time is why Prometheus' data model is called multi-dimensional.

Prometheus stores data locally on disk, which helps for fast data storage and fast querying. There is the ability to store metrics in remote storage.

Data collection

Prometheus collects data in the form of time series. The time series are built through a pull model: the Prometheus server queries a list of data sources (sometimes called exporters) at a specific polling frequency. Each of the data sources serves the current values of the metrics for that data source at the endpoint queried by Prometheus. The Prometheus server then aggregates data across the data sources. Prometheus has a number of mechanisms to automatically discover resources that should be used as data sources.

PromQL

Prometheus provides its own query language PromQL (Prometheus Query Language) that lets users select and aggregate data. PromQL is specifically adjusted to work in convention with a Time-Series Database and therefore provides time-related query functionalities. Examples include the rate() function, the instant vector and the range vector which can provide many samples for each queried time series. Prometheus has four clearly defined metric types around which the PromQL components revolve. The four types are
* Gauge
* Counter
* Histogram
* Summary

Alerts and monitoring

Configuration for alerts can be specified in Prometheus which specifies a condition that needs to be maintained for a specific duration in order for an alert to trigger. When alerts trigger, they are forwarded to the Alertmanager service. Alertmanager can include logic to silence alerts and also to forward them to email, Slack, or notification services such as PagerDuty. Some other messaging systems like Microsoft Teams could be configured using the Alertmanager Webhook Receiver as a mechanism for external integrations. also Prometheus Alerts can be used to receive alerts directly on android devices even without the requirement of any targets configuration in Alert Manager.

Dashboards

Prometheus is not intended as a dashboarding solution. Although it can be used to graph specific queries, it is not a full-fledged dashboarding solution and needs to be hooked up with Grafana to generate dashboards; this has been cited as a disadvantage due to the additional setup complexity.

Interoperability

Prometheus favors white-box monitoring. Applications are encouraged to publish (export) internal metrics to be collected periodically by Prometheus. Some exporters and agents for various applications are available to provide metrics. Prometheus supports some monitoring and administration protocols to allow interoperability for transitioning: Graphite, StatsD, SNMP, JMX, and CollectD.

Prometheus focuses on the availability of the platform and basic operations. The metrics are typically stored for a few weeks. For long-term storage, the metrics can be streamed to remote storage solutions.

Standardization into OpenMetrics

There is an effort to promote Prometheus exposition format into a standard known as OpenMetrics. Some products adopted the format: InfluxData's TICK suite, InfluxDB, Google Cloud Platform, and DataDog.

Usage

Prometheus was first used in-house at SoundCloud, where it was developed, for monitoring their systems. The Cloud Native Computing Foundation has a number of case studies of other companies using Prometheus. These include digital hosting service DigitalOcean, digital festival DreamHack, and email and contact migration service ShuttleCloud. Separately, Pandora Radio has mentioned using Prometheus to monitor its data pipeline.

GitLab provides a Prometheus integration guide to export GitLab metrics to Prometheus and it is activated by default since version 9.0

Conferences

A variety of conferences and attached conferences which focused on Prometheus and its ecosystem have been held

* PromCon 2016, August 25 & 26, Berlin, 80 attendees (sold out)
* PrometheusDay 2016, Seattle
* PromCon 2017, August 17 & 18, Munich, 220 attendees (sold out)
* PromCon 2018, August 09 & 10, Munich, 220 attendees (sold out)
* PromCon 2019, November 07 & 08, Munich, 220 attendees (sold out)
* PromCon 2020, July 14 - 16, Online
* PromCon 2021, May 3, Online
* PromCon North America 2021, October 11, Los Angeles
* PrometheusDay Europe 2022, May 17, Valencia, ~400 attendees
* PrometheusDay North America 2022, October 25, Detroit
* PromCon 2022, November 8 & 9, Munich

See also

* Check MK
* Ganglia (software)
* Zabbix
* Comparison of network monitoring systems
* List of systems management systems

References

Further reading

*
*
*
*
*
* {{cite book, title=Native Docker Clustering with Swarm , first=Chanwit , last=Kaewkasi , year=2016 , isbn=978-1786469755

Software using the Apache license
System monitors
Time series software
Free software programmed in Go
Management systems
Systems management