Apache Mesos is an

open-source Open source is source code that is made freely available for possible modification and redistribution. Products include permission to use the source code, design documents, or content of the product. The open-source model is a decentralized sof ...

project to manage

computer cluster A computer cluster is a set of computers that work together so that they can be viewed as a single system. Unlike grid computers, computer clusters have each node set to perform the same task, controlled and scheduled by software. The comp ...

s. It was developed at the

University of California, Berkeley The University of California, Berkeley (UC Berkeley, Berkeley, Cal, or California) is a public land-grant research university in Berkeley, California. Established in 1868 as the University of California, it is the state's first land-grant u ...

History

Mesos began as a research project in the UC Berkeley RAD Lab by then PhD students Benjamin Hindman, Andy Konwinski, and

Matei Zaharia Matei Zaharia is a Romanian-Canadian computer scientist, educator and the creator of Apache Spark. As of April 2022, Forbes ranked him and Ion Stoica as the 3rd- richest people in Romania with a net worth of $1.6 billion. Biography Zaharia gr ...

, as well as professor

Ion Stoica Ion Stoica is a Romanian-American computer scientist specializing in distributed systems, cloud computing and computer networking. He is a professor of computer science at the University of California, Berkeley and co-director of AMPLab. He co-fo ...

. The students started working on the project as part of a course taught by

David Culler David Ethan Culler (born November 12, 1959) is a computer scientist and former chair of the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He is a principal investigator in the Software Defi ...

. It was originally named ''Nexus'' but due to a conflict with another university's project, was renamed to Mesos. Mesos was first presented in 2009 (while still named Nexus) by Andy Konwinski at HotCloud '09 in a talk accompanying the first paper published about the project. Later in 2011 it was presented in a more mature state in a talk by Zaharia at the Usenix Symposium on Networked Systems Design and Implementation conference about the paper "Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center" by Benjamin Hindman, Andy Konwinski, Zaharia,

Ali Ghodsi Ali Ghodsi is an Iranian-Swedish computer scientist and entrepreneur specializing in distributed systems and big data. He is a co-founder and CEO of Databricks and an adjunct professor at UC Berkeley. Ideas from his academic research in the area of ...

, Anthony D. Joseph,

Randy Katz Randy Howard Katz is a distinguished professor at University of California, Berkeley of the electrical engineering and computer science department. Biography Katz was born in Brooklyn, New York in 1955. He was first exposed to computers in Can ...

Scott Shenker Scott J. Shenker (born January 24, 1956 in Alexandria, Virginia) is an American computer scientist, and professor of computer science at the University of California, Berkeley. He is also the leader of the Extensible Internet Group at the Interna ...

. On July 27, 2016, the

Apache Software Foundation The Apache Software Foundation (ASF) is an American nonprofit corporation (classified as a 501(c)(3) organization in the United States) to support a number of open source software projects. The ASF was formed from a group of developers of the A ...

announced version 1. It added the ability to centrally supply Docker, rkt and appc instances. On April 5, 2021, it was voted to move Mesos to the

Apache Attic Apache Attic is a project of Apache Software Foundation to provide processes to make it clear when an Apache project has reached its end-of-life. The Attic project was created in November 2008. Also the retired projects can be retained. Projects m ...

, however the vote was cancelled two days later due to increased interest.

Technology

Mesos uses Linux

cgroups cgroups (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. Engineers at Google started the work on this ...

to provide isolation for CPU,

memory Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered, ...

, I/O and

file system In computing, file system or filesystem (often abbreviated to fs) is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one larg ...

. Mesos is comparable to

Google Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...

's Borg scheduler, a platform used internally to manage and distribute Google's services.

Apache Aurora

Apache Aurora is a Mesos framework for both long-running services and cron jobs, originally developed by Twitter starting in 2010 and open sourced in late 2013. It can scale to tens of thousands of servers, and holds many similarities to Borg including its rich

domain-specific language A domain-specific language (DSL) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains. There are a wide variety of DSLs, ranging f ...

(DSL) for configuring services. As of February 2020 the project was retired to the Attic. A fork of the project is maintained by former members, hosted on GitHub under the name Aurora Scheduler.

Chronos

Chronos is a distributed cron-like system which is elastic and capable of expressing dependencies between jobs.

Marathon

Marathon is promoted for

platform as a service Platform as a service (PaaS) or application platform as a service (aPaaS) or platform-based service is a category of cloud computing services that allows customers to provision, instantiate, run, and manage a modular bundle comprising a computin ...

container A container is any receptacle or enclosure for holding a product used in storage, packaging, and transportation, including shipping. Things kept inside of a container are protected on several sides by being inside of its structure. The term ...

orchestration Orchestration is the study or practice of writing music for an orchestra (or, more loosely, for any musical ensemble, such as a concert band) or of adapting music composed for another medium for an orchestra. Also called "instrumentation", orc ...

system scaling to thousands of physical servers. It is fully

REST Rest or REST may refer to: Relief from activity * Sleep ** Bed rest * Kneeling * Lying (position) * Sitting * Squatting position Structural support * Structural support ** Rest (cue sports) ** Armrest ** Headrest ** Footrest Arts and entert ...

-based and allows canary-style deployments and deployment topologies. It is written in the programming language Scala.

Users

The social networking site Twitter began using Mesos and Apache Aurora in 2010, after Hindman gave a presentation to a group of Twitter engineers. Airbnb said in July 2013 that it uses Mesos to run data processing systems like

Apache Hadoop Apache Hadoop () is a collection of open-source software utilities that facilitates using a network of many computers to solve problems involving massive amounts of data and computation. It provides a software framework for distributed storage ...

and

Apache Spark Apache Spark is an open-source unified analytics engine for large-scale data processing. Spark provides an interface for programming clusters with implicit data parallelism and fault tolerance. Originally developed at the University of Californi ...

. The Internet auction website

eBay eBay Inc. ( ) is an American multinational e-commerce company based in San Jose, California, that facilitates consumer-to-consumer and business-to-consumer sales through its website. eBay was founded by Pierre Omidyar in 1995 and became a ...

stated in April 2014 that it used Mesos to run continuous integration on a per-developer basis. They accomplish this by using a custom Mesos plugin that allows developers to launch their own private Jenkins instance. In April 2015, it was announced that

Apple An apple is an edible fruit produced by an apple tree (''Malus domestica''). Apple fruit tree, trees are agriculture, cultivated worldwide and are the most widely grown species in the genus ''Malus''. The tree originated in Central Asia, wh ...

service

Siri Siri ( ) is a virtual assistant that is part of Apple Inc.'s iOS, iPadOS, watchOS, macOS, tvOS, and audioOS operating systems. It uses voice queries, gesture based control, focus-tracking and a natural-language user interface to answer questio ...

is using its own Mesos framework called Jarvis. In August 2015, it was announced that

Verizon Verizon Communications Inc., commonly known as Verizon, is an American multinational telecommunications conglomerate and a corporate component of the Dow Jones Industrial Average. The company is headquartered at 1095 Avenue of the Americas in ...

selected Mesosphere's DC/OS, which is based on open source Apache Mesos, for data center service orchestration. In November 2015,

Yelp Yelp Inc. is an American company that develops the Yelp.com website and the Yelp mobile app, which publish crowd-sourced reviews about businesses. It also operates Yelp Guest Manager, a table reservation service. It is headquartered in San Fra ...

announced they had been using Mesos and Marathon for a year and a half for production services.

Commercial support

Software startup Mesosphere, Inc. sells the Datacenter Operating System, a

distributed operating system A distributed operating system is system software over a collection of independent software, networked, communicating, and physically separate computational nodes. They handle jobs which are serviced by multiple CPUs. Each individual node holds a ...

, based on Apache Mesos. In September 2015,

Microsoft Microsoft Corporation is an American multinational technology corporation producing computer software, consumer electronics, personal computers, and related services headquartered at the Microsoft Redmond campus located in Redmond, Washing ...

announced a commercial partnership with Mesosphere to build container scheduling and orchestration services for

Microsoft Azure Microsoft Azure, often referred to as Azure ( , ), is a cloud computing platform operated by Microsoft for application management via around the world-distributed data centers. Microsoft Azure has multiple capabilities such as software as a ...

. In October 2015,

Oracle An oracle is a person or agency considered to provide wise and insightful counsel or prophetic predictions, most notably including precognition of the future, inspired by deities. As such, it is a form of divination. Description The word '' ...

announced support for Mesos through Oracle Container Cloud Service.

References

External links

* {{Linux containers Mesos Mesos Cloud infrastructure Free software for cloud computing Linux containerization Software using the Apache license