Tokio is a
software library
In computer science, a library is a collection of non-volatile resources used by computer programs, often for software development. These may include configuration data, documentation, help data, message templates, pre-written code and subr ...
for the
Rust
Rust is an iron oxide, a usually reddish-brown oxide formed by the reaction of iron and oxygen in the catalytic presence of water or air moisture. Rust consists of hydrous iron(III) oxides (Fe2O3·nH2O) and iron(III) oxide-hydroxide (FeO(OH ...
programming language. It provides a
runtime and functions that enable the use of
asynchronous I/O
In computer science, asynchronous I/O (also non-sequential I/O) is a form of input/output processing that permits other processing to continue before the transmission has finished. A name used for asynchronous I/O in the Windows API is overlappe ...
, allowing for concurrency in regards to task completion.
Tokio was released in August 2016 for Rust, a
general-purpose programming language
In computer software, a general-purpose programming language (GPL) is a programming language for building software in a wide variety of application domains. Conversely, a domain-specific programming language is used within a specific area. For exam ...
. Developed by Carl Lerche, Tokio began as a network application framework and supports features such as socket listening and
broadcasting
Broadcasting is the distribution (business), distribution of sound, audio or video content to a dispersed audience via any electronic medium (communication), mass communications medium, but typically one using the electromagnetic spectrum (radio ...
, allowing messages to be transferred between computers.
History
Tokio began in August 2016 by Carl Lerche as a network application framework for Rust built on
futures
Futures may mean:
Finance
*Futures contract, a tradable financial derivatives contract
*Futures exchange, a financial market where futures contracts are traded
* ''Futures'' (magazine), an American finance magazine
Music
* ''Futures'' (album), a ...
, allowing for network-based
middleware
Middleware is a type of computer software that provides services to software applications beyond those available from the operating system. It can be described as "software glue".
Middleware makes it easier for software developers to implement co ...
and a
non-blocking, or asynchronous, implementation of readiness interest to the
reactor. Tokio was inspired by Finagle, a
Scala-based asynchronous
remote procedure call
In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (l ...
(RPC) system developed at Twitter for
Java virtual machine
A Java virtual machine (JVM) is a virtual machine that enables a computer to run Java programs as well as programs written in other languages that are also compiled to Java bytecode. The JVM is detailed by a specification that formally describes ...
s (JVM), allowing distributed systems to communicate within a JVM. Tokio utilizes the lower-level Rust crate
mio
, itself using system calls such as
epoll
epoll is a Linux kernel system call for a scalable I/O event notification mechanism, first introduced in version 2.5.44 of the Linux kernel. Its function is to monitor multiple file descriptors to see whether I/O is possible on any of them. It i ...
(
Linux
Linux ( or ) is a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds. Linux is typically packaged as a Linux distribution, which ...
),
kqueue
Kqueue is a scalable event notification interface introduced in FreeBSD 4.1 in July 2000, also supported in NetBSD, OpenBSD, DragonFly BSD, and macOS. Kqueue was originally authored in 2000 by Jonathan Lemon, then involved with the FreeBSD Cor ...
(
FreeBSD
FreeBSD is a free and open-source Unix-like operating system descended from the Berkeley Software Distribution (BSD), which was based on Research Unix. The first version of FreeBSD was released in 1993. In 2005, FreeBSD was the most popular ...
), and the
input/output completion port
Input/output completion port (IOCP) is an API for performing multiple simultaneous asynchronous input/output operations in Windows NT versions 3.5 and later, AIX and on Solaris 10 and later. An input/output completion port object is created and ...
(IOCP) API (
Windows
Windows is a group of several proprietary graphical operating system families developed and marketed by Microsoft. Each family caters to a certain sector of the computing industry. For example, Windows NT for consumers, Windows Server for serv ...
).
The name "Tokio" is derived from
Tokyo
Tokyo (; ja, 東京, , ), officially the Tokyo Metropolis ( ja, 東京都, label=none, ), is the capital and largest city of Japan. Formerly known as Edo, its metropolitan area () is the most populous in the world, with an estimated 37.468 ...
and mio. The preliminary version of Tokio was released in January 2017, followed by a full release in December 2020.
In 2017, Tokio received a grant from the
Mozilla Open Source Support fund. In April 2021, Tokio funded its first paid contributor, Alice Ryhl, for her work both developing the project and assisting its users.
While Rust has supported asynchronous functions since version 1.39, released in November 2019, it provides no facilities to execute them, requiring an external runtime for that purpose. Tokio provides a runtime that uses a multi-threaded
work stealing
In parallel computing
Parallel computing is a type of computation in which many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There ...
scheduler.
Rust's futures are
lazily evaluated, requiring functions to call
.await
before they do any work. When
.await
is invoked, Tokio's runtime may pause the original future until its I/O completes, and unpauses a different task that is ready for further processing.
Notable users of Tokio include the development teams behind
Discord
Discord is a VoIP and instant messaging social platform. Users have the ability to communicate with voice calls, video calls, text messaging, media and files in private chats or as part of communities called "servers".The developer documenta ...
and
AWS Lambda
Amazon Web Services, Inc. (AWS) is a subsidiary of Amazon.com, Amazon that provides Software as a service, on-demand cloud computing computing platform, platforms and Application programming interface, APIs to individuals, companies, and gover ...
.
The
JavaScript
JavaScript (), often abbreviated as JS, is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS. As of 2022, 98% of Website, websites use JavaScript on the Client (computing), client side ...
and
TypeScript
TypeScript is a free and open source programming language
A programming language is a system of notation for writing computer programs. Most programming languages are text-based formal languages, but they may also be graphical. They are ...
runtime
Deno uses Tokio under the hood, in comparison to the JavaScript runtime
Node.js
Node.js is an open-source server environment. Node.js is cross-platform and runs on Windows, Linux, Unix, and macOS. Node.js is a back-end JavaScript runtime environment. Node.js runs on the V8 JavaScript Engine and executes JavaScript code o ...
, which uses the
libuv
libuv is a multi-platform C library that provides support for asynchronous I/O based on event loops. It supports epoll(4), kqueue(2), Windows IOCP, and Solaris event ports. It is primarily designed for use in Node.js but it is also used by ...
library.
Features
Asynchronous code
Tokio allows for the usage of asynchronous functions in Rust through the creation of an asynchronous runtime. This can be accomplished through the
# okio::main/code> macro.
For example:
# okio::mainasync fn main() -> Result<()>
In this example, the reqwest
crate is used to request the HyperText Markup Language
The HyperText Markup Language or HTML is the standard markup language for documents designed to be displayed in a web browser. It can be assisted by technologies such as Cascading Style Sheets (CSS) and scripting languages such as JavaScript ...
(HTML) for English Wikipedia
The English Wikipedia is, along with the Simple English Wikipedia, one of two English-language editions of Wikipedia, an online encyclopedia. It was founded on January 15, 2001, as Wikipedia's first edition, and, as of
, has the most arti ...
. To ensure that the request is not immediately handled, Tokio wraps the function call into an asynchronous runtime, waiting for the request to complete before calling println()
.
Tokio also includes a version of the Rust standard library that is designed for being used asynchronously. For example, tokio::fs::read_to_end()
, which reads the contents of a file, is the asynchronous version of std::fs::read_to_end()
. In addition, Tokio supports io_uring, a Linux asynchronous I/O syscall interface, in a separate crate named tokio-uring
.
Compile-time green-threading
Tokio further allows users to create tasks, which are green threads
In computer programming, a green thread is a thread that is scheduled by a runtime library or virtual machine (VM) instead of natively by the underlying operating system (OS). Green threads emulate multithreaded environments without relying on an ...
, using a tokio::spawn()
function. Green threads run at the user level, providing parallelism when native threads are not always available. Previous versions of Rust implemented green threading; this functionality was removed in Rust 1.0. Unlike futures, tasks do not need to use .await
, as the task will be automatically executed when a thread is available.
Socket listening
Tokio is capable of listening on a socket through a non-blocking approach. In particular, the TcpListener
structure binds a Transmission Control Protocol
The Transmission Control Protocol (TCP) is one of the main protocols of the Internet protocol suite. It originated in the initial network implementation in which it complemented the Internet Protocol (IP). Therefore, the entire suite is commonly ...
(TCP) socket listener to an address and asynchronously executes function.
Broadcasting
Tokio provides a broadcast
Broadcasting is the distribution of audio or video content to a dispersed audience via any electronic mass communications medium, but typically one using the electromagnetic spectrum ( radio waves), in a one-to-many model. Broadcasting began ...
channel type, allowing for messages to be broadcast to multiple receivers. Upon sending a message, it is received by such receivers. This enables real-time communication and distributed systems, among other applications.
References
External links
*
* {{GitHub, tokio-rs/tokio, Tokio
Tokio
on crates.io
Free software programmed in Rust
Free computer libraries