Amazon S3 or Amazon Simple Storage Service is a service offered by

Amazon Web Services Amazon Web Services, Inc. (AWS) is a subsidiary of Amazon.com, Amazon that provides Software as a service, on-demand cloud computing computing platform, platforms and Application programming interface, APIs to individuals, companies, and gover ...

(AWS) that provides

object storage Object storage (also known as object-based storage) is a computer data storage that manages data as objects, as opposed to other storage architectures like file systems which manages data as a file hierarchy, and block storage which manages data as ...

through a web service interface. Amazon S3 uses the same scalable storage infrastructure that

Amazon.com Amazon.com, Inc. ( ) is an American multinational technology company focusing on e-commerce, cloud computing, online advertising, digital streaming, and artificial intelligence. It has been referred to as "one of the most influential economi ...

uses to run its e-commerce network. Amazon S3 can store any type of object, which allows uses like storage for Internet applications, backups, disaster recovery, data archives, data lakes for analytics, and

hybrid cloud storage Hybrid cloud storage, in Computer data storage, data storage, is a term for a storage infrastructure that uses a combination of on-premises storage resources with a public cloud storage provider. The on-premises storage is usually managed by the org ...

. AWS launched Amazon S3 in the United States on March 14, 2006, then in Europe in November 2007.

Design

Amazon S3 manages data with an

architecture which aims to provide

scalability Scalability is the property of a system to handle a growing amount of work by adding resources to the system. In an economic context, a scalable business model implies that a company can increase sales given increased resources. For example, a ...

high availability High availability (HA) is a characteristic of a system which aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period. Modernization has resulted in an increased reliance on these systems. Fo ...

, and

low latency Latency, from a general point of view, is a time delay between the cause and the effect of some physical change in the system being observed. Lag, as it is known in gaming circles, refers to the latency between the input to a simulation and t ...

with high

durability Durability is the ability of a physical product to remain functional, without requiring excessive maintenance or repair, when faced with the challenges of normal operation over its design lifetime. There are several measures of durability in use, ...

. The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key. Buckets can be managed using the console provided by Amazon S3, programmatically with the AWS SDK, or the

REST Rest or REST may refer to: Relief from activity * Sleep ** Bed rest * Kneeling * Lying (position) * Sitting * Squatting position Structural support * Structural support ** Rest (cue sports) ** Armrest ** Headrest ** Footrest Arts and entert ...

application programming interface. Objects can be up to five

terabyte The byte is a units of information, unit of digital information that most commonly consists of eight bits. Historically, the byte was the number of bits used to encode a single character (computing), character of text in a computer and for this ...

s in size. Requests are authorized using an

access control list In computer security, an access-control list (ACL) is a list of permissions associated with a system resource (object). An ACL specifies which users or system processes are granted access to objects, as well as what operations are allowed on giv ...

associated with each object bucket and support versioning which is disabled by default. Since buckets are typically the size of an entire file system mount in other systems, this access control scheme is very coarse-grained. In other words, unique access controls cannot be associated with individual files. Amazon S3 can be used to replace static

web-hosting A web hosting service is a type of Internet hosting service that hosts websites for clients, i.e. it offers the facilities required for them to create and maintain a site and makes it accessible on the World Wide Web. Companies providing web h ...

infrastructure with HTTP client-accessible objects, index document support and error document support. The Amazon AWS authentication mechanism allows the creation of authenticated URLs, valid for a specified amount of time. Every item in a bucket can also be served as a BitTorrent feed. The Amazon S3 store can act as a seed host for a

torrent Torrent or torrents may refer to: * A fast flowing stream Animals * Torrent duck, a species of the family Anatidae * Torrent fish * Torrent frog, various unrelated frogs * Torrent robin, a bird species * Torrent salamander, a family of sala ...

and any BitTorrent client can retrieve the file. This can drastically reduce the bandwidth cost for the download of popular objects. A bucket can be configured to save HTTP log information to a sibling bucket; this can be used in data mining operations. There are various User Mode File System (FUSE)–based file systems for Unix-like operating systems (for example,

Linux Linux ( or ) is a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds. Linux is typically packaged as a Linux distribution, which ...

) that can be used to mount an S3 bucket as a file system. The semantics of the Amazon S3 file system are not that of a

POSIX The Portable Operating System Interface (POSIX) is a family of standards specified by the IEEE Computer Society for maintaining compatibility between operating systems. POSIX defines both the system- and user-level application programming interf ...

file system, so the file system may not behave entirely as expected.

Amazon S3 storage classes

Amazon S3 offers eight different storage classes with different levels of durability, availability, and performance requirements. * Amazon S3 Standard is the default. It is general purpose storage for frequently accessed data. * Amazon S3 Standard-Infrequent Access (Standard-IA) is designed for less frequently accessed data, such as backups and disaster recovery data. * Amazon S3 One Zone-Infrequent Access (One Zone-IA) performs like the Standard-IA, but stores data only in one availability zone. * Amazon S3 Intelligent-Tiering moves objects automatically to a more cost-efficient storage class. * Amazon S3 on Outposts brings storage to installations not hosted by Amazon. * Amazon S3 Glacier Instant Retrieval is a low-cost storage for rarely accessed data, but which still requires rapid retrieval. * Amazon S3 Glacier Flexible Retrieval is also a low-cost option for long-lived data; it offers 3 retrieval speeds, ranging from minutes to hours. * Amazon S3 Glacier Deep Archive is another low-cost option. The Amazon S3 Glacier storage classes above are distinct from

Amazon Glacier Amazon S3 Glacier is an online file storage web service that provides storage for data archiving and backup. Glacier is part of the Amazon Web Services suite of cloud computing services, and is designed for long-term storage of data that is inf ...

, which is a separate product with its own APIs.

File size limits

An object in S3 can be between 1 byte and 5TB. If an object is larger than 5TB, it must be divided into chunks prior to uploading. When uploading, Amazon S3 allows a maximum of 5GB in a single upload operation; hence, objects larger than 5GB must be uploaded via the S3 multipart upload API.

Notable users

* Photo hosting service

SmugMug SmugMug is a paid image sharing, image hosting service, and online video platform on which users can upload photos and videos. The company also facilitates the sale of digital and print media for amateur and professional photographers. On April ...

has used Amazon S3 since April 2006. They experienced a number of initial outages and slowdowns, but after one year they described it as being "considerably more reliable than our own internal storage" and claimed to have saved almost $1 million in storage costs. *

Netflix Netflix, Inc. is an American subscription video on-demand over-the-top streaming service and production company based in Los Gatos, California. Founded in 1997 by Reed Hastings and Marc Randolph in Scotts Valley, California, it offers a fil ...

uses Amazon S3 as their

system of record A system of record (SOR) or source system of record (SSoR) is a data management term for an information storage system (commonly implemented on a computer system running a database management system) that is the authoritative data source for a ...

. Netflix implemented a tool, S3mper, to address the Amazon S3 limitations of

eventual consistency Eventual consistency is a consistency model used in distributed computing to achieve high availability that informally guarantees that, if no new updates are made to a given data item, eventually all accesses to that item will return the last upd ...

. S3mper stores the filesystem metadata: filenames, directory structure, and permissions in

Amazon DynamoDB Amazon DynamoDB is a fully managed proprietary NoSQL database service that supports key–value and document data structures and is offered by Amazon.com as part of the Amazon Web Services portfolio. DynamoDB exposes a similar data model to and ...

. *

Reddit Reddit (; stylized in all lowercase as reddit) is an American social news aggregation, content rating, and discussion website. Registered users (commonly referred to as "Redditors") submit content to the site such as links, text posts, images ...

is hosted on Amazon S3. *

Bitcasa Bitcasa, Inc. was an American cloud storage company founded in 2011 in St. Louis, Missouri. The company was later based in Mountain View, California until it shut down in 2017. Bitcasa provided client software for Microsoft Windows, OS X, And ...

, and

Tahoe-LAFS Tahoe-LAFS (Tahoe Least-Authority File Store) is a free and open, secure, decentralized, fault-tolerant, distributed data store and distributed file system. It can be used as an online backup system, or to serve as a file or Web host similar to ...

-on-S3, among others, use Amazon S3 for online backup and synchronization services. In 2016, Dropbox stopped using Amazon S3 services and developed its own cloud server. *

Tumblr Tumblr (stylized as tumblr; pronounced "tumbler") is an American microblogging and social networking website founded by David Karp in 2007 and currently owned by Automattic. The service allows users to post multimedia and other content to a sho ...

Formspring Spring.me was a social networking service. Until a rebranding in 2013, it was known as Formspring, a question-and-answer-based social network launched in 2009 by Ade Olonoh, the founder of online form builder Formstack. As of August 1, 2015, t ...

, and

Pinterest Pinterest is an American image sharing and social media service designed to enable saving and discovery of information (specifically "ideas") on the internet using images, and on a smaller scale, animated GIFs and videos, in the form of pinboard ...

host images on Amazon S3. *

Swiftype Swiftype is a search and index company based in San Francisco, California, that provides search software for organizations, websites, and computer programs. Notable customers include AT&T, Dr. Pepper, Hubspot and TechCrunch. History Swiftype was ...

's CEO has mentioned that the company uses Amazon S3.

S3 API and competing services

The broad adoption of Amazon S3 and related tooling has given rise to competing services based on the S3 API. These services use the standard programming interface but are differentiated by their underlying technologies and business models. A standard interface enables better competition from rival providers and allows

economies of scale In microeconomics, economies of scale are the cost advantages that enterprises obtain due to their scale of operation, and are typically measured by the amount of output produced per unit of time. A decrease in cost per unit of output enables ...

in implementation, among other benefits.

History

Amazon Web Services introduced Amazon S3 in 2006. Amazon reported it stored more than 100 trillion objects , up from 10 billion objects in October 2007, 14 billion objects in January 2008, 29 billion objects in October 2008, 52 billion objects in March 2009, 64 billion objects in August 2009, 102 billion objects in March 2010, and 2 trillion objects in April 2013. In November 2017 AWS added default encryption capabilities at bucket level.

References

Citations

Sources

* * * * {{Cloud computing S3 Cloud storage File hosting Network file systems de:Amazon Web Services#Amazon Simple Storage Service (S3)

Design

Amazon S3 storage classes

File size limits

Notable users

S3 API and competing services

History

See also

References

Citations

Sources