A database is an organized collection of data. A relational
database, more restrictively, is a collection of schemas, tables,
queries, reports, views, and other elements.
typically organize the data to model aspects of reality in a way that
supports processes requiring information, such as (for example)
modelling the availability of rooms in hotels in a way that supports
finding a hotel with vacancies.
A database-management system (DBMS) is a computer-software application
that interacts with end-users, other applications, and the database
itself to capture and analyze data. A general-purpose
DBMS allows the
definition, creation, querying, update, and administration of
A database is not generally portable across different DBMSs, but
different DBMSs can interoperate by using standards such as
JDBC to allow a single application to work with more than one
DBMS. Computer scientists may classify database-management systems
according to the database models that they support; the most popular
database systems since the 1980s have all supported the relational
model - generally associated with the
SQL language.[disputed –
discuss] Sometimes a
DBMS is loosely referred to as a "database".
1 Terminology and overview
3 General-purpose and special-purpose DBMSs
4.1 1960s, navigational DBMS
4.2 1970s, relational DBMS
4.3 Integrated approach
4.4 Late 1970s,
4.5 1980s, on the desktop
4.6 1990s, object-oriented
4.7 2000s, No
SQL and NewSQL
7 Design and modeling
7.2 External, conceptual, and internal views
9 Performance, security, and availability
9.1.1 Materialized views
9.3 Transactions and concurrency
9.5 Building, maintaining, and tuning
Backup and restore
9.7 Static analysis
10 See also
14 Further reading
15 External links
Terminology and overview
Formally, a "database" refers to a set of related data and the way it
is organized. Access to this data is usually provided by a "database
management system" (DBMS) consisting of an integrated set of computer
software that allows users to interact with one or more databases and
provides access to all of the data contained in the database (although
restrictions may exist that limit access to particular data). The DBMS
provides various functions that allow entry, storage and retrieval of
large quantities of information and provides ways to manage how that
information is organized.
Because of the close relationship between them, the term "database" is
often used casually to refer to both a database and the
DBMS used to
Outside the world of professional information technology, the term
database is often used to refer to any collection of related data
(such as a spreadsheet or a card index). This article is concerned
only with databases where the size and usage requirements necessitate
use of a database management system.
Existing DBMSs provide various functions that allow management of a
database and its data which can be classified into four main
Data definition – Creation, modification and removal of definitions
that define the organization of the data.
Update – Insertion, modification, and deletion of the actual
Retrieval – Providing information in a form directly usable or for
further processing by other applications. The retrieved data may be
made available in a form basically the same as it is stored in the
database or in a new form obtained by altering or combining existing
data from the database.
Administration – Registering and monitoring users, enforcing data
security, monitoring performance, maintaining data integrity, dealing
with concurrency control, and recovering information that has been
corrupted by some event such as an unexpected system failure.
Both a database and its
DBMS conform to the principles of a particular
database model. "
Database system" refers collectively to the
database model, database management system, and database.
Physically, database servers are dedicated computers that hold the
actual databases and run only the
DBMS and related software. Database
servers are usually multiprocessor computers, with generous memory and
RAID disk arrays used for stable storage.
RAID is used for recovery of
data if any of the disks fail. Hardware database accelerators,
connected to one or more servers via a high-speed channel, are also
used in large volume transaction processing environments. DBMSs are
found at the heart of most database applications. DBMSs may be built
around a custom multitasking kernel with built-in networking support,
but modern DBMSs typically rely on a standard operating system to
provide these functions.
Since DBMSs comprise a significant market, computer and storage
vendors often take into account
DBMS requirements in their own
Databases and DBMSs can be categorized according to the database
model(s) that they support (such as relational or XML), the type(s) of
computer they run on (from a server cluster to a mobile phone), the
query language(s) used to access the database (such as
SQL or XQuery),
and their internal engineering, which affects performance,
scalability, resilience, and security.
This section does not cite any sources. Please help improve this
section by adding citations to reliable sources. Unsourced material
may be challenged and removed. (March 2013) (Learn how and when to
remove this template message)
Databases are used to support internal operations of organizations and
to underpin online interactions with customers and suppliers (see
Databases are used to hold administrative information and more
specialized data, such as engineering data or economic models.
Examples of database applications include computerized library
systems, flight reservation systems, computerized parts inventory
systems, and many content management systems that store websites as
collections of webpages in a database.
General-purpose and special-purpose DBMSs
DBMS may become a complex software system and its development
typically requires thousands of human years of development effort.[a]
Some general-purpose DBMSs such as Adabas, Oracle and DB2 have been
upgraded since the 1970s. General-purpose DBMSs aim to meet the needs
of as many applications as possible, which adds to the complexity.
However, since their development cost can be spread over a large
number of users, they are often the most cost-effective approach. On
the other hand, a general-purpose
DBMS may introduce unnecessary
overhead. Therefore, many systems use a special-purpose DBMS. A common
example is an email system that performs many of the functions of a
DBMS such as the insertion and deletion of messages
composed of various items of data or associating messages with a
particular email address; but these functions are limited to what is
required to handle email and don't provide the user with all of the
functionality that would be available using a general-purpose DBMS.
Application software can often access a database on behalf of
end-users, without exposing the
DBMS interface directly. Application
programmers may use a wire protocol directly, or more likely through
an application programming interface.
Database designers and database
administrators interact with the
DBMS through dedicated interfaces to
build and maintain the applications' databases, and thus need some
more knowledge and understanding about how DBMSs operate and the
DBMSs' external interfaces and tuning parameters.
The sizes, capabilities, and performance of databases and their
respective DBMSs have grown in orders of magnitude. These performance
increases were enabled by the technology progress in the areas of
processors, computer memory, computer storage, and computer networks.
The development of database technology can be divided into three eras
based on data model or structure: navigational, SQL/relational, and
The two main early navigational data models were the hierarchical
model, epitomized by IBM's IMS system, and the
CODASYL model (network
model), implemented in a number of products such as IDMS.
The relational model, first proposed in 1970 by Edgar F. Codd,
departed from this tradition by insisting that applications should
search for data by content, rather than by following links. The
relational model employs sets of ledger-style tables, each used for a
different type of entity. Only in the mid-1980s did computing hardware
become powerful enough to allow the wide deployment of relational
systems (DBMSs plus applications). By the early 1990s, however,
relational systems dominated in all large-scale data processing
applications, and as of 2018[update] they remain dominant: IBM DB2,
Oracle, MySQL, and Microsoft
SQL Server are the top DBMS. The
dominant database language, standardised
SQL for the relational model,
has influenced database languages for other data models.[citation
Object databases were developed in the 1980s to overcome the
inconvenience of object-relational impedance mismatch, which led to
the coining of the term "post-relational" and also the development of
hybrid object-relational databases.
The next generation of post-relational databases in the late 2000s
became known as No
SQL databases, introducing fast key-value stores and
document-oriented databases. A competing "next generation" known as
SQL databases attempted new implementations that retained the
SQL model while aiming to match the high performance of
SQL compared to commercially available relational DBMSs.
1960s, navigational DBMS
Further information: Navigational database
Basic structure of navigational
CODASYL database model
The introduction of the term database coincided with the availability
of direct-access storage (disks and drums) from the mid-1960s onwards.
The term represented a contrast with the tape-based systems of the
past, allowing shared interactive use rather than daily batch
Oxford English Dictionary
Oxford English Dictionary cites a 1962 report by
the System Development Corporation of California as the first to use
the term "data-base" in a specific technical sense.
As computers grew in speed and capability, a number of general-purpose
database systems emerged; by the mid-1960s a number of such systems
had come into commercial use. Interest in a standard began to grow,
and Charles Bachman, author of one such product, the Integrated Data
Store (IDS), founded the "
Database Task Group" within CODASYL, the
group responsible for the creation and standardization of COBOL. In
Database Task Group delivered their standard, which
generally became known as the "
CODASYL approach", and soon a number of
commercial products based on this approach entered the market.
CODASYL approach relied on the "manual" navigation of a linked
data set which was formed into a large network. Applications could
find records by one of three methods:
Use of a primary key (known as a CALC key, typically implemented by
Navigating relationships (called sets) from one record to another
Scanning all the records in a sequential order
Later systems added B-trees to provide alternate access paths. Many
CODASYL databases also added a very straightforward query language.
However, in the final tally,
CODASYL was very complex and required
significant training and effort to produce useful applications.
IBM also had their own
DBMS in 1966, known as Information Management
System (IMS). IMS was a development of software written for the Apollo
program on the System/360. IMS was generally similar in concept to
CODASYL, but used a strict hierarchy for its model of data navigation
instead of CODASYL's network model. Both concepts later became known
as navigational databases due to the way data was accessed, and
Turing Award presentation was The Programmer as
Navigator. IMS is classified[by whom?] as a hierarchical database.
IDMS and Cincom Systems' TOTAL database are classified as network
databases. IMS remains in use as of 2014[update].
1970s, relational DBMS
Edgar Codd worked at IBM in San Jose, California, in one of their
offshoot offices that was primarily involved in the development of
hard disk systems. He was unhappy with the navigational model of the
CODASYL approach, notably the lack of a "search" facility. In 1970, he
wrote a number of papers that outlined a new approach to database
construction that eventually culminated in the groundbreaking A
Relational Model of
Data for Large Shared
In this paper, he described a new system for storing and working with
large databases. Instead of records being stored in some sort of
linked list of free-form records as in CODASYL, Codd's idea was to use
a "table" of fixed-length records, with each table used for a
different type of entity. A linked-list system would be very
inefficient when storing "sparse" databases where some of the data for
any one record could be left empty. The relational model solved this
by splitting the data into a series of normalized tables (or
relations), with optional elements being moved out of the main table
to where they would take up room only if needed.
Data may be freely
inserted, deleted and edited in these tables, with the
whatever maintenance needed to present a table view to the
In the relational model, records are "linked" using virtual keys not
stored in the database but defined as needed between the data
contained in the records.
The relational model also allowed the content of the database to
evolve without constant rewriting of links and pointers. The
relational part comes from entities referencing other entities in what
is known as one-to-many relationship, like a traditional hierarchical
model, and many-to-many relationship, like a navigational (network)
model. Thus, a relational model can express both hierarchical and
navigational models, as well as its native tabular model, allowing for
pure or combined modeling in terms of these three models, as the
For instance, a common use of a database system is to track
information about users, their name, login information, various
addresses and phone numbers. In the navigational approach, all of this
data would be placed in a single record, and unused items would simply
not be placed in the database. In the relational approach, the data
would be normalized into a user table, an address table and a phone
number table (for instance). Records would be created in these
optional tables only if the address or phone numbers were actually
Linking the information back together is the key to this system. In
the relational model, some bit of information was used as a "key",
uniquely defining a particular record. When information was being
collected about a user, information stored in the optional tables
would be found by searching for this key. For instance, if the login
name of a user is unique, addresses and phone numbers for that user
would be recorded with the login name as its key. This simple
"re-linking" of related data back into a single collection is
something that traditional computer languages are not designed for.
Just as the navigational approach would require programs to loop in
order to collect records, the relational approach would require loops
to collect information about any one record. Codd's suggestions was a
set-oriented language, that would later spawn the ubiquitous SQL.
Using a branch of mathematics known as tuple calculus, he demonstrated
that such a system could support all the operations of normal
databases (inserting, updating etc.) as well as providing a simple
system for finding and returning sets of data in a single operation.
Codd's paper was picked up by two people at Berkeley, Eugene Wong and
Michael Stonebraker. They started a project known as
funding that had already been allocated for a geographical database
project and student programmers to produce code. Beginning in 1973,
INGRES delivered its first test products which were generally ready
for widespread use in 1979.
INGRES was similar to System R in a number
of ways, including the use of a "language" for data access, known as
QUEL. Over time,
INGRES moved to the emerging
IBM itself did one test implementation of the relational model, PRTV,
and a production one, Business System 12, both now discontinued.
Honeywell wrote MRDS for Multics, and now there are two new
Dataphor and Rel. Most other DBMS
implementations usually called relational are actually
In 1970, the
University of Michigan
University of Michigan began development of the MICRO
Information Management System based on D.L. Childs' Set-Theoretic
Data model. MICRO was used to manage very large data sets
by the US Department of Labor, the U.S. Environmental Protection
Agency, and researchers from the University of Alberta, the University
of Michigan, and Wayne State University. It ran on IBM mainframe
computers using the Michigan Terminal System. The system remained
in production until 1998.
In the 1970s and 1980s, attempts were made to build database systems
with integrated hardware and software. The underlying philosophy was
that such integration would provide higher performance at lower cost.
Examples were IBM System/38, the early offering of Teradata, and the
Britton Lee, Inc.
Britton Lee, Inc. database machine.
Another approach to hardware support for database management was ICL's
CAFS accelerator, a hardware disk controller with programmable search
capabilities. In the long term, these efforts were generally
unsuccessful because specialized database machines could not keep pace
with the rapid development and progress of general-purpose computers.
Thus most database systems nowadays are software systems running on
general-purpose hardware, using general-purpose computer data storage.
However this idea is still pursued for certain applications by some
Netezza and Oracle (Exadata).
IBM started working on a prototype system loosely based on Codd's
concepts as System R in the early 1970s. The first version was ready
in 1974/5, and work then started on multi-table systems in which the
data could be split so that all of the data for a record (some of
which is optional) did not have to be stored in a single large
"chunk". Subsequent multi-user versions were tested by customers in
1978 and 1979, by which time a standardized query language –
SQL – had been added. Codd's ideas were
establishing themselves as both workable and superior to CODASYL,
pushing IBM to develop a true production version of System R, known as
SQL/DS, and, later,
Database 2 (DB2).
Oracle Database (or more simply, Oracle) started from
a different chain, based on IBM's papers on System R. Though Oracle V1
implementations were completed in 1978, it wasn't until Oracle Version
2 when Ellison beat IBM to market in 1979.
Stonebraker went on to apply the lessons from
INGRES to develop a new
database, Postgres, which is now known as PostgreSQL. Postgre
often used for global mission critical applications (the .org and
.info domain name registries use it as their primary data store, as do
many large companies and financial institutions).
In Sweden, Codd's paper was also read and Mimer
SQL was developed from
the mid-1970s at Uppsala University. In 1984, this project was
consolidated into an independent enterprise. In the early 1980s, Mimer
introduced transaction handling for high robustness in applications,
an idea that was subsequently implemented on most other DBMSs.
Another data model, the entity–relationship model, emerged in 1976
and gained popularity for database design as it emphasized a more
familiar description than the earlier relational model. Later on,
entity–relationship constructs were retrofitted as a data modeling
construct for the relational model, and the difference between the two
have become irrelevant.
1980s, on the desktop
The 1980s ushered in the age of desktop computing. The new computers
empowered their users with spreadsheets like
Lotus 1-2-3 and database
software like dBASE. The dBASE product was lightweight and easy for
any computer user to understand out of the box.
C. Wayne Ratliff the
creator of dBASE stated: "dBASE was different from programs like
BASIC, C, FORTRAN, and
COBOL in that a lot of the dirty work had
already been done. The data manipulation is done by dBASE instead of
by the user, so the user can concentrate on what he is doing, rather
than having to mess with the dirty details of opening, reading, and
closing files, and managing space allocation." dBASE was one of
the top selling software titles in the 1980s and early 1990s.
The 1990s, along with a rise in object-oriented programming, saw a
growth in how data in various databases were handled. Programmers and
designers began to treat the data in their databases as objects. That
is to say that if a person's data were in a database, that person's
attributes, such as their address, phone number, and age, were now
considered to belong to that person instead of being extraneous data.
This allows for relations between data to be relations to objects and
their attributes and not to individual fields. The term
"object-relational impedance mismatch" described the inconvenience of
translating between programmed objects and database tables. Object
databases and object-relational databases attempt to solve this
problem by providing an object-oriented language (sometimes as
extensions to SQL) that programmers can use as alternative to purely
relational SQL. On the programming side, libraries known as
object-relational mappings (ORMs) attempt to solve the same problem.
SQL and NewSQL
Main articles: No
SQL and NewSQL
XML databases are a type of structured document-oriented database that
allows querying based on
XML document attributes.
XML databases are
mostly used in enterprise database management, where
XML is being used
as the machine-to-machine data interoperability standard.
management systems include commercial software
MarkLogic and Oracle
Berkeley DB XML, and a free use software
XML/JSON Database. All are enterprise software database platforms and
support industry standard ACID-compliant transaction processing with
strong database consistency characteristics and high level of database
SQL databases are often very fast, do not require fixed table
schemas, avoid join operations by storing denormalized data, and are
designed to scale horizontally. The most popular No
SQL systems include
MongoDB, Couchbase, Riak, Memcached, Redis, CouchDB, Hazelcast, Apache
Cassandra, and HBase, which are all open-source software products.
In recent years, there was a high demand for massively distributed
databases with high partition tolerance but according to the CAP
theorem it is impossible for a distributed system to simultaneously
provide consistency, availability, and partition tolerance guarantees.
A distributed system can satisfy any two of these guarantees at the
same time, but not all three. For that reason, many No
are using what is called eventual consistency to provide both
availability and partition tolerance guarantees with a reduced level
of data consistency.
SQL is a class of modern relational databases that aims to provide
the same scalable performance of No
SQL systems for online transaction
processing (read-write) workloads while still using
ACID guarantees of a traditional database system. Such
databases include Google F1/Spanner, Citus, CockroachDB, TiDB,
ScaleBase, MemSQL, NuoDB, and VoltDB.
Database technology has been an active research topic since the 1960s,
both in academia and in the research and development groups of
companies (for example IBM Research). Research activity includes
theory and development of prototypes. Notable research topics have
included models, the atomic transaction concept, and related
concurrency control techniques, query languages and query optimization
methods, RAID, and more.
The database research area has several dedicated academic journals
(for example, ACM Transactions on
Knowledge Engineering-DKE) and annual conferences (e.g., ACM SIGMOD,
ACM PODS, VLDB,
One way to classify databases involves the type of their contents, for
example: bibliographic, document-text, statistical, or multimedia
objects. Another way is by their application area, for example:
accounting, music compositions, movies, banking, manufacturing, or
insurance. A third way is by some technical aspect, such as the
database structure or interface type. This section lists a few of the
adjectives used to characterize different kinds of databases.
An in-memory database is a database that primarily resides in main
memory, but is typically backed-up by non-volatile computer data
Main memory databases are faster than disk databases, and so
are often used where response time is critical, such as in
telecommunications network equipment.
An active database includes an event-driven architecture which can
respond to conditions both inside and outside the database. Possible
uses include security monitoring, alerting, statistics gathering and
authorization. Many databases provide active database features in the
form of database triggers.
A cloud database relies on cloud technology. Both the database and
most of its
DBMS reside remotely, "in the cloud", while its
applications are both developed by programmers and later maintained
and utilized by (application's) end-users through a web browser and
Data warehouses archive data from operational databases and often from
external sources such as market research firms. The warehouse becomes
the central source of data for use by managers and other end-users who
may not have access to operational data. For example, sales data might
be aggregated to weekly totals and converted from internal product
codes to use UPCs so that they can be compared with
Some basic and essential components of data warehousing include
extracting, analyzing, and mining data, transforming, loading, and
managing data so as to make them available for further use.
A deductive database combines logic programming with a relational
database, for example by using the
A distributed database is one in which both the data and the
A document-oriented database is designed for storing, retrieving, and
managing document-oriented, or semi structured data, information.
Document-oriented databases are one of the main categories of NoSQL
An embedded database system is a
DBMS which is tightly integrated with
an application software that requires access to stored data in such a
way that the
DBMS is hidden from the application's end-users and
requires little or no ongoing maintenance.
End-user databases consist of data developed by individual end-users.
Examples of these are collections of documents, spreadsheets,
presentations, multimedia, and other files. Several products exist to
support such databases. Some of them are much simpler than
full-fledged DBMSs, with more elementary
A federated database system comprises several distinct databases, each
with its own DBMS. It is handled as a single database by a federated
database management system (FDBMS), which transparently integrates
multiple autonomous DBMSs, possibly of different types (in which case
it would also be a heterogeneous database system), and provides them
with an integrated conceptual view.
Sometimes the term multi-database is used as a synonym to federated
database, though it may refer to a less integrated (e.g., without an
DBMS and a managed integrated schema) group of databases that
cooperate in a single application. In this case, typically middleware
is used for distribution, which typically includes an atomic commit
protocol (ACP), e.g., the two-phase commit protocol, to allow
distributed (global) transactions across the participating databases.
A graph database is a kind of No
SQL database that uses graph
structures with nodes, edges, and properties to represent and store
information. General graph databases that can store any graph are
distinct from specialized graph databases such as triplestores and
DBMS is a kind of No
DBMS that allows modeling, storage,
and retrieval of (usually large) multi-dimensional arrays such as
satellite images and climate simulation output.
In a hypertext or hypermedia database, any word or a piece of text
representing an object, e.g., another piece of text, an article, a
picture, or a film, can be hyperlinked to that object. Hypertext
databases are particularly useful for organizing large amounts of
disparate information. For example, they are useful for organizing
online encyclopedias, where users can conveniently jump around the
World Wide Web
World Wide Web is thus a large distributed hypertext
A knowledge base (abbreviated KB, kb or Δ) is a special kind
of database for knowledge management, providing the means for the
computerized collection, organization, and retrieval of knowledge.
Also a collection of data representing problems with their solutions
and related experiences.
A mobile database can be carried on or synchronized from a mobile
Operational databases store detailed data about the operations of an
organization. They typically process relatively high volumes of
updates using transactions. Examples include customer databases that
record contact, credit, and demographic information about a business'
customers, personnel databases that hold information such as salary,
benefits, skills data about employees, enterprise resource planning
systems that record details about product components, parts inventory,
and financial databases that keep track of the organization's money,
accounting and financial dealings.
A parallel database seeks to improve performance through
parallelization for tasks such as loading data, building indexes and
The major parallel
DBMS architectures which are induced by the
underlying hardware architecture are:
Shared memory architecture, where multiple processors share the main
memory space, as well as other data storage.
Shared disk architecture, where each processing unit (typically
consisting of multiple processors) has its own main memory, but all
units share the other storage.
Shared nothing architecture, where each processing unit has its own
main memory and other storage.
Probabilistic databases employ fuzzy logic to draw inferences from
Real-time databases process transactions fast enough for the result to
come back and be acted on right away.
A spatial database can store the data with multidimensional features.
The queries on such data include location-based queries, like "Where
is the closest hotel in my area?".
A temporal database has built-in time aspects, for example a temporal
data model and a temporal version of SQL. More specifically the
temporal aspects usually include valid-time and transaction-time.
A terminology-oriented database builds upon an object-oriented
database, often customized for a specific field.
An unstructured data database is intended to store in a manageable and
protected way diverse objects that do not fit naturally and
conveniently in common databases. It may include email messages,
documents, journals, multimedia objects, etc. The name may be
misleading since some objects can be highly structured. However, the
entire possible object collection does not fit into a predefined
structured framework. Most established DBMSs now support unstructured
data in various ways, and new dedicated DBMSs are emerging.
Design and modeling
The first task of a database designer is to produce a conceptual data
model that reflects the structure of the information to be held in the
database. A common approach to this is to develop an
entity-relationship model, often with the aid of drawing tools.
Another popular approach is the Unified Modeling Language. A
successful data model will accurately reflect the possible state of
the external world being modeled: for example, if people can have more
than one phone number, it will allow this information to be captured.
Designing a good conceptual data model requires a good understanding
of the application domain; it typically involves asking deep questions
about the things of interest to an organization, like "can a customer
also be a supplier?", or "if a product is sold with two different
forms of packaging, are those the same product or different
products?", or "if a plane flies from New York to Dubai via Frankfurt,
is that one flight or two (or maybe even three)?". The answers to
these questions establish definitions of the terminology used for
entities (customers, products, flights, flight segments) and their
relationships and attributes.
Producing the conceptual data model sometimes involves input from
business processes, or the analysis of workflow in the organization.
This can help to establish what information is needed in the database,
and what can be left out. For example, it can help when deciding
whether the database needs to hold historic data as well as current
Having produced a conceptual data model that users are happy with, the
next stage is to translate this into a schema that implements the
relevant data structures within the database. This process is often
called logical database design, and the output is a logical data model
expressed in the form of a schema. Whereas the conceptual data model
is (in theory at least) independent of the choice of database
technology, the logical data model will be expressed in terms of a
particular database model supported by the chosen DBMS. (The terms
data model and database model are often used interchangeably, but in
this article we use data model for the design of a specific database,
and database model for the modelling notation used to express that
The most popular database model for general-purpose databases is the
relational model, or more precisely, the relational model as
represented by the
SQL language. The process of creating a logical
database design using this model uses a methodical approach known as
normalization. The goal of normalization is to ensure that each
elementary "fact" is only recorded in one place, so that insertions,
updates, and deletions automatically maintain consistency.
The final stage of database design is to make the decisions that
affect performance, scalability, recovery, security, and the like,
which depend on the particular DBMS. This is often called physical
database design, and the output is the physical data model. A key goal
during this stage is data independence, meaning that the decisions
made for performance optimization purposes should be invisible to
end-users and applications. There are two types of data independence:
Physical data independence and logical data independence. Physical
design is driven mainly by performance requirements, and requires a
good knowledge of the expected workload and access patterns, and a
deep understanding of the features offered by the chosen DBMS.
Another aspect of physical database design is security. It involves
both defining access control to database objects as well as defining
security levels and methods for the data itself.
Collage of five types of database models
A database model is a type of data model that determines the logical
structure of a database and fundamentally determines in which manner
data can be stored, organized, and manipulated. The most popular
example of a database model is the relational model (or the SQL
approximation of relational), which uses a table-based format.
Common logical data models for databases include:
Hierarchical database model
Enhanced entity–relationship model
An object-relational database combines the two related structures.
Physical data models include:
Other models include:
Specialized models are optimized for particular types of data:
Time series model
External, conceptual, and internal views
Traditional view of data
A database management system provides three views of the database
The external level defines how each group of end-users sees the
organization of data in the database. A single database can have any
number of views at the external level.
The conceptual level unifies the various external views into a
compatible global view. It provides the synthesis of all the
external views. It is out of the scope of the various database
end-users, and is rather of interest to database application
developers and database administrators.
The internal level (or physical level) is the internal organization of
data inside a DBMS. It is concerned with cost, performance,
scalability and other operational matters. It deals with storage
layout of the data, using storage structures such as indexes to
enhance performance. Occasionally it stores data of individual views
(materialized views), computed from generic data, if performance
justification exists for such redundancy. It balances all the external
views' performance requirements, possibly conflicting, in an attempt
to optimize overall performance across all activities.
While there is typically only one conceptual (or logical) and physical
(or internal) view of the data, there can be any number of different
external views. This allows users to see database information in a
more business-related way rather than from a technical, processing
viewpoint. For example, a financial department of a company needs the
payment details of all employees as part of the company's expenses,
but does not need details about employees that are the interest of the
human resources department. Thus different departments need different
views of the company's database.
The three-level database architecture relates to the concept of data
independence which was one of the major initial driving forces of the
relational model. The idea is that changes made at a certain level do
not affect the view at a higher level. For example, changes in the
internal level do not affect application programs written using
conceptual level interfaces, which reduces the impact of making
physical changes to improve performance.
The conceptual view provides a level of indirection between internal
and external. On one hand it provides a common view of the database,
independent of different external view structures, and on the other
hand it abstracts away details of how the data are stored or managed
(internal level). In principle every level, and even every external
view, can be presented by a different data model. In practice usually
DBMS uses the same data model for both the external and the
conceptual levels (e.g., relational model). The internal level, which
is hidden inside the
DBMS and depends on its implementation, requires
a different level of detail and uses its own types of data structure
Separating the external, conceptual and internal levels was a major
feature of the relational database model implementations that dominate
21st century databases.
Database languages are special-purpose languages, which allows one or
more of the following tasks, sometimes distinguished as sublanguages:
Data control language (DCL) – controls access to data;
Data definition language (DDL) – defines data types such as
creating, altering, or dropping and the relationships among them;
Data manipulation language (DML) – performs tasks such as inserting,
updating, or deleting data occurrences;
Data query language (DQL) – allows searching for information and
computing derived information.
Database languages are specific to a particular data model. Notable
SQL combines the roles of data definition, data manipulation, and
query in a single language. It was one of the first commercial
languages for the relational model, although it departs in some
respects from the relational model as described by Codd (for example,
the rows and columns of a table can be ordered).
SQL became a standard
American National Standards Institute
American National Standards Institute (ANSI) in 1986, and of
International Organization for Standardization
International Organization for Standardization (ISO) in 1987. The
standards have been regularly enhanced since and is supported (with
varying degrees of conformance) by all mainstream commercial
OQL is an object model language standard (from the Object Data
Management Group). It has influenced the design of some of the newer
query languages like JD
OQL and EJB QL.
XQuery is a standard
XML query language implemented by
systems such as
MarkLogic and eXist, by relational databases with XML
capability such as Oracle and DB2, and also by in-memory XML
processors such as Saxon.
XQuery with SQL.
A database language may also incorporate features like:
DBMS-specific Configuration and storage engine management
Computations to modify query results, like counting, summing,
averaging, sorting, grouping, and cross-referencing
Constraint enforcement (e.g. in an automotive database, only allowing
one engine type per car)
Application programming interface version of the query language, for
Performance, security, and availability
Because of the critical importance of database technology to the
smooth running of an enterprise, database systems include complex
mechanisms to deliver the required performance, security, and
availability, and allow database administrators to control the use of
Computer data storage
Computer data storage and
Database storage is the container of the physical materialization of a
database. It comprises the internal (physical) level in the database
architecture. It also contains all the information needed (e.g.,
metadata, "data about the data", and internal data structures) to
reconstruct the conceptual level and external level from the internal
level when needed. Putting data into permanent storage is generally
the responsibility of the database engine a.k.a. "storage engine".
Though typically accessed by a
DBMS through the underlying operating
system (and often utilizing the operating systems' file systems as
intermediates for storage layout), storage properties and
configuration setting are extremely important for the efficient
operation of the DBMS, and thus are closely maintained by database
administrators. A DBMS, while in operation, always has its database
residing in several types of storage (e.g., memory and external
storage). The database data and the additional needed information,
possibly in very large amounts, are coded into bits.
reside in the storage in structures that look completely different
from the way the data look in the conceptual and external levels, but
in ways that attempt to optimize (the best possible) these levels'
reconstruction when needed by users and programs, as well as for
computing additional types of needed information from the data (e.g.,
when querying the database).
Some DBMSs support specifying which character encoding was used to
store data, so multiple encodings can be used in the same database.
Various low-level database storage structures are used by the storage
engine to serialize the data model so it can be written to the medium
of choice. Techniques such as indexing may be used to improve
performance. Conventional storage is row-oriented, but there are also
column-oriented and correlation databases.
Main article: Materialized view
Often storage redundancy is employed to increase performance. A common
example is storing materialized views, which consist of frequently
needed external views or query results. Storing such views saves the
expensive computing of them each time they are needed. The downsides
of materialized views are the overhead incurred when updating them to
keep them synchronized with their original updated database data, and
the cost of storage redundancy.
Occasionally a database employs storage redundancy by database objects
replication (with one or more copies) to increase data availability
(both to improve performance of simultaneous multiple end-user
accesses to a same database object, and to provide resiliency in a
case of partial failure of a distributed database). Updates of a
replicated object need to be synchronized across the object copies. In
many cases, the entire database is replicated.
This article appears to contradict the article
Please see discussion on the linked talk page. (March 2013) (Learn how
and when to remove this template message)
Database security deals with all various aspects of protecting the
database content, its owners, and its users. It ranges from protection
from intentional unauthorized database uses to unintentional database
accesses by unauthorized entities (e.g., a person or a computer
Database access control deals with controlling who (a person or a
certain computer program) is allowed to access what information in the
database. The information may comprise specific database objects
(e.g., record types, specific records, data structures), certain
computations over certain objects (e.g., query types, or specific
queries), or utilizing specific access paths to the former (e.g.,
using specific indexes or other data structures to access
Database access controls are set by special authorized
(by the database owner) personnel that uses dedicated protected
This may be managed directly on an individual basis, or by the
assignment of individuals and privileges to groups, or (in the most
elaborate models) through the assignment of individuals and groups to
roles which are then granted entitlements.
Data security prevents
unauthorized users from viewing or updating the database. Using
passwords, users are allowed access to the entire database or subsets
of it called "subschemas". For example, an employee database can
contain all the data about an individual employee, but one group of
users may be authorized to view only payroll data, while others are
allowed access to only work history and medical data. If the DBMS
provides a way to interactively enter and update the database, as well
as interrogate it, this capability allows for managing personal
Data security in general deals with protecting specific chunks of
data, both physically (i.e., from corruption, or destruction, or
removal; e.g., see physical security), or the interpretation of them,
or parts of them to meaningful information (e.g., by looking at the
strings of bits that they comprise, concluding specific valid
credit-card numbers; e.g., see data encryption).
Change and access logging records who accessed which attributes, what
was changed, and when it was changed. Logging services allow for a
forensic database audit later by keeping a record of access
occurrences and changes. Sometimes application-level code is used to
record changes rather than leaving this to the database. Monitoring
can be set up to attempt to detect security breaches.
Transactions and concurrency
Further information: Concurrency control
Database transactions can be used to introduce some level of fault
tolerance and data integrity after recovery from a crash. A database
transaction is a unit of work, typically encapsulating a number of
operations over a database (e.g., reading a database object, writing,
acquiring lock, etc.), an abstraction supported in database and also
other systems. Each transaction has well defined boundaries in terms
of which program/code executions are included in that transaction
(determined by the transaction's programmer via special transaction
ACID describes some ideal properties of a database
transaction: Atomicity, Consistency, Isolation, and Durability.
Data migration §
A database built with one
DBMS is not portable to another
DBMS cannot run it). However, in some situations, it is
desirable to move, migrate a database from one
DBMS to another. The
reasons are primarily economical (different DBMSs may have different
total costs of ownership or TCOs), functional, and operational
(different DBMSs may have different capabilities). The migration
involves the database's transformation from one
DBMS type to another.
The transformation should maintain (if possible) the database related
application (i.e., all related application programs) intact. Thus, the
database's conceptual and external architectural levels should be
maintained in the transformation. It may be desired that also some
aspects of the architecture internal level are maintained. A complex
or large database migration may be a complicated and costly (one-time)
project by itself, which should be factored into the decision to
migrate. This in spite of the fact that tools may exist to help
migration between specific DBMSs. Typically, a
DBMS vendor provides
tools to help importing databases from other popular DBMSs.
Building, maintaining, and tuning
After designing a database for an application, the next stage is
building the database. Typically, an appropriate general-purpose DBMS
can be selected to be utilized for this purpose. A
DBMS provides the
needed user interfaces to be utilized by database administrators to
define the needed application's data structures within the DBMS's
respective data model. Other user interfaces are used to select needed
DBMS parameters (like security related, storage allocation parameters,
When the database is ready (all its data structures and other needed
components are defined), it is typically populated with initial
application's data (database initialization, which is typically a
distinct project; in many cases using specialized
DBMS interfaces that
support bulk insertion) before making it operational. In some cases,
the database becomes operational while empty of application data, and
data are accumulated during its operation.
After the database is created, initialised and populated it needs to
be maintained. Various database parameters may need changing and the
database may need to be tuned (tuning) for better performance;
application's data structures may be changed or added, new related
application programs may be written to add to the application's
Backup and restore
Main article: Backup
Sometimes it is desired to bring a database back to a previous state
(for many reasons, e.g., cases when the database is found corrupted
due to a software error, or if it has been updated with erroneous
data). To achieve this, a backup operation is done occasionally or
continuously, where each desired database state (i.e., the values of
its data and their embedding in database's data structures) is kept
within dedicated backup files (many techniques exist to do this
effectively). When this state is needed, i.e., when it is decided by a
database administrator to bring the database back to this state (e.g.,
by specifying this state by a desired point in time when the database
was in this state), these files are utilized to restore that state.
Static analysis techniques for software verification can be applied
also in the scenario of query languages. In particular, the *Abstract
interpretation framework has been extended to the field of query
languages for relational databases as a way to support sound
approximation techniques. The semantics of query languages can be
tuned according to suitable abstractions of the concrete domain of
data. The abstraction of relational database system has many
interesting applications, in particular, for security purposes, such
as fine grained access control, watermarking, etc.
DBMS features might include:
Graphics component for producing graphs and charts, especially in a
data warehouse system
Query optimizer – Performs query optimization on every query to
choose an efficient query plan (a partial order (tree) of operations)
to be executed to compute the query result. May be specific to a
particular storage engine.
Tools or hooks for database design, application programming,
application program maintenance, database performance analysis and
monitoring, database configuration monitoring,
DBMS and related database may span computers,
networks, and storage units) and related database mapping (especially
for a distributed DBMS), storage allocation and database layout
monitoring, storage migration, etc.
Increasingly, there are calls for a single system that incorporates
all of these core functionalities into the same build, test, and
deployment framework for database management and source control.
Borrowing from other developments in the software industry, some
market such offerings as "
DevOps for database".
For a topical guide to this subject, see Outline of databases.
Comparison of database tools
Comparison of object database management systems
Comparison of object-relational database management systems
Comparison of relational database management systems
^ This article quotes a development time of 5 years involving 750
people for DB2 release 9 alone.(Chong et al. 2007)
Database – Definition of database by Merriam-Webster".
^ Ullman & Widom 1997, p. 1.
^ "Update – Definition of update by Merriam-Webster".
^ "Retrieval – Definition of retrieval by Merriam-Webster".
^ "Administration – Definition of administration by
^ Tsitchizris & Lochovsky 1982.
^ Beynon–Davies 2003.
^ Nelson & Nelson 2001.
^ Bachman 1973.
^ "TOPDB Top
Database index". pypl.github.io.
^ "database, n". OED Online. Oxford University Press. June 2013.
Retrieved July 12, 2013.
^ IBM Corporation. "
IBM Information Management System
IBM Information Management System (IMS) 13
Database Servers delivers high performance and low
total cost of ownership". Retrieved Feb 20, 2014.
^ Codd 1970.
^ Hershey & Easthope 1972.
^ North 2010.
^ Childs 1968a.
^ Childs 1968b.
^ MICRO Information Management System (Version 5.0) Reference Manual,
M.A. Kahn, D.L. Rumelhart, and B.L. Bronson, October 1977, Institute
of Labor and Industrial Relations (ILIR),
University of Michigan
University of Michigan and
Wayne State University
^ "Oracle 30th Anniversary Timeline" (PDF). Retrieved 23 August
^ Interview with Wayne Ratliff. The FoxPro History. Retrieved on
^ Development of an object-oriented DBMS; Portland, Oregon, United
States; Pages: 472–482; 1986; ISBN 0-89791-204-7
^ "Oracle Berkeley DB XML" (PDF). Retrieved 10 March 2015.
ACID Transactions, MarkLogic". Retrieved 10 March 2015.
Database at a Glance". Archived from the original on 2
April 2015. Retrieved 10 March 2015.
^ "DB-Engines Ranking". January 2013. Retrieved 22 January 2013.
^ Proctor 2013.
^ Graves, Steve. "COTS Databases For Embedded Systems", Embedded
Computing Design magazine, January 2007. Retrieved on August 13, 2008.
^ Argumentation in Artificial Intelligence by Iyad Rahwan, Guillermo
^ "OWL DL Semantics". Retrieved 10 December 2010.
^ itl.nist.gov (1993) Integration Definition for Information Modeling
(IDEFIX). 21 December 1993.
^ a b Date 2003, pp. 31–32.
^ Chapple 2005.
^ "Structured Query Language (SQL)". International Business Machines.
October 27, 2006. Retrieved 2007-06-10.
^ Wagner 2010.
^ Halder & Cortesi 2011.
^ Ben Linders (January 28, 2016). "How
Database Administration Fits
into DevOps". Retrieved April 15, 2017.
Bachman, Charles W. (1973). "The Programmer as Navigator".
Communications of the ACM. 16 (11): 653–658.
doi:10.1145/355611.362534. (Subscription required (help)).
Beynon–Davies, Paul (2003).
Database Systems (3rd ed.). Palgrave
Macmillan. ISBN 978-1403916013.
Chapple, Mike (2005). "
SQL Fundamentals". Databases. About.com.
Archived from the original on 22 February 2009. Retrieved 28 January
Childs, David L. (1968a). "Description of a set-theoretic data
structure" (PDF). CONCOMP (Research in Conversational Use of
Computers) Project. Technical Report 3. University of Michigan.
Childs, David L. (1968b). "Feasibility of a set-theoretic data
structure: a general structure based on a reconstituted definition"
(PDF). CONCOMP (Research in Conversational Use of Computers) Project.
Technical Report 6. University of Michigan.
Chong, Raul F.; Wang, Xiaomei; Dang, Michael; Snow, Dwaine R. (2007).
"Introduction to DB2". Understanding DB2: Learning Visually with
Examples (2nd ed.). ISBN 978-0131580183. Retrieved 17 March
Codd, Edgar F. (1970). "A Relational Model of
Data for Large Shared
Data Banks" (PDF). Communications of the ACM. 13 (6): 377–387.
Date, C. J. (2003). An Introduction to
Database Systems (8th ed.).
Pearson. ISBN 978-0321197849.
Halder, Raju; Cortesi, Agostino (2011). "Abstract Interpretation of
Database Query Languages" (PDF). COMPUTER LANGUAGES, SYSTEMS &
STRUCTURES. Elsevier. 38 (2): 123–157. doi:10.1016/j.cl.2011.10.004.
Hershey, William; Easthope, Carol (1972). A set theoretic data
structure and retrieval language. Spring Joint Computer Conference,
May 1972. ACM SIGIR Forum. 7 (4). pp. 45–55.
Nelson, Anne Fulcher; Nelson, William Harris Morehead (2001). Building
Electronic Commerce: With Web
Database Constructions. Prentice Hall.
North, Ken (10 March 2010). "Sets,
Data Models and
Dr. Dobb's. Archived from the original on 24 October 2010.
Proctor, Seth (12 July 2013). "Exploring the Architecture of the NuoDB
Database, Part 1". Archived from the original on 15 July 2013.
Retrieved 12 July 2013.
Tsitchizris, Dionysios C.; Lochovsky, Fred H. (1982).
Prentice–Hall. ISBN 978-0131964280.
Ullman, Jeffrey; Widom, Jennifer (1997). A First Course in Database
Systems. Prentice–Hall. ISBN 0138613370.
Wagner, Michael (2010), SQL/XML:2006 – Evaluierung der
Standardkonformität ausgewählter Datenbanksysteme, Diplomica Verlag,
Ling Liu and Tamer M. Özsu (Eds.) (2009). "Encyclopedia of Database
Systems, 4100 p. 60 illus. ISBN 978-0-387-49616-0.
Connolly, Thomas and Carolyn Begg.
Database Systems. New York: Harlow,
Gray, J. and Reuter, A. Transaction Processing: Concepts and
Techniques, 1st edition, Morgan Kaufmann Publishers, 1992.
Kroenke, David M. and David J. Auer.
Database Concepts. 3rd ed. New
York: Prentice, 2007.
Raghu Ramakrishnan and Johannes Gehrke,
Database Management Systems
Abraham Silberschatz, Henry F. Korth, S. Sudarshan,
Lightstone, S.; Teorey, T.; Nadeau, T. (2007). Physical Database
Design: the database professional's guide to exploiting indexes,
views, storage, and more. Morgan Kaufmann Press.
Teorey, T.; Lightstone, S. and Nadeau, T.
Database Modeling &
Design: Logical Design, 4th edition, Morgan Kaufmann Press, 2005.
Find more aboutDatabaseat's sister projects
Definitions from Wiktionary
Media from Wikimedia Commons
News from Wikinews
Quotations from Wikiquote
Texts from Wikisource
Textbooks from Wikibooks
Learning resources from Wikiversity
File extension – information about files with the DB
Database management system
Entities and relationships (and Enhanced notation)
Locks with ordered sharing
Database management systems
Administration and automation
Federated database system
XML data store
Creating the data warehouse
Data vault modeling
Operational data store
Sixth normal form
Using the data warehouse
Decision support system
Decision support system (DSS)
Data warehouse automation
Data Mining Extensions (DMX)
MultiDimensional eXpressions (MDX)
XML for Analysis (XMLA)
Business intelligence software
Comparison of OLAP Servers
Data warehousing products and their producers