A digital library, also called an online library, an internet library, a digital repository, or a digital collection is an
online database of digital objects that can include text, still images, audio, video,
digital document
An electronic document is any electronic media content (other than computer programs or system files) that is intended to be used in either an electronic form or as printed output. Originally, any computer data were considered as something intern ...
s, or other
digital media formats or a
library
A library is a collection of materials, books or media that are accessible for use and not just for display purposes. A library provides physical (hard copies) or digital access (soft copies) materials, and may be a physical location or a vir ...
accessible through the
internet
The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices. It is a '' network of networks'' that consists of private, pub ...
. Objects can consist of
digitized content like
print or
photographs
A photograph (also known as a photo, image, or picture) is an image created by light falling on a photosensitive surface, usually photographic film or an electronic image sensor, such as a CCD or a CMOS chip. Most photographs are now created ...
, as well as
originally produced digital content like
word processor
A word processor (WP) is a device or computer program that provides for input, editing, formatting, and output of text, often with some additional features.
Early word processors were stand-alone devices dedicated to the function, but current ...
files or
social media
Social media are interactive media technologies that facilitate the creation and sharing of information, ideas, interests, and other forms of expression through virtual communities and networks. While challenges to the definition of ''social medi ...
posts. In addition to storing content, digital libraries provide means for organizing, searching, and
retrieving the content contained in the collection. Digital libraries can vary immensely in size and scope, and can be maintained by individuals or organizations. The digital content may be stored locally, or accessed remotely via computer networks. These information retrieval systems are able to exchange information with each other through
interoperability
Interoperability is a characteristic of a product or system to work with other products or systems. While the term was initially defined for information technology or systems engineering services to allow for information exchange, a broader defi ...
and
sustainability.
History
The early history of digital libraries is not well documented, but several key thinkers are connected to the emergence of the concept. Predecessors include
Paul Otlet
Paul Marie Ghislain Otlet (; ; 23 August 1868 – 10 December 1944) was a Belgian author, entrepreneur, lawyer and peace activist; predicting the arrival of the internet before World War II, he is among those considered to be the father of infor ...
and
Henri La Fontaine
Henri La Fontaine (; 22 April 1854 – 14 May 1943), was a Belgian international lawyer and president of the International Peace Bureau. He received the Nobel Prize for Peace in 1913 because "he was the effective leader of the peace movement in ...
's
Mundaneum
The Mundaneum was an institution which aimed to gather together all the world's knowledge and classify it according to a system called the Universal Decimal Classification. It was developed at the turn of the 20th century by Belgian lawyers Paul ...
, an attempt begun in 1895 to gather and systematically catalogue the world's knowledge, with the hope of bringing about world peace. The visions of the digital library were largely realized a century later during the great expansion of the Internet, with access to the books and searching of the documents by millions of individuals on the World Wide Web.
Vannevar Bush
Vannevar Bush ( ; March 11, 1890 – June 28, 1974) was an American engineer, inventor and science administrator, who during World War II headed the U.S. Office of Scientific Research and Development (OSRD), through which almost all warti ...
and
J.C.R. Licklider are two contributors that advanced this idea into then current technology. Bush had supported research that led to the bomb that was dropped on
Hiroshima. After seeing the disaster, he wanted to create a machine that would show how technology can lead to understanding instead of destruction. This machine would include a desk with two screens, switches and buttons, and a keyboard. He named this the "
Memex
Memex is a hypothetical electromechanical device for interacting with microform documents and described in Vannevar Bush's 1945 article "As We May Think". Bush envisioned the memex as a device in which individuals would compress and store all of ...
". This way individuals would be able to access stored books and files at a rapid speed. In 1956,
Ford Foundation
The Ford Foundation is an American private foundation with the stated goal of advancing human welfare. Created in 1936 by Edsel Ford and his father Henry Ford, it was originally funded by a US$25,000 gift from Edsel Ford. By 1947, after the death ...
funded Licklider to analyze how libraries could be improved with technology. Almost a decade later, his book entitled "''Libraries of the Future''" included his vision. He wanted to create a system that would use computers and networks so human knowledge would be accessible for human needs and feedback would be automatic for machine purposes. This system contained three components, the corpus of knowledge, the question, and the answer. Licklider called it a procognitive system.
Early projects centered on the creation of an electronic card catalogue known as
Online Public Access Catalog
The online public access catalog (OPAC), now frequently synonymous with ''library catalog'', is an online database of materials held by a library or group of libraries. Online catalogs have largely replaced the analog card catalogs previously u ...
(OPAC). By the 1980s, the success of these endeavors resulted in OPAC replacing the traditional
card catalog
A library catalog (or library catalogue in British English) is a register of all bibliographic items found in a library or group of libraries, such as a network of libraries at several locations. A catalog for a group of libraries is also c ...
in many academic, public and special libraries. This permitted libraries to undertake additional rewarding co-operative efforts to support resource sharing and expand access to library materials beyond an individual library.
An early example of a digital library is the
Education Resources Information Center
The Education Resources Information Center (ERIC) is an online digital library of education research and information. ERIC is sponsored by the Institute of Education Sciences of the United States Department of Education.
Description
The missio ...
(ERIC), a database of education citations, abstracts and texts that was created in 1964 and made available online through
DIALOG
Dialog is an online information service owned by ProQuest, who acquired it from Thomson Reuters in mid-2008.
Dialog was one of the predecessors of the World Wide Web as a provider of information, though not in form. The earliest form of the Dial ...
in 1969.
In 1994, digital libraries became widely visible in the research community due to a $24.4 million
NSF managed program supported jointly by
DARPA
The Defense Advanced Research Projects Agency (DARPA) is a research and development agency of the United States Department of Defense responsible for the development of emerging technologies for use by the military.
Originally known as the Ad ...
's Intelligent Integration of Information (I3) program,
NASA
The National Aeronautics and Space Administration (NASA ) is an independent agencies of the United States government, independent agency of the US federal government responsible for the civil List of government space agencies, space program ...
, and NSF itself. Successful research proposals came from six U.S. universities. The universities included
Carnegie Mellon University,
University of California-Berkeley
The University of California, Berkeley (UC Berkeley, Berkeley, Cal, or California) is a public land-grant research university in Berkeley, California. Established in 1868 as the University of California, it is the state's first land-grant univ ...
,
University of Michigan
, mottoeng = "Arts, Knowledge, Truth"
, former_names = Catholepistemiad, or University of Michigania (1817–1821)
, budget = $10.3 billion (2021)
, endowment = $17 billion (2021)As o ...
,
University of Illinois
The University of Illinois Urbana-Champaign (U of I, Illinois, University of Illinois, or UIUC) is a public land-grant research university in Illinois in the twin cities of Champaign and Urbana. It is the flagship institution of the Univer ...
,
University of California-Santa Barbara
The University of California, Santa Barbara (UC Santa Barbara or UCSB) is a public land-grant research university in Santa Barbara, California with 23,196 undergraduates and 2,983 graduate students enrolled in 2021–2022. It is part of the Un ...
, and
Stanford University. Articles from the projects summarized their progress at their halfway point in May 1996. Stanford research, by
Sergey Brin
Sergey Mikhailovich Brin (russian: link=no, Сергей Михайлович Брин; born August 21, 1973) is an American business magnate, computer scientist, and internet entrepreneur, who co-founded Google with Larry Page. Brin was th ...
and
Larry Page
Lawrence Edward Page (born March 26, 1973) is an American business magnate, computer scientist and internet entrepreneur. He is best known for co-founding Google with Sergey Brin.
Page was the chief executive officer of Google from 1997 unti ...
, led to the founding of
Google
Google LLC () is an American Multinational corporation, multinational technology company focusing on Search Engine, search engine technology, online advertising, cloud computing, software, computer software, quantum computing, e-commerce, ar ...
.
Early attempts at creating a model for digital libraries included the DELOS ''Digital Library Reference Model''
[L. Candela et al.: ''The DELOS Digital Library Reference Model: Foundations for Digital Libraries''. Version 0.98, February 2008]
PDF
) and the 5S Framework.
Terminology
The term ''digital library'' was first popularized by the
NSF/
DARPA
The Defense Advanced Research Projects Agency (DARPA) is a research and development agency of the United States Department of Defense responsible for the development of emerging technologies for use by the military.
Originally known as the Ad ...
/
NASA
The National Aeronautics and Space Administration (NASA ) is an independent agencies of the United States government, independent agency of the US federal government responsible for the civil List of government space agencies, space program ...
Digital Libraries Initiative in 1994. With the availability of the computer networks the information resources are expected to stay distributed and accessed as needed, whereas in
Vannevar Bush
Vannevar Bush ( ; March 11, 1890 – June 28, 1974) was an American engineer, inventor and science administrator, who during World War II headed the U.S. Office of Scientific Research and Development (OSRD), through which almost all warti ...
's essay ''
As We May Think'' (1945) they were to be collected and kept within the researcher's
Memex
Memex is a hypothetical electromechanical device for interacting with microform documents and described in Vannevar Bush's 1945 article "As We May Think". Bush envisioned the memex as a device in which individuals would compress and store all of ...
.
The term ''virtual library'' was initially used interchangeably with ''digital library,'' but is now primarily used for libraries that are virtual in other senses (such as libraries which aggregate distributed content). In the early days of digital libraries, there was discussion of the similarities and differences among the terms ''digital'', ''virtual'', and ''electronic''.
A distinction is often made between content that was created in a digital format, known as
born-digital
The term born-digital refers to materials that originate in a digital form.NDIIPP"Preserving Digital Culture,"Library of Congress. This is in contrast to digital reformatting, through which analog materials become digital, as in the case of fil ...
, and information that has been converted from a physical medium, e.g. paper, through
digitization. Not all electronic content is in
digital data format. The term
hybrid library
Hybrid library is a term used by librarians to describe libraries containing a mix of traditional print library resources and the growing number of electronic resources.
Overview
Hybrid libraries are mixes of traditional print material such as bo ...
is sometimes used for libraries that have both physical collections and electronic collections. For example,
American Memory
American Memory is an internet-based archive for public domain image resources, as well as audio, video, and archived Web content. Published by the Library of Congress, the archive launched on October 13, 1994, after $13 million was raised in ...
is a digital library within the
Library of Congress
The Library of Congress (LOC) is the research library that officially serves the United States Congress and is the ''de facto'' national library of the United States. It is the oldest federal cultural institution in the country. The library ...
.
Some important digital libraries also serve as long term archives, such as
arXiv
arXiv (pronounced "archive"—the X represents the Greek letter chi ⟨χ⟩) is an open-access repository of electronic preprints and postprints (known as e-prints) approved for posting after moderation, but not peer review. It consists of ...
and the
Internet Archive
The Internet Archive is an American digital library with the stated mission of "universal access to all knowledge". It provides free public access to collections of digitized materials, including websites, software applications/games, music, ...
. Others, such as the
Digital Public Library of America
The Digital Public Library of America (DPLA) is a US project aimed at providing public access to digital holdings in order to create a large-scale public digital library. It officially launched on April 18, 2013, after two and a half years of dev ...
, seek to make digital information from various institutions widely accessible online.
Types of digital libraries
Institutional repositories
Many academic libraries are actively involved in building repositories of their institution's books, papers, theses, and other works that can be digitized or were 'born digital'. Many of these repositories are made available to the general public with few restrictions, in accordance with the goals of
open access, in contrast to the publication of research in commercial journals, where the publishers usually limit access rights. Irrespective of access rights, institutional, truly free, and corporate repositories can be referred to as digital libraries.
Institutional repository
An institutional repository is an archive for collecting, preserving, and disseminating digital copies of the intellectual output of an institution, particularly a research institution. Academics also utilize their IRs for archiving published work ...
software is designed for archiving, organizing, and searching a library's content. Popular open-source solutions include
DSpace
DSpace is an open source repository software package typically used for creating open access repositories for scholarly and/or published digital content. While DSpace shares some feature overlap with content management systems and document manag ...
,
Greenstone Digital Library (GSDL),
EPrints
EPrints is a free and open-source software package for building open access repositories that are compliant with the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). It shares many of the features commonly seen in document ...
,
Digital Commons
Digital Commons is a commercial, hosted institutional repository platform owned by RELX Group. This hosted service, licensed by bepress, is used by over 500 academic institutions, healthcare centers, public libraries, and research centers to show ...
, and the
Fedora Commons
Fedora (or Flexible Extensible Digital Object Repository Architecture) is a digital asset management (DAM) content repository architecture upon which institutional repositories, digital archives, and digital library systems might be built. Fedor ...
-based systems
Islandora and
Samvera
Samvera, originally known as Hydra, is an open-source digital repository software product. Samvera main components are Fedora Commons, Solr, Blacklight, and HydraHead (a Ruby on Rails plugin and gem, respectively). Each Samvera implementation is ...
.
National library collections
Legal deposit
Legal deposit is a legal requirement that a person or group submit copies of their publications to a repository, usually a library. The number of copies required varies from country to country. Typically, the national library is the primary reposi ...
is often covered by
copyright
A copyright is a type of intellectual property that gives its owner the exclusive right to copy, distribute, adapt, display, and perform a creative work, usually for a limited time. The creative work may be in a literary, artistic, educatio ...
legislation and sometimes by laws specific to legal deposit, and requires that one or more copies of all material published in a country should be submitted for preservation in an institution, typically the
national library
A national library is a library established by a government as a country's preeminent repository of information. Unlike public libraries, these rarely allow citizens to borrow books. Often, they include numerous rare, valuable, or significant wo ...
. Since the advent of
electronic document
An electronic document is any electronic media content (other than computer programs or system files) that is intended to be used in either an electronic form or as printed output. Originally, any computer data were considered as something inter ...
s, legislation has had to be amended to cover the new formats, such as the 2016 amendment to the ''
Copyright Act 1968
The copyright law of Australia defines the legally enforceable rights of creators of creative and artistic works under Australian law. The scope of copyright in Australia is defined in the '' Copyright Act 1968'' (as amended), which applies the ...
'' in Australia.
Since then various types of electronic depositories have been built. The
British Library
The British Library is the national library of the United Kingdom and is one of the largest libraries in the world. It is estimated to contain between 170 and 200 million items from many countries. As a legal deposit library, the British ...
's Publisher Submission Portal and the German model at the
Deutsche Nationalbibliothek
The German National Library (DNB; german: Deutsche Nationalbibliothek) is the central archival library and national bibliographic centre for the Federal Republic of Germany. It is one of the largest libraries in the world. Its task is to colle ...
have one deposit point for a network of libraries, but public access is only available in the reading rooms in the libraries. The Australian
National edeposit
National edeposit (NED) is a collaboration between Australia's nine national, state and territory libraries which provides for the legal deposit, management, storage and preservation of, and access to, published electronic material across Aus ...
system has the same features, but also allows for remote access by the general public for most of the content.
Digital archives
Physical
archives
An archive is an accumulation of historical records or materials – in any medium – or the physical facility in which they are located.
Archives contain primary source documents that have accumulated over the course of an individual or ...
differ from physical libraries in several ways. Traditionally, archives are defined as:
# Containing
primary source
In the study of history as an academic discipline, a primary source (also called an original source) is an artifact, document, diary, manuscript, autobiography, recording, or any other source of information that was created at the time under ...
s of information (typically letters and papers directly produced by an individual or organization) rather than the secondary sources found in a library (books, periodicals, etc.).
# Having their contents organized in groups rather than individual items.
# Having unique contents.
The technology used to create digital libraries is even more revolutionary for archives since it breaks down the second and third of these general rules. In other words, "digital archives" or "online archives" will still generally contain primary sources, but they are likely to be described individually rather than (or in addition to) in groups or collections. Further, because they are digital, their contents are easily reproducible and may indeed have been reproduced from elsewhere. The
Oxford Text Archive
Oxford Text Archive (OTA) is an archive of electronic texts and other literary and language resources which have been created, collected and distributed for the purpose of research into literary and linguistic topics at the University of Oxford, E ...
is generally considered to be the oldest digital archive of academic physical primary source materials.
Archives differ from libraries in the nature of the materials held. Libraries collect individual published books and serials, or bounded sets of individual items. The books and journals held by libraries are not unique, since multiple copies exist and any given copy will generally prove as satisfactory as any other copy. The material in archives and manuscript libraries are "the unique records of corporate bodies and the papers of individuals and families".
A fundamental characteristic of archives is that they have to keep the context in which their records have been created and the network of relationships between them in order to preserve their informative content and provide understandable and useful information over time. The fundamental characteristic of archives resides in their hierarchical organization expressing the context by means of the
archival bond The archival bond is a concept in archival theory
Archival science, or archival studies, is the study and theory of building and curating archives, which are collections of documents, recordings and data storage devices.
To build and curate a ...
.
Archival descriptions are the fundamental means to describe, understand, retrieve and access archival material. At the digital level, archival descriptions are usually encoded by means of the
Encoded Archival Description Encoded Archival Description (EAD) is a standard for encoding descriptive information regarding archival records.Pitti, D (2012). "Encoded Archival Description (EAD)." In Bates, Marcia J., (ed.) ''Understanding Information Retrieval Systems: Managem ...
XML format. The EAD is a standardized electronic representation of archival description which makes it possible to provide union access to detailed archival descriptions and resources in repositories distributed throughout the world.
Given the importance of archives, a dedicated formal model, called NEsted SeTs for Object Hierarchies (NESTOR), built around their peculiar constituents, has been defined. NESTOR is based on the idea of expressing the hierarchical relationships between objects through the inclusion property between sets, in contrast to the binary relation between nodes exploited by the tree. NESTOR has been used to formally extend the 5S model to define a digital archive as a specific case of digital library able to take into consideration the peculiar features of archives.
Features of digital libraries
The advantages of digital libraries as a means of easily and rapidly accessing books, archives and images of various types are now widely recognized by commercial interests and public bodies alike.
Traditional libraries are limited by storage space; digital libraries have the potential to store much more information, simply because digital information requires very little physical space to contain it. As such, the cost of maintaining a digital library can be much lower than that of a traditional library. A physical library must spend large sums of money paying for staff, book maintenance, rent, and additional books. Digital libraries may reduce or, in some instances, do away with these fees. Both types of library require cataloging input to allow users to locate and retrieve material. Digital libraries may be more willing to adopt innovations in technology providing users with improvements in electronic and audio book technology as well as presenting new forms of communication such as wikis and blogs; conventional libraries may consider that providing online access to their OP AC catalog is sufficient. An important advantage to digital conversion is increased accessibility to users. They also increase availability to individuals who may not be traditional patrons of a library, due to geographic location or organizational affiliation.
* No physical boundary: The user of a digital library need not to go to the library physically; people from all over the world can gain access to the same information, as long as an Internet connection is available.
* Round the clock availability: A major advantage of digital libraries is that people can gain access 24/7 to the information.
* Multiple access: The same resources can be used simultaneously by a number of institutions and patrons. This may not be the case for copyrighted material: a library may have a license for "lending out" only one copy at a time; this is achieved with a system of
digital rights management where a resource can become inaccessible after expiration of the lending period or after the lender chooses to make it inaccessible (equivalent to returning the resource).
* Information retrieval: The user is able to use any search term (word, phrase, title, name, subject) to search the entire collection. Digital libraries can provide very user-friendly interfaces, giving click able access to its resources.
* Preservation and conservation: Digitization is not a long-term preservation solution for physical collections, but does succeed in providing access copies for materials that would otherwise fall to degradation from repeated use. Digitized collections and born-digital objects pose many preservation and conservation concerns that analog materials do not. Please see the following "Problems" section of this page for examples.
* Space: Whereas traditional libraries are limited by storage space, digital libraries have the potential to store much more information, simply because digital information requires very little physical space to contain them and media storage technologies are more affordable than ever before.
* Added value: Certain characteristics of objects, primarily the quality of images, may be improved. Digitization can enhance legibility and remove visible flaws such as stains and discoloration.
Software
There are a number of software packages for use in general digital libraries (for notable ones see
:Digital library software). Institutional repository software, which focuses primarily on ingest, preservation and access of locally produced documents, particularly locally produced academic outputs, can be found in
Institutional repository software
Institutions are humanly devised structures of rules and norms that shape and constrain individual behavior. All definitions of institutions generally entail that there is a level of persistence and continuity. Laws, rules, social conventions a ...
. This software may be proprietary, as is the case with the Library of Congress which uses Digiboard and CTS to manage digital content.
The design and implementation in digital libraries are constructed so computer systems and software can make use of the information when it is exchanged. These are referred to as semantic digital libraries. Semantic libraries are also used to socialize with different communities from a mass of social networks. DjDL is a type of semantic digital library. Keywords-based and semantic search are the two main types of searches. A tool is provided in the semantic search that create a group for augmentation and refinement for keywords-based search. Conceptual knowledge used in DjDL is centered around two forms; the subject
ontology
In metaphysics, ontology is the philosophical study of being, as well as related concepts such as existence, becoming, and reality.
Ontology addresses questions like how entities are grouped into categories and which of these entities exi ...
and the set of
concept search A concept search (or conceptual search) is an automated information retrieval method that is used to search electronically stored unstructured data, unstructured text (for example, digital archives, email, scientific literature, etc.) for informatio ...
patterns based on the ontology. The three type of ontologies that are associated to this search are
bibliographic ontologies, community-aware ontologies, and subject ontologies.
Metadata
In traditional libraries, the ability to find works of interest is directly related to how well they were cataloged. While cataloging electronic works digitized from a library's existing holding may be as simple as copying or moving a record from the print to the electronic form, complex and born-digital works require substantially more effort. To handle the growing volume of electronic publications, new tools and technologies have to be designed to allow effective automated semantic classification and searching. While
full-text search
In text retrieval, full-text search refers to techniques for searching a single computer-stored document or a collection in a full-text database. Full-text search is distinguished from searches based on metadata or on parts of the original texts ...
can be used for some items, there are many common catalog searches which cannot be performed using full text, including:
* finding texts which are translations of other texts
* differentiating between editions/volumes of a text/periodical
* inconsistent descriptors (especially subject headings)
* missing, deficient or poor quality
taxonomy
Taxonomy is the practice and science of categorization or classification.
A taxonomy (or taxonomical classification) is a scheme of classification, especially a hierarchical classification, in which things are organized into groups or types. ...
practices
* linking texts published under pseudonyms to the real authors (
Samuel Clemens
Samuel Langhorne Clemens (November 30, 1835 – April 21, 1910), known by his pen name Mark Twain, was an American writer, humorist, entrepreneur, publisher, and lecturer. He was praised as the "greatest humorist the United States has p ...
and
Mark Twain, for example)
* differentiating non-fiction from parody (''
The Onion
''The Onion'' is an American digital media company and newspaper organization that publishes satire, satirical articles on international, national, and local news. The company is based in Chicago but originated as a weekly print publication on ...
'' from ''
The New York Times
''The New York Times'' (''the Times'', ''NYT'', or the Gray Lady) is a daily newspaper based in New York City with a worldwide readership reported in 2020 to comprise a declining 840,000 paid print subscribers, and a growing 6 million paid d ...
'')
Searching
Most digital libraries provide a search interface which allows resources to be found. These resources are typically
deep web
The deep web, invisible web, or hidden web are parts of the World Wide Web whose contents are not indexed by standard web search-engine programs. This is in contrast to the "surface web", which is accessible to anyone using the Internet. Co ...
(or invisible web) resources since they frequently cannot be located by
search engine crawlers. Some digital libraries create special pages or
sitemap
A sitemap is a list of pages of a web site within a domain.
There are three primary kinds of sitemap:
* Sitemaps used during the planning of a website by its designers.
* Human-visible listings, typically hierarchical, of the pages on a site.
* St ...
s to allow search engines to find all their resources. Digital libraries frequently use the
Open Archives Initiative Protocol for Metadata Harvesting The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a protocol developed for harvesting metadata descriptions of records in an archive so that services can be built using metadata from many archives. An implementation of OAI ...
(OAI-PMH) to expose their metadata to other digital libraries, and search engines like
Google Scholar
Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines. Released in beta in November 2004, the Google Scholar index includes ...
,
Yahoo!
Yahoo! (, styled yahoo''!'' in its logo) is an American web services provider. It is headquartered in Sunnyvale, California and operated by the namesake company Yahoo Inc., which is 90% owned by investment funds managed by Apollo Global Manage ...
and
Scirus can also use OAI-PMH to find these deep web resources.
There are two general strategies for searching a ''federation'' of digital libraries: distributed searching and searching previously harvested
metadata.
Distributed searching typically involves a client sending multiple search requests in parallel to a number of servers in the federation. The results are gathered, duplicates are eliminated or clustered, and the remaining items are sorted and presented back to the client. Protocols like
Z39.50 are frequently used in distributed searching. A benefit to this approach is that the resource-intensive tasks of indexing and storage are left to the respective servers in the federation. A drawback to this approach is that the search mechanism is limited by the different indexing and ranking capabilities of each database; therefore, making it difficult to assemble a combined result consisting of the most relevant found items.
Searching over previously harvested metadata involves searching a locally stored
index of information that has previously been collected from the libraries in the federation. When a search is performed, the search mechanism does not need to make connections with the digital libraries it is searching—it already has a local representation of the information. This approach requires the creation of an indexing and harvesting mechanism which operates regularly, connecting to all the digital libraries and querying the whole collection in order to discover new and updated resources.
OAI-PMH The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a protocol developed for harvesting metadata descriptions of records in an archive so that services can be built using metadata from many archives. An implementation of OAI- ...
is frequently used by digital libraries for allowing metadata to be harvested. A benefit to this approach is that the search mechanism has full control over indexing and ranking algorithms, possibly allowing more consistent results. A drawback is that harvesting and indexing systems are more resource-intensive and therefore expensive.
Digital preservation
Digital preservation aims to ensure that digital media and information systems are still interpretable into the indefinite future. Each necessary component of this must be migrated, preserved or
emulated
In computing, an emulator is Computer hardware, hardware or software that enables one computer system (called the ''host'') to behave like another computer system (called the ''guest''). An emulator typically enables the host system to run so ...
.
[Cain, Mark. "Managing Technology: Being a Library of Record in a Digital Age", Journal of Academic Librarianship 29:6 (2003).] Typically lower levels of systems (
floppy disks for example) are emulated, bit-streams (the actual files stored in the disks) are preserved and operating systems are emulated as a
virtual machine
In computing, a virtual machine (VM) is the virtualization/ emulation of a computer system. Virtual machines are based on computer architectures and provide functionality of a physical computer. Their implementations may involve specialized h ...
. Only where the meaning and content of digital media and information systems are well understood is migration possible, as is the case for office documents.
However, at least one organization, the Wider Net Project, has created an offline digital library, the
eGranary
The eGranary Digital Library is a product of the WiderNet Project, a non-profit aimed at providing access to information technology and library services in developing countries.
History
In 2000 Cliff Missen and Michael McNulty founded WiderNet ...
, by reproducing materials on a 6
TB hard drive
A hard disk drive (HDD), hard disk, hard drive, or fixed disk is an electro-mechanical data storage device that stores and retrieves digital data using magnetic storage with one or more rigid rapidly rotating platters coated with magne ...
. Instead of a bit-stream environment, the digital library contains a built-in
proxy server
In computer networking, a proxy server is a server application that acts as an intermediary between a client requesting a resource and the server providing that resource.
Instead of connecting directly to a server that can fulfill a reques ...
and
search engine so the digital materials can be accessed using an
Internet browser
A web browser is application software for accessing websites. When a user requests a web page from a particular website, the browser retrieves its files from a web server and then displays the page on the user's screen. Browsers are used on ...
.
Also, the materials are not preserved for the future. The eGranary is intended for use in places or situations where Internet connectivity is very slow, non-existent, unreliable, unsuitable or too expensive.
In the past few years, procedures for
digitizing
DigitizationTech Target. (2011, April). Definition: digitization. ''WhatIs.com''. Retrieved December 15, 2021, from https://whatis.techtarget.com/definition/digitization is the process of converting information into a digital (i.e. computer- ...
books at high speed and comparatively low cost have improved considerably with the result that it is now possible to digitize millions of books per year. The Google book-scanning project is also working with libraries to offer digitize books pushing forward on the digitize book realm.
Copyright and licensing
Digital libraries are hampered by
copyright
A copyright is a type of intellectual property that gives its owner the exclusive right to copy, distribute, adapt, display, and perform a creative work, usually for a limited time. The creative work may be in a literary, artistic, educatio ...
law because, unlike with traditional printed works, the laws of digital copyright are still being formed. The republication of material on the web by libraries may require permission from rights holders, and there is a conflict of interest between libraries and the publishers who may wish to create online versions of their acquired content for commercial purposes. In 2010, it was estimated that twenty-three percent of books in existence were created before 1923 and thus out of copyright. Of those printed after this date, only five percent were still in print as of 2010. Thus, approximately seventy-two percent of books were not available to the public.
There is a dilution of responsibility that occurs as a result of the distributed nature of digital resources. Complex intellectual property matters may become involved since digital material is not always owned by a library. The content is, in many cases,
public domain
The public domain (PD) consists of all the creative work to which no exclusive intellectual property rights apply. Those rights may have expired, been forfeited, expressly waived, or may be inapplicable. Because those rights have expired, ...
or self-generated content only. Some digital libraries, such as
Project Gutenberg
Project Gutenberg (PG) is a volunteer effort to digitize and archive cultural works, as well as to "encourage the creation and distribution of eBooks."
It was founded in 1971 by American writer Michael S. Hart and is the oldest digital libr ...
, work to digitize out-of-copyright works and make them freely available to the public. An estimate of the number of distinct books still existent in library catalogues from 2000 BC to 1960, has been made.
The
Fair Use
Fair use is a doctrine in United States law that permits limited use of copyrighted material without having to first acquire permission from the copyright holder. Fair use is one of the limitations to copyright intended to balance the interests ...
Provisions
(17 USC § 107) under the
Copyright Act of 1976 provide specific guidelines under which circumstances libraries are allowed to copy digital resources. Four factors that constitute fair use are "Purpose of the use, Nature of the work, Amount or substantiality used and Market impact".
Some digital libraries acquire a license to lend their resources. This may involve the restriction of lending out only one copy at a time for each license, and applying a system of
digital rights management for this purpose (see also above).
The
Digital Millennium Copyright Act
The Digital Millennium Copyright Act (DMCA) is a 1998 United States copyright law that implements two 1996 treaties of the World Intellectual Property Organization (WIPO). It criminalizes production and dissemination of technology, devices, or ...
of 1998 was an act created in the United States to attempt to deal with the introduction of digital works. This Act incorporates two treaties from the year 1996. It criminalizes the attempt to circumvent measures which limit access to copyrighted materials. It also criminalizes the act of attempting to circumvent access control.
[United States Copyright Office]
"The Digital Millennium Copyright Act of 1998 – U.S. Copyright Office Summary"
1998, 2. This act provides an exemption for nonprofit libraries and archives which allows up to three copies to be made, one of which may be digital. This may not be made public or distributed on the web, however. Further, it allows libraries and archives to copy a work if its format becomes obsolete.
[
Copyright issues persist. As such, proposals have been put forward suggesting that digital libraries be exempt from copyright law. Although this would be very beneficial to the public, it may have a negative economic effect and authors may be less inclined to create new works.
Another issue that complicates matters is the desire of some publishing houses to restrict the use of digit materials such as e-books purchased by libraries. Whereas with printed books, the library owns the book until it can no longer be circulated, publishers want to limit the number of times an e-book can be checked out before the library would need to repurchase that book. " arperCollinsbegan licensing use of each e-book copy for a maximum of 26 loans. This affects only the most popular titles and has no practical effect on others. After the limit is reached, the library can repurchase access rights at a lower cost than the original price." While from a publishing perspective, this sounds like a good balance of library lending and protecting themselves from a feared decrease in book sales, libraries are not set up to monitor their collections as such. They acknowledge the increased demand of digital materials available to patrons and the desire of a digital library to become expanded to include best sellers, but publisher licensing may hinder the process.
]
Recommendation systems
Many digital libraries offer recommender systems
A recommender system, or a recommendation system (sometimes replacing 'system' with a synonym such as platform or engine), is a subclass of information filtering system that provide suggestions for items that are most pertinent to a particular ...
to reduce information overload and help their users discovering relevant literature. Some examples of digital libraries offering recommender systems are IEEE Xplore
IEEE Xplore digital library is a research database for discovery and access to journal articles, conference proceedings, technical standards, and related materials on computer science, electrical engineering and electronics, and allied fields. It ...
, Europeana
Europeana is a web portal created by the European Union containing digitised cultural heritage collections of more than 3,000 institutions across Europe. It includes records of over 50 million cultural and scientific artefacts, brought togethe ...
, and GESIS Sowiport. The recommender systems work mostly based on content-based filtering but also other approaches are used such as collaborative filtering
Collaborative filtering (CF) is a technique used by recommender systems.Francesco Ricci and Lior Rokach and Bracha ShapiraIntroduction to Recommender Systems Handbook Recommender Systems Handbook, Springer, 2011, pp. 1-35 Collaborative filtering ...
and citation-based recommendations. Beel et al. report that there are more than 90 different recommendation approaches for digital libraries, presented in more than 200 research articles.
Typically, digital libraries develop and maintain their own recommender systems based on existing search and recommendation frameworks such as Apache Lucene
Apache Lucene is a free and open-source search engine software library, originally written in Java by Doug Cutting. It is supported by the Apache Software Foundation and is released under the Apache Software License. Lucene is widely used as a ...
or Apache Mahout
Apache Mahout is a project of the Apache Software Foundation to produce free implementations of distributed or otherwise scalable machine learning algorithms focused primarily on linear algebra. In the past, many of the implementations use the ...
. However, there are also some recommendation-as-a-service provider specializing in offering a recommender system for digital libraries as a service
As a service (AAS) is a business model in which something is being presented to a customer, either internal or external, as a service. As-a-Service offerings provide endpoints for customers/consumers to interface with which are usually API
A ...
.
Drawbacks of digital libraries
Digital libraries, or at least their digital collections, unfortunately also have brought their own problems and challenges in areas such as:
* User authentication for access to collections
* Copyright
A copyright is a type of intellectual property that gives its owner the exclusive right to copy, distribute, adapt, display, and perform a creative work, usually for a limited time. The creative work may be in a literary, artistic, educatio ...
* Digital preservation (see above)
* Equity of access (see digital divide)
* Interface design
User interface (UI) design or user interface engineering is the design of user interfaces for machines and software, such as computers, home appliances, mobile devices, and other electronic devices, with the focus on maximizing usability and t ...
* Interoperability
Interoperability is a characteristic of a product or system to work with other products or systems. While the term was initially defined for information technology or systems engineering services to allow for information exchange, a broader defi ...
between systems and software
* Information organization
Knowledge organization (KO), organization of knowledge, organization of information, or information organization is an intellectual discipline concerned with activities such as document description, indexing, and classification that serve to ...
* Inefficient or non-existent taxonomy
Taxonomy is the practice and science of categorization or classification.
A taxonomy (or taxonomical classification) is a scheme of classification, especially a hierarchical classification, in which things are organized into groups or types. ...
practices (especially with historical material)
* Training and development
Training and development involve improving the effectiveness of organizations and the individuals and teams within them. Training may be viewed as related to immediate changes in organizational effectiveness via organized instruction, while devel ...
* Quality of metadata
* Exorbitant cost of building/maintaining the terabytes of storage, servers, and redundancies necessary for a functional digital collection.
There are many large scale digitisation projects that perpetuate these problems.
Future development
Large scale digitization projects are underway at Google
Google LLC () is an American Multinational corporation, multinational technology company focusing on Search Engine, search engine technology, online advertising, cloud computing, software, computer software, quantum computing, e-commerce, ar ...
, the Million Book Project
The Million Book Project (or the Universal Library) was a book digitization project led by Carnegie Mellon University School of Computer Science and University Libraries from 2007–2008. Working with government and research partners in India ( D ...
, and Internet Archive
The Internet Archive is an American digital library with the stated mission of "universal access to all knowledge". It provides free public access to collections of digitized materials, including websites, software applications/games, music, ...
. With continued improvements in book handling and presentation technologies such as optical character recognition
Optical character recognition or optical character reader (OCR) is the electronic or mechanical conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document, a sc ...
and development of alternative depositories and business models, digital libraries are rapidly growing in popularity. Just as libraries have ventured into audio and video collections, so have digital libraries such as the Internet Archive. Google Books
Google Books (previously known as Google Book Search, Google Print, and by its code-name Project Ocean) is a service from Google Inc. that searches the full text of books and magazines that Google has scanned, converted to text using optical ...
project recently received a court victory on proceeding with their book-scanning project that was halted by the Authors' guild. This helped open the road for libraries to work with Google to better reach patrons who are accustomed to computerized information.
According to Larry Lannom, Director of Information Management Technology at the nonprofit Corporation for National Research Initiatives
The Corporation for National Research Initiatives (CNRI), based in Reston, Virginia, is a non-profit organization founded in 1986 by Robert E. Kahn as an "activities center around strategic development of network-based information technologies", in ...
(CNRI), "all the problems associated with digital libraries are wrapped up in archiving". He goes on to state, "If in 100 years people can still read your article, we'll have solved the problem." Daniel Akst, author of The Webster Chronicle, proposes that "the future of libraries—and of information—is digital". Peter Lyman and Hal Variant, information scientists at the University of California, Berkeley
The University of California, Berkeley (UC Berkeley, Berkeley, Cal, or California) is a public land-grant research university in Berkeley, California. Established in 1868 as the University of California, it is the state's first land-grant u ...
, estimate that "the world's total yearly production of print, film, optical, and magnetic content would require roughly 1.5 billion gigabytes of storage". Therefore, they believe that "soon it will be technologically possible for an average person to access virtually all recorded information".
Digital archives are an evolving medium and they develop under various circumstances. Alongside large scale repositories, other digital archiving projects have also evolved in response to needs in research and research communication on various institutional levels. For example, during the COVID-19 pandemic
The COVID-19 pandemic, also known as the coronavirus pandemic, is an ongoing global pandemic of coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The novel virus was first identi ...
, libraries
A library is a collection of Document, materials, books or media that are accessible for use and not just for display purposes. A library provides physical (hard copies) or electronic media, digital access (soft copies) materials, and may be a ...
and higher education institutions have launched digital archiving projects to document life during the pandemic, thus creating a digital, cultural record of collective memories from the period. Researchers have also utilized digital archiving to create specialized research databases. These databases compile digital records for use on international and interdisciplinary levels. COVID CORPUS, launched in October 2020, is an example of such a database, built in response to scientific communication needs in light of the pandemic. Beyond academia, digital collections have also recently been developed to appeal to a more general audience, as is the case with the Selected General Audience Content of the Internet-First University Press developed by Cornell University. This general-audience database contains specialized research information but is digitally organized for accessibility. The establishment of these archives has facilitated specialized forms of digital recordkeeping to fulfill various niches in online, research-based communication.
See also
* Anna's Archive
* Bibliographic database
A bibliographic database is a database of bibliographic records, an organized digital collection of references to published literature, including journal and newspaper articles, conference proceedings, reports, government and legal publications, ...
* Content repository A content repository or content store is a database of digital content with an associated set of data management, search and access methods allowing application-independent access to the content, rather like a digital library, but with the ability ...
* Digital Library Federation
The Digital Library Federation (DLF) is a program of the Council on Library and Information Resources (CLIR) that brings together a consortium of college and university libraries, public libraries, museums, and related institutions with the stated ...
* Digital Collections Selection Criteria
* Digitization
* '' D-Lib'', magazine about digital libraries
* Full-text database
A full-text database or a complete-text database is a database that contains the complete text of books, dissertations, journals, magazines, newspapers or other kinds of textual documents. They differ from bibliographic databases (which con ...
* List of digital library projects
* Mobile library
A bookmobile or mobile library is a vehicle designed for use as a library. They have been known by many names throughout history, including traveling library, library wagon, book wagon, book truck, library-on-wheels, and book auto service. Bookm ...
* Shadow library
Shadow libraries are online databases of readily available content that is normally obscured or otherwise not readily accessible. Such content may be inaccessible for a number of reasons, including the use of paywalls, copyright controls, or othe ...
* Traveling library A traveling library is a collection of books lent for stated periods by a central library to a branch library, club, or other organization or, in some instances, to an individual. The chief characteristics from which it derives its name are its temp ...
* Web archive
The Web ARChive (WARC) archive format specifies a method for combining multiple digital resources into an aggregate archive file together with related information. The WARC format is a revision of the Internet Archive's ARC_IA File Format that ...
References
Further reading
*
* Pomerantz, Jeffrey, and Gary Marchionini. 2007. "The Digital Library as Place". ''Journal of Documentation'' 63(4): 505–33.
External links
{{Authority control
Library science
Types of library