HOME

TheInfoList



OR:

Scientometrics is a subfield of informetrics that studies quantitative aspects of scholarly literature. Major research issues include the measurement of the impact of research papers and academic journals, the understanding of scientific citations, and the use of such measurements in policy and management contexts. Leydesdorff, L. and Milojevic, S., "Scientometrics
arXiv:1208.4566
(2013), forthcoming in: Lynch, M. (editor), ''International Encyclopedia of Social and Behavioral Sciences'' subsection 85030. (2015)
In practice there is a significant overlap between scientometrics and other scientific fields such as
information system An information system (IS) is a formal, sociotechnical, organizational system designed to collect, process, Information Processing and Management, store, and information distribution, distribute information. From a sociotechnical perspective, info ...
s, information science, science of science policy, sociology of science, and
metascience Metascience (also known as meta-research) is the use of scientific methodology to study science itself. Metascience seeks to increase the quality of scientific research while reducing inefficiency. It is also known as "research on research" and ...
. Critics have argued that overreliance on scientometrics has created a system of perverse incentives, producing a publish or perish environment that leads to low-quality research.


Historical development

Modern scientometrics is mostly based on the work of
Derek J. de Solla Price Derek John de Solla Price (22 January 1922 – 3 September 1983) was a British physicist, history of science, historian of science, and information science, information scientist. He was known for his investigation of the Antikythera mechanism, ...
and
Eugene Garfield Eugene Eli Garfield (September 16, 1925 – February 26, 2017) was an American linguistics, linguist and businessman, one of the founders of bibliometrics and scientometrics. He helped to create ''Current Contents'', ''Science Citation Index'' ( ...
. The latter created the
Science Citation Index The Science Citation Index Expanded (SCIE) is a citation index owned by Clarivate and previously by Thomson Reuters. It was created by the Eugene Garfield at the Institute for Scientific Information, launched in 1964 as Science Citation Index ( ...
and founded the
Institute for Scientific Information The Institute for Scientific Information (ISI) was an academic publishing service, founded by Eugene Garfield in Philadelphia in 1956. ISI offered scientometric and bibliographic database services. Its specialty was citation indexing and analysis, ...
which is heavily used for scientometric analysis. A dedicated academic journal, ''Scientometrics'', was established in 1978. The industrialization of science increased the number of publications and research outcomes and the rise of the computers allowed effective analysis of this data. While the sociology of science focused on the behavior of scientists, scientometrics focused on the analysis of
publications To publish is to make Content (media), content available to the general public.Berne Conv ...
. Accordingly, scientometrics is also referred to as the scientific and empirical study of science and its outcomes.Lowry, Paul Benjamin; Moody, Gregory D.; Gaskin, James; Galletta, Dennis F.; Humpherys, Sean; Barlow, Jordan B.; and Wilson, David W. (2013).
Evaluating journal quality and the Association for Information Systems (AIS) Senior Scholars' journal basket via bibliometric measures: Do expert journal assessments add value?
" MIS Quarterly (MISQ), vol. 37(4), 993–1012. Also, see a YouTube video narrative of this paper at: https://www.youtube.com/watch?v=LZQIDkA-ke0.
The International Society for Scientometrics and Informetrics founded in 1993 is an association of professionals in the field. Later, around the turn of the century, evaluation and ranking of scientists and institutions came more into the spotlights. Based on bibliometric analysis of scientific publications and citations, the
Academic Ranking of World Universities The ''Academic Ranking of World Universities'' (''ARWU''), also known as the Shanghai Ranking, is one of the annual publications of world university rankings. The league table was originally compiled and issued by Shanghai Jiao Tong Universi ...
("Shanghai ranking") was first published in 2004 by the Shanghai Jiao Tong University.
Impact factor The impact factor (IF) or journal impact factor (JIF) of an academic journal is a type of journal ranking. Journals with higher impact factor values are considered more prestigious or important within their field. The Impact Factor of a journa ...
s became an important tool to choose between different journals. Rankings such as the Academic Ranking of World Universities and the
Times Higher Education World University Rankings The ''Times Higher Education World University Rankings'', often referred to as the THE Rankings, is the annual publication of university rankings by the ''Times Higher Education'' magazine. The publisher had collaborated with Quacquarelli Symon ...
(THE-ranking) became an indicator for the status of universities. The ''h''-index became an important indicator of the productivity and impact of the work of a scientist. However, alternative author-level metrics have been proposed. Around the same time, the interest of governments in evaluating research for the purpose of assessing the impact of science funding increased. As the investments in scientific research were included as part of the U.S.
American Recovery and Reinvestment Act of 2009 The American Recovery and Reinvestment Act of 2009 (ARRA) (), nicknamed the Recovery Act, was a Stimulus (economics), stimulus package enacted by the 111th U.S. Congress and signed into law by President Barack Obama in February 2009. Developed ...
(ARRA), a major economic stimulus package, programs like STAR METRICS were set up to assess if the positive impact on the economy would actually occur.


Methods and findings

Methods of research include qualitative, quantitative and computational approaches. The main focus of studies have been on institutional productivity comparisons, institutional research rankings, journal rankings establishing faculty productivity and tenure standards, assessing the influence of top scholarly articles, and developing profiles of top authors and institutions in terms of research performance. One significant finding in the field is a principle of cost escalation to the effect that achieving further findings at a given level of importance grow exponentially more costly in the expenditure of effort and resources. However, new algorithmic methods in search,
machine learning Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of Computational statistics, statistical algorithms that can learn from data and generalise to unseen data, and thus perform Task ( ...
and
data mining Data mining is the process of extracting and finding patterns in massive data sets involving methods at the intersection of machine learning, statistics, and database systems. Data mining is an interdisciplinary subfield of computer science and ...
are showing that is not the case for many information retrieval and extraction-based problems. More recent methods rely on
open source Open source is source code that is made freely available for possible modification and redistribution. Products include permission to use and view the source code, design documents, or content of the product. The open source model is a decentrali ...
and
open data Open data are data that are openly accessible, exploitable, editable and shareable by anyone for any purpose. Open data are generally licensed under an open license. The goals of the open data movement are similar to those of other "open(-so ...
to ensure transparency and reproducibility in line with modern
open science Open science is the movement to make scientific research (including publications, data, physical samples, and software) and its dissemination accessible to all levels of society, amateur or professional. Open science is transparent and accessib ...
requirements. For instance, the
Unpaywall OurResearch, formerly known as ImpactStory, is a nonprofit organization that creates and distributes tools and services for libraries, institutions and researchers. The organization follows open practices with their data (to the extent allowed b ...
index and attendant research on
open access Open access (OA) is a set of principles and a range of practices through which nominally copyrightable publications are delivered to readers free of access charges or other barriers. With open access strictly defined (according to the 2001 de ...
trends is based on data retrieved from OAI-PMH endpoints of thousands of
open archive An open repository or open-access repository is a digital platform that holds research output and provides free, immediate and permanent access to research results for anyone to use, download and distribute. To facilitate open access such reposito ...
s provided by libraries and institutions worldwide. Recommendations to avoid common errors in scientometrics include: select topics with sufficient data; use data mining and web scraping, combine methods, and eliminate "false positives". It is also necessary to understand the limits of search engines (e.g. Web of Science, Scopus and Google Scholar) which fail to index thousands of studies in small journals and underdeveloped countries.


Common scientometric indexes

Indexes may be classified as article-level metrics, author-level metrics, and journal-level metrics depending on which feature they evaluate.


Impact factor

The impact factor (IF) or journal impact factor (JIF) of an
academic journal An academic journal (or scholarly journal or scientific journal) is a periodical publication in which Scholarly method, scholarship relating to a particular academic discipline is published. They serve as permanent and transparent forums for the ...
is a measure reflecting the yearly average number of
citation A citation is a reference to a source. More precisely, a citation is an abbreviated alphanumeric expression embedded in the body of an intellectual work that denotes an entry in the bibliographic references section of the work for the purpose o ...
s to recent articles published in that journal. It is frequently used as a proxy for the relative importance of a journal within its field; journals with higher impact factors are often deemed to be more important than those with lower ones. The impact factor was devised by
Eugene Garfield Eugene Eli Garfield (September 16, 1925 – February 26, 2017) was an American linguistics, linguist and businessman, one of the founders of bibliometrics and scientometrics. He helped to create ''Current Contents'', ''Science Citation Index'' ( ...
, the founder of the
Institute for Scientific Information The Institute for Scientific Information (ISI) was an academic publishing service, founded by Eugene Garfield in Philadelphia in 1956. ISI offered scientometric and bibliographic database services. Its specialty was citation indexing and analysis, ...
(ISI).


Science Citation Index

The Science Citation Index (SCI) is a
citation index A citation index is a kind of bibliographic index, an index of citations between publications, allowing the user to easily establish which later documents cite which earlier documents. A form of citation index is first found in 12th-century H ...
originally produced by the
Institute for Scientific Information The Institute for Scientific Information (ISI) was an academic publishing service, founded by Eugene Garfield in Philadelphia in 1956. ISI offered scientometric and bibliographic database services. Its specialty was citation indexing and analysis, ...
(ISI) and created by
Eugene Garfield Eugene Eli Garfield (September 16, 1925 – February 26, 2017) was an American linguistics, linguist and businessman, one of the founders of bibliometrics and scientometrics. He helped to create ''Current Contents'', ''Science Citation Index'' ( ...
. It was officially launched in 1964. It is now owned by
Clarivate Analytics Clarivate Plc is a British-American publicly traded analytics company that operates a collection of subscription-based services, in the areas of bibliometrics and scientometrics; business and market intelligence, and competitive profiling ...
(previously the Intellectual Property and Science business of
Thomson Reuters Thomson Reuters Corporation ( ) is a Canadian multinational corporation, multinational content-driven technology Conglomerate (company), conglomerate. The company was founded in Toronto, Ontario, Canada, and maintains its headquarters at 1 ...
). The larger version (Science Citation Index Expanded) covers more than 8,500 notable and significant journals, across 150 disciplines, from 1900 to the present. These are alternatively described as the world's leading journals of
science Science is a systematic discipline that builds and organises knowledge in the form of testable hypotheses and predictions about the universe. Modern science is typically divided into twoor threemajor branches: the natural sciences, which stu ...
and
technology Technology is the application of Conceptual model, conceptual knowledge to achieve practical goals, especially in a reproducible way. The word ''technology'' can also mean the products resulting from such efforts, including both tangible too ...
, because of a rigorous selection process.


Acknowledgment index

An acknowledgment index (British acknowledgement index) is a method for indexing and analyzing acknowledgments in the
scientific literature Scientific literature encompasses a vast body of academic papers that spans various disciplines within the natural and social sciences. It primarily consists of academic papers that present original empirical research and theoretical ...
and, thus, quantifies the impact of acknowledgments. Typically, a scholarly article has a section in which the authors acknowledge entities such as funding, technical staff, colleagues, etc. that have contributed materials or knowledge or have influenced or inspired their work. Like a
citation index A citation index is a kind of bibliographic index, an index of citations between publications, allowing the user to easily establish which later documents cite which earlier documents. A form of citation index is first found in 12th-century H ...
, it measures influences on scientific work, but in a different sense; it measures institutional and economic influences as well as informal influences of individual people, ideas, and artifacts. Unlike the impact factor, it does not produce a single overall metric, but analyzes the components separately. However, the total number of acknowledgments to an acknowledged entity can be measured and so can the number of citations to the papers in which the acknowledgment appears. The ratio of this total number of citations to the total number of papers in which the acknowledge entity appears can be construed as the impact of that acknowledged entity.


Altmetrics

In scholarly and scientific publishing, altmetrics are nontraditional
bibliometrics Bibliometrics is the application of statistical methods to the study of bibliographic data, especially in scientific and library and information science contexts, and is closely associated with scientometrics (the analysis of scientific metri ...
proposed as an alternative or complement to more traditional
citation impact Citation impact or citation rate is a measure of how many times an academic journal article or book or author is cited by other articles, books or authors. Citation counts are interpreted as measures of the impact or influence of academic work a ...
metrics, such as
impact factor The impact factor (IF) or journal impact factor (JIF) of an academic journal is a type of journal ranking. Journals with higher impact factor values are considered more prestigious or important within their field. The Impact Factor of a journa ...
and ''h''-index. The term altmetrics was proposed in 2010, as a generalization of article level metrics, and has its roots in the #altmetrics hashtag. Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc. Altmetrics use public APIs across platforms to gather data with open scripts and algorithms. Altmetrics did not originally cover
citation A citation is a reference to a source. More precisely, a citation is an abbreviated alphanumeric expression embedded in the body of an intellectual work that denotes an entry in the bibliographic references section of the work for the purpose o ...
counts, but calculate scholar impact based on diverse online research output, such as social media, online news media, online reference managers and so on. It demonstrates both the impact and the detailed composition of the impact. Altmetrics could be applied to research filter, promotion and tenure dossiers, grant applications and for ranking newly published articles in academic search engines.


Criticisms

Critics have argued that overreliance on scientometrics has created a publish or perish environment with perverse incentives that lead to low-quality research.


In popular culture

The main character in Michael Frayn’s novel Skios is a Professor of Scientometrics.


See also

* Academic careerism * Author-level metrics * Citation analysis * College and university rankings * Erdős number * Eigenfactor * Expert elicitation * Goodhart's law * Journal ranking *
Journalology Journalology (also known as publication science) is the scholarly study of all aspects of the academic publishing process. The field seeks to improve the quality of scholarly research by implementing evidence-based practices in academic publishing. ...
*
Lists of science and technology awards This is a list of notable awards for specific areas of science and technology. Typically these lists give the country of the sponsoring organization, the award name, sponsor name and a description of the award criteria. Some of the awards have bro ...
*
Peer review Peer review is the evaluation of work by one or more people with similar competencies as the producers of the work (:wiktionary:peer#Etymology 2, peers). It functions as a form of self-regulation by qualified members of a profession within the ...
* *
SCImago Journal Rank The SCImago Journal Rank (SJR) indicator is a measure of the prestige of scholarly journals that accounts for both the number of citations received by a journal and the prestige of the journals where the citations come from. Etymology SCImago ...
*
Scopus Scopus is a scientific abstract and citation database, launched by the academic publisher Elsevier as a competitor to older Web of Science in 2004. The ensuing competition between the two databases has been characterized as "intense" and is c ...
* Research Integrity Risk Index * Semantic Scholar * Science of science


Journals

* '' Scientometrics'' * '' Journal of the American Society for Information Science and Technology'' * '' Journal of Informetrics''


References and footnotes


External links

* * *
The Places & Spaces: Mapping Science
' exhibit at the American Museum of Science and Energy, September 7, 2007 – January 7, 2008.
Over-optimization of academic publishing metrics: observing Goodhart's Law in action
GigaScience, Volume 8, Issue 6, June 2019 {{Authority control Bibliometrics Science policy Library science Metascience