Open-source intelligence (OSINT) is the collection and analysis of data gathered from open sources (covert and publicly available sources) to produce actionable intelligence. OSINT is primarily used in

national security National security, or national defence, is the security and defence of a sovereign state, including its citizens, economy, and institutions, which is regarded as a duty of government. Originally conceived as protection against military att ...

law enforcement Law enforcement is the activity of some members of government who act in an organized manner to enforce the law by discovering, deterring, rehabilitating, or punishing people who violate the rules and norms governing that society. The term ...

, and

business intelligence Business intelligence (BI) comprises the strategies and technologies used by enterprises for the data analysis and management of business information. Common functions of business intelligence technologies include reporting, online analytical ...

functions and is of value to analysts who use non-sensitive intelligence in answering

classified Classified may refer to: General *Classified information, material that a government body deems to be sensitive *Classified advertising or "classifieds" Music *Classified (rapper) (born 1977), Canadian rapper * The Classified, a 1980s American ro ...

unclassified Classified information is material that a government body deems to be sensitive information that must be protected. Access is restricted by law or regulation to particular groups of people with the necessary security clearance and need to know, ...

, or proprietary intelligence

requirements In product development and process optimization, a requirement is a singular documented physical or functional need that a particular design, product or process aims to satisfy. It is commonly used in a formal sense in engineering design, inclu ...

across the previous intelligence disciplines. OSINT sources can be divided up into six different categories of information flow: *Media, print

newspapers A newspaper is a periodical publication containing written information about current events and is often typed in black ink with a white or gray background. Newspapers can cover a wide variety of fields such as politics, business, spor ...

magazines A magazine is a periodical publication, generally published on a regular schedule (often weekly or monthly), containing a variety of content. They are generally financed by advertising, purchase price, prepaid subscriptions, or by a combination ...

radio Radio is the technology of signaling and communicating using radio waves. Radio waves are electromagnetic waves of frequency between 30 hertz (Hz) and 300 gigahertz (GHz). They are generated by an electronic device called a tr ...

, and

television Television, sometimes shortened to TV, is a telecommunication medium for transmitting moving images and sound. The term can refer to a television set, or the medium of television transmission. Television is a mass medium for advertising, ...

from across and between countries. *

Internet The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices. It is a '' network of networks'' that consists of private, p ...

online publication Electronic publishing (also referred to as publishing, digital publishing, or online publishing) includes the digital publication of e-books, digital magazines, and the development of digital libraries and catalogues. It also includes the editi ...

blogs A blog (a truncation of "weblog") is a discussion or informational website published on the World Wide Web consisting of discrete, often informal diary-style text entries (posts). Posts are typically displayed in reverse chronological order ...

, discussion groups, citizen media (i.e. – cell phone

videos Video is an electronic medium for the recording, copying, playback, broadcasting, and display of moving visual media. Video was first developed for mechanical television systems, which were quickly replaced by cathode-ray tube (CRT) system ...

, and

user created content User-generated content (UGC), alternatively known as user-created content (UCC), is any form of content, such as images, videos, text, testimonials, and audio, that has been posted by users on online platforms such as social media, discussion f ...

YouTube YouTube is a global online video sharing and social media platform headquartered in San Bruno, California. It was launched on February 14, 2005, by Steve Chen, Chad Hurley, and Jawed Karim. It is owned by Google, and is the second mo ...

, and other

social media Social media are interactive media technologies that facilitate the creation and sharing of information, ideas, interests, and other forms of expression through virtual communities and networks. While challenges to the definition of ''social me ...

websites (i.e. –

Facebook Facebook is an online social media and social networking service owned by American company Meta Platforms. Founded in 2004 by Mark Zuckerberg with fellow Harvard College students and roommates Eduardo Saverin, Andrew McCollum, Dust ...

Twitter Twitter is an online social media and social networking service owned and operated by American company Twitter, Inc., on which users post and interact with 280-character-long messages known as "tweets". Registered users can post, like, and ...

, Instagram, etc.). This source also outpaces a variety of other sources due to its timeliness and ease of access. *Public government data, public government reports, budgets, hearings,

telephone directories A telephone directory, commonly called a telephone book, telephone address book, phonebook, or the white and yellow pages, is a listing of telephone subscribers in a geographical area or subscribers to services provided by the organization that ...

, press conferences, websites, and speeches. Although this source comes from an official source they are publicly accessible and may be used openly and freely. *Professional and academic publications, information acquired from journals, conferences, symposia,

academic papers Academic publishing is the subfield of publishing which distributes academic research and scholarship. Most academic work is published in academic journal articles, books or theses. The part of academic written output that is not formally publ ...

, dissertations, and theses. *Commercial data, commercial imagery, financial and industrial assessments, and databases. *

Grey literature Grey literature (or gray literature) is materials and research produced by organizations outside of the traditional commercial or academic publishing and distribution channels. Common grey literature publication types include reports ( annual, r ...

, technical reports,

preprint In academic publishing, a preprint is a version of a scholarly or scientific paper that precedes formal peer review and publication in a peer-reviewed scholarly or scientific journal. The preprint may be available, often as a non-typeset versi ...

s, patents, working papers, business documents, unpublished works, and

newsletter A newsletter is a printed or electronic report containing news concerning the activities of a business or an organization that is sent to its members, customers, employees or other subscribers. Newsletters generally contain one main topic of ...

s. OSINT is distinguished from research in that it applies the

process of intelligence Military intelligence is a military discipline that uses information collection and analysis approaches to provide guidance and direction to assist commanders in their decisions. This aim is achieved by providing an assessment of data from a ...

to create tailored knowledge supportive of a specific decision by a specific individual or group.

Definition

OSINT is defined in the United States of America by Public Law 109-163 as cited by both the U.S.

Director of National Intelligence The director of national intelligence (DNI) is a senior, cabinet-level United States government official, required by the Intelligence Reform and Terrorism Prevention Act of 2004 to serve as executive head of the United States Intelligence Co ...

and the U.S.

Department of Defense Department of Defence or Department of Defense may refer to: Current departments of defence * Department of Defence (Australia) * Department of National Defence (Canada) * Department of Defence (Ireland) * Department of National Defense (Philipp ...

(DoD), as intelligence "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement." As defined by NATO, OSINT is intelligence "derived from publicly available information, as well as other unclassified information that has limited public distribution or access." According to political scientist Jeffrey T. Richelson, “open source acquisition involves procuring verbal, written, or electronically transmitted material that can be obtained legally. In addition to documents and videos available via the

or provided by a human source, others are obtained after U.S. or allied forces have taken control of a facility or site formerly operated by a foreign government or

terrorist Terrorism, in its broadest sense, is the use of criminal violence to provoke a state of terror or fear, mostly with the intention to achieve political or religious aims. The term is used in this regard primarily to refer to intentional violen ...

group.” Former Assistant Director of Central Intelligence for Analysis Mark M. Lowenthal defines OSINT as “any and all information that can be derived from overt collection: all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, the Internet, and so on. The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable."

History

OSINT in the

United States The United States of America (U.S.A. or USA), commonly known as the United States (U.S. or US) or America, is a country Continental United States, primarily located in North America. It consists of 50 U.S. state, states, a Washington, D.C., ...

traces its origins to the creation of the Foreign Broadcast Monitoring Service (FBMS), an agency responsible for the monitoring of foreign broadcasts. An example of their work is reflected in the application of the correlation of changes in the price of oranges in Paris with that of railway bridges being bombed successfully. The Aspin-Brown Commission stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and

DCI DCI may be an abbreviation for: Technology * D-chiro-inositol, an isomer of inositol * Data, context and interaction, an architectural pattern in computer software development * Direct Count & Intersect, an algorithm for discovering frequent se ...

attention. In July 2004, following the

September 11 attacks The September 11 attacks, commonly known as 9/11, were four coordinated suicide terrorist attacks carried out by al-Qaeda against the United States on Tuesday, September 11, 2001. That morning, nineteen terrorists hijacked four commer ...

, the

9/11 Commission The National Commission on Terrorist Attacks Upon the United States, also known as the 9/11 Commission, was set up on November 27, 2002, "to prepare a full and complete account of the circumstances surrounding the September 11 attacks", includin ...

recommended the creation of an open-source intelligence agency. In March 2005, the Iraq Intelligence Commission recommended the creation of an open-source directorate at the CIA. Following these recommendations, in November 2005 the

announced the creation of the DNI Open Source Center. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery." In addition to collecting openly available information, it would train analysts to make better use of this information. The center absorbed the

CIA The Central Intelligence Agency (CIA ), known informally as the Agency and historically as the Company, is a civilian foreign intelligence service of the federal government of the United States, officially tasked with gathering, processing, ...

's previously existing

Foreign Broadcast Information Service The Foreign Broadcast Information Service (FBIS) was an open source intelligence component of the Central Intelligence Agency's Directorate of Science and Technology. It monitored, translated, and disseminated within the U.S. government openly ...

(FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the center. Then, following the events of

9/11 The September 11 attacks, commonly known as 9/11, were four coordinated suicide terrorist attacks carried out by al-Qaeda against the United States on Tuesday, September 11, 2001. That morning, nineteen terrorists hijacked four commerci ...

the Intelligence Reform and Terrorism Prevention Act merged FBIS and other research elements into the

Office of the Director of National Intelligence The director of national intelligence (DNI) is a senior, cabinet-level United States government official, required by the Intelligence Reform and Terrorism Prevention Act of 2004 to serve as executive head of the United States Intelligence Comm ...

creating the Open Source Enterprise. Furthermore, the private sector has invested in tools which aid in OSINT collection and analysis. Specifically, In-Q-Tel, a

Central Intelligence Agency The Central Intelligence Agency (CIA ), known informally as the Agency and historically as the Company, is a civilian foreign intelligence service of the federal government of the United States, officially tasked with gathering, processing, ...

supported venture capital firm in Arlington, VA assisted companies develop web-monitoring and predictive analysis tools. In December 2005, the Director of National Intelligence appointed Eliot A. Jardines as the Assistant Deputy Director of National Intelligence for Open Source to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the

National Open Source Enterprise The Open Source Enterprise (OSE) is a United States Government organization dedicated to open-source intelligence that was established by Assistant Deputy Director of National Intelligence for Open Source (ADDNI/OS), Eliot A. Jardines. They prov ...

. Mr. Jardines has established the National Open Source Enterprise and authored intelligence community directive 301. In 2008, Mr. Jardines returned to the private sector and was succeeded by

Dan Butler Daniel Eugene Butler (born December 2, 1954) is an American actor known for his role as Bob "Bulldog" Briscoe on the TV series '' Frasier'' (1993–2004); Art in ''Roseanne'' (1991–1992); for the voice of Mr. Simmons on the ''Nickelodeon'' T ...

who is ADDNI/OS and previously Mr. Jardines' Senior Advisor for Policy.

Tools

The web browser is a powerful OSINT tool that provides access to numerous websites and both open source and proprietary software tools that are either purpose-built for open source information collection or which can be exploited for the purposes of either gathering of open source information or to facilitate analysis and validation to provide intelligence. A cottage industry of both for-profit and not-for-profit investigative and educational groups such as Bellingcat, IntelTechniques SANS and others offer indices, books, podcasts and video training materials on OSINT tools and techniques. Books such as Michael Bazzell's ''Open Source Intelligence Techniques'' serve as indices to resources across multiple domains but according the author, due to the rapidly changing information landscape, some tools and techniques change or become obsolete frequently, hence it is imperative for OSINT researchers to study, train and survey the landscape of source material regularly. A guide by Ryan Fedasiuk, an analyst at the

Center for Security and Emerging Technology The Center for Security and Emerging Technology (CSET) is a think tank dedicated to policy analysis at the intersection of national and international security and emerging technologies, based at Georgetown University's School of Foreign Service. ...

, lists six tools open-source analysts can use to stay safe and utlize operational security (

OPSEC Operations security (OPSEC) is a process that identifies critical information to determine if friendly actions can be observed by enemy intelligence, determines if information obtained by adversaries could be interpreted to be useful to them, a ...

) when conducting online investigations. These include

VPNs A virtual private network (VPN) extends a private network across a public network and enables users to send and receive data across shared or public networks as if their computing devices were directly connected to the private network. The be ...

, cached webpages,

digital archive An archive is an accumulation of historical records or materials – in any medium – or the physical facility in which they are located. Archives contain primary source documents that have accumulated over the course of an individual or ...

services, URL and file scanners, browser sandbox applications, and

antivirus software Antivirus software (abbreviated to AV software), also known as anti-malware, is a computer program used to prevent, detect, and remove malware. Antivirus software was originally developed to detect and remove computer viruses, hence the name. ...

. Numerous lists of aggregated OSINT content are available on the web. The OSINT Framework contains over 30 primary categories of tools and is maintained as an open source project on

GitHub GitHub, Inc. () is an Internet hosting service for software development and version control using Git. It provides the distributed version control of Git plus access control, bug tracking, software feature requests, task management, cont ...

Risks for practitioners

A main hindrance to practical OSINT is the volume of information it has to deal with ("

information explosion The information explosion is the rapid increase in the amount of published information or data and the effects of this abundance. As the amount of available data grows, the problem of managing the information becomes more difficult, which can lead ...

"). The amount of data being distributed increases at a rate that it becomes difficult to evaluate sources in intelligence analysis. To a small degree the work has sometimes been done by amateur crowd-sourcing. Accredited journalists have some protection in asking questions, and researching for recognized media outlets. Even so, they can be imprisoned, even executed, for seeking out OSINT. Private individuals illegally collecting data for a foreign military or intelligence agency is considered

espionage Espionage, spying, or intelligence gathering is the act of obtaining secret or confidential information ( intelligence) from non-disclosed sources or divulging of the same without the permission of the holder of the information for a tang ...

in most countries. Of course, espionage that is not

treason Treason is the crime of attacking a state authority to which one owes allegiance. This typically includes acts such as participating in a war against one's native country, attempting to overthrow its government, spying on its military, its diplo ...

(i.e. betraying one's country of citizenship) has been a tool of statecraft since ancient times.

Professional Association

The OSINT Foundation is a professional association for OSINT practitioners in the United States Intelligence Community. It is open to U.S. Citizens and seeks to raise the prominence of the open-source intelligence discipline.

References

WashTimes.com
Washington Times – CIA mines 'rich' content from blogs, 19 April 2006

Government Computer News – Intelligence units mine the benefits of public sources 20 March 2006
AFCEA.org
SIGNAL Magazine – Intelligence Center Mines Open Sources March 2006
FindAcricles.com
Military Intelligence Professional Bulletin October–December, 2005 by Barbara G. Fast
FAS.org
Congressional Testimony on OSINT and Homeland Security 21 June 2005
FirstMonday.org
Open Source Intelligence by Stalder and Hirsh, 15 May 2002

When Everyone Can Mine Your Data by Taylor Buley, 11.21.08] *
Cnet.com
Maltego and the science of 'open-source' snooping by Matt Asay, November 25, 2008

Literature

;Scientific Publications * Arthur S. Hulnick:
The Dilemma of Open Source Intelligence: Is OSINT Really Intelligence?
, pages 229–241, The Oxford Handbook of National Security Intelligence, 2010
Cody Burke: 'Freeing knowledge, telling secrets: Open source intelligence and development', Bond University, May 2007

Florian Schaurer, Jan Störger: 'The Evolution of Open Source Intelligence', OSINT Report 3/2010, ISN, ETH Zürich, October 2010

Abdelrahman Rashdan: 'The Social Media OSINT Challenge to US Intelligence: Culture Not Gigabytes', in New Media Politics Rethinking Activism and National Security in Cyberspace. Ed. Banu Baybars-Hawks. Cambridge Scholars Publishing, 2015
*

External links

A Reverse Search Engine for OSINTHide and Seek : How to find anyone online (and hide)An OSINT search of historic domain name whois recordsThe Open Source Intelligence Resource Discovery Toolkit
* ttps://web.archive.org/web/20080216224135/http://www.cm2limited.com/casestudies/casestudies.php Actual Intelligence Case Studies Leveraging Open Source Intelligence (OSINT)br>Sailing the Sea of OSINT in the Information Age
* *
Open Source Center
– U.S. government arm focusing on open source intelligence under the DNI *
Open Source Intelligence (OSINT): Issues for Congress
Congressional Research Service, December 5, 2007
Open Source Intelligence (OSINT): Issues for Congress
Congressional Research Service, January 28, 2008
The Free Library
FMSO-JRIC and Open Source Intelligence: speaking prose in a world of verse, Military Intelligence Professional Bulletin, Oct–Dec, 2005 by Jacob W. Kipp {{DEFAULTSORT:Open Source Intelligence Applied data mining Intelligence gathering disciplines Collective intelligence American inventions 2005 introductions 2005 establishments in the United States

Definition

History

Tools

Risks for practitioners

Professional Association

See also

References

Literature

External links