Beautiful Soup is a
Python
Python may refer to:
Snakes
* Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia
** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia
* Python (mythology), a mythical serpent
Computing
* Python (pro ...
package for parsing
HTML
The HyperText Markup Language or HTML is the standard markup language for documents designed to be displayed in a web browser. It can be assisted by technologies such as Cascading Style Sheets (CSS) and scripting languages such as JavaSc ...
and
XML
Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing arbitrary data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable ...
documents (including having malformed markup, i.e. non-closed tags, so named after
tag soup). It creates a parse tree for parsed pages that can be used to extract data from HTML, which is useful for
web scraping
Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. While web scrapin ...
.
Beautiful Soup was started by Leonard Richardson, who continues to contribute to the project, and is additionally supported by Tidelift, a paid subscription to open-source maintenance.
Code example
Beautiful Soup represents parsed data as a tree which can be searched and iterated over with ordinary Python
loops. The example below uses the Python
standard library's urllib to load
Wikipedia
Wikipedia is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system. Wikipedia is the largest and most-read refer ...
's main page, then uses Beautiful Soup to parse the document and search for all links within.
#!/usr/bin/env python3
# Anchor extraction from HTML document
from bs4 import BeautifulSoup
from urllib.request import urlopen
with urlopen('https://en.wikipedia.org/wiki/Main_Page') as response:
soup = BeautifulSoup(response, 'html.parser')
for anchor in soup.find_all('a'):
print(anchor.get('href', '/'))
Release
Beautiful Soup 3 was the official release line of Beautiful Soup from May 2006 to March 2012. The current release i
Beautiful Soup 4.x Beautiful Soup 4 can be installed with
pip install beautifulsoup4.
In 2021, Python 2.7 support was retired and the release 4.9.3 was the last to support Python 2.7.
See also
*
Comparison of HTML parsers
HTML parsers are software for automated Hypertext Markup Language (HTML) parsing. They have two main purposes:
* HTML traversal: offer an interface for programmers to easily access and modify the "HTML string code". Canonical example: DOM par ...
*
jsoup
*
Nokogiri
References
Python (programming language) libraries
Software using the MIT license
HTML parsers
XML parsers
{{compu-library-stub