![Deleted comments](https://upload.wikimedia.org/wikipedia/commons/5/56/Deleted_comments.png)
On
Internet
The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices. It is a '' network of networks'' that consists of private, pub ...
websites
A website (also written as a web site) is a collection of web pages and related content that is identified by a common domain name and published on at least one web server. Examples of notable websites are Google, Facebook, Amazon, and Wikipe ...
that invite users to post comments, content moderation is the process of detecting contributions that are irrelevant, obscene, illegal, harmful, or insulting with regards to useful or informative contributions. The purpose of content moderation is to remove or apply a warning label to problematic content or allow users to
block
Block or blocked may refer to:
Arts, entertainment and media Broadcasting
* Block programming, the result of a programming strategy in broadcasting
* W242BX, a radio station licensed to Greenville, South Carolina, United States known as ''96.3 ...
and filter content themselves.
Various types of Internet sites permit
user-generated content
User-generated content (UGC), alternatively known as user-created content (UCC), is any form of content, such as images, videos, text, testimonials, and audio, that has been posted by users on online platforms such as social media, discussion f ...
such as comments, including
Internet forum
An Internet forum, or message board, is an online discussion site where people can hold conversations in the form of posted messages. They differ from chat rooms in that messages are often longer than one line of text, and are at least temporar ...
s,
blog
A blog (a truncation of "weblog") is a discussion or informational website published on the World Wide Web consisting of discrete, often informal diary-style text entries (posts). Posts are typically displayed in reverse chronological order ...
s, and news sites powered by scripts such as
phpBB
phpBB is an Internet forum package written in the PHP scripting language. The name "phpBB" is an abbreviation of ''PHP Bulletin Board''. Available under the GNU General Public License, phpBB is free and open-source.
Features of phpBB include s ...
, a
Wiki
A wiki ( ) is an online hypertext publication collaboratively edited and managed by its own audience, using a web browser. A typical wiki contains multiple pages for the subjects or scope of the project, and could be either open to the pu ...
, or
PHP-Nuke
''PHP-Nuke'' is a web-based automated news publishing and content management system based on PHP and MySQL originally written by Francisco Burzi. The system is controlled using a web-based user interface. PHP-Nuke was originally a fork of the Th ...
. Depending on the site's content and intended audience, the site's administrators will decide what kinds of user comments are appropriate, then delegate the responsibility of sifting through comments to lesser
moderators. Most often, they will attempt to eliminate
trolling
In slang, a troll is a person who posts or makes inflammatory, insincere, digressive, extraneous, or off-topic messages online (such as in social media, a newsgroup, a forum, a chat room, a online video game), or in real life, with the i ...
,
spamming
Spamming is the use of messaging systems to send multiple unsolicited messages (spam) to large numbers of recipients for the purpose of commercial advertising, for the purpose of non-commercial proselytizing, for any prohibited purpose (especial ...
, or
flaming, although this varies widely from site to site.
Major platforms use a combination of algorithmic tools, user reporting and human review.
Social media sites may also employ content moderators to manually inspect or remove content flagged for
hate speech
Hate speech is defined by the ''Cambridge Dictionary'' as "public speech that expresses hate or encourages violence towards a person or group based on something such as race, religion, sex, or sexual orientation". Hate speech is "usually thoug ...
or other objectionable content. Other content issues include
revenge porn,
graphic content
Graphic violence refers to the depiction of especially vivid, brutal and realistic acts of violence in visual media such as film, television, and video games. It may be real, simulated live action, or animated.
Intended limitedly for mature ...
,
child abuse material and
propaganda
Propaganda is communication that is primarily used to influence or persuade an audience to further an agenda, which may not be objective and may be selectively presenting facts to encourage a particular synthesis or perception, or using loaded ...
.
Some websites must also make their content hospitable to advertisements.
Supervisor moderation
Also known as unilateral moderation, this kind of moderation system is often seen on
Internet forum
An Internet forum, or message board, is an online discussion site where people can hold conversations in the form of posted messages. They differ from chat rooms in that messages are often longer than one line of text, and are at least temporar ...
s. A group of people are chosen by the site's administrators (usually on a long-term basis) to act as delegates, enforcing the community rules on their behalf. These
moderators are given special privileges to delete or edit others' contributions and/or exclude people based on their
e-mail address
An email address identifies an email box to which messages are delivered. While early messaging systems used a variety of formats for addressing, today, email addresses follow a set of specific rules originally standardized by the Internet Engineer ...
or
IP address
An Internet Protocol address (IP address) is a numerical label such as that is connected to a computer network that uses the Internet Protocol for communication.. Updated by . An IP address serves two main functions: network interface ident ...
, and generally attempt to remove negative contributions throughout the community. They act as an invisible backbone, underpinning the
social web
The social web is a set of social relations that link people through the World Wide Web. The social web encompasses how websites and software are designed and developed in order to support and foster social interaction. These online social inte ...
in a crucial but undervalued role.
In the case of
Facebook
Facebook is an online social media and social networking service owned by American company Meta Platforms. Founded in 2004 by Mark Zuckerberg with fellow Harvard College students and roommates Eduardo Saverin, Andrew McCollum, Dustin M ...
, the company has increased the number of content moderators from 4,500 to 7,500 in 2017 due to legal and other
controversies
Controversy is a state of prolonged public dispute or debate, usually concerning a matter of conflicting opinion or point of view. The word was coined from the Latin ''controversia'', as a composite of ''controversus'' – "turned in an opposite d ...
. In Germany, Facebook is responsible for removing hate speech within 24 hours of when it is posted.
Social media site Twitter
has a suspension policy. Between August 2015 and December 2017 it suspended over 1.2 million accounts for terrorist content in an effort to reduce the number of followers and amount of content associated with the Islamic State.
Commercial content moderation (CCM)
Commercial Content Moderation is a term coined by
Sarah T. Roberts
Sarah T. Roberts (born September 2, 1975) is a professor, author, and scholar who specializes in content moderation of social media. She is an expert in the areas of internet culture, social media, digital labor, and the intersections of media an ...
to describe the practice of "monitoring and vetting
user-generated content
User-generated content (UGC), alternatively known as user-created content (UCC), is any form of content, such as images, videos, text, testimonials, and audio, that has been posted by users on online platforms such as social media, discussion f ...
(UGC) for
social media
Social media are interactive media technologies that facilitate the creation and sharing of information, ideas, interests, and other forms of expression through virtual communities and networks. While challenges to the definition of ''social medi ...
platforms of all types, in order to ensure that the content complies with legal and regulatory exigencies, site/community guidelines, user agreements, and that it falls within norms of taste and acceptability for that site and its cultural context."
While at one time this work may have been done by volunteers within the online community, for commercial websites this is largely achieved through
outsourcing
Outsourcing is an agreement in which one company hires another company to be responsible for a planned or existing activity which otherwise is or could be carried out internally, i.e. in-house, and sometimes involves transferring employees and ...
the task to specialized companies, often in low-wage areas such as India and the Philippines. Outsourcing of content moderation jobs grew as a result of the
social media boom. With the overwhelming growth of users and UGC, companies needed many more employees to moderate the content. In the late 1980s and early 1990s, tech companies began to outsource jobs to foreign countries that had an educated workforce but were willing to work for cheap.
Employees work by viewing, assessing and deleting disturbing content, and may suffer psychological damage.
Secondary trauma may arise, with symptoms similar to
PTSD
Post-traumatic stress disorder (PTSD) is a mental and behavioral disorder that can develop because of exposure to a traumatic event, such as sexual assault, warfare, traffic collisions, child abuse, domestic violence, or other threats on a ...
.
Some large companies such as Facebook offer psychological support
and increasingly rely on the use of
Artificial Intelligence (AI)
Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech re ...
to sort out the most graphic and inappropriate content, but critics claim that it is insufficient.
Facebook
Facebook has decided to create an oversight board that will decide what content remains and what content is removed. This idea was proposed in late 2018. The "Supreme Court" at Facebook is to replace making decisions in an ad hoc manner.
Distributed moderation
Distributed moderation comes in two types: user moderation and spontaneous moderation.
User moderation
User moderation allows any user to moderate any other user's contributions. Billions of people are currently making decisions on what to share, forward or give visibility to on a daily basis.
On a large site with a sufficiently large active population, this usually works well, since relatively small numbers of troublemakers are screened out by the votes of the rest of the community. Strictly speaking,
wiki
A wiki ( ) is an online hypertext publication collaboratively edited and managed by its own audience, using a web browser. A typical wiki contains multiple pages for the subjects or scope of the project, and could be either open to the pu ...
s such as
Wikipedia
Wikipedia is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system. Wikipedia is the largest and most-read refer ...
are the ultimate in user moderation, but in the context of
Internet forum
An Internet forum, or message board, is an online discussion site where people can hold conversations in the form of posted messages. They differ from chat rooms in that messages are often longer than one line of text, and are at least temporar ...
s, the definitive example of a user moderation system is
Slashdot
''Slashdot'' (sometimes abbreviated as ''/.'') is a social news website that originally advertised itself as "News for Nerds. Stuff that Matters". It features news stories concerning science, technology, and politics that are submitted and evalu ...
.
For example, each moderator is given a limited number of "mod points," each of which can be used to moderate an individual comment up or down by one point. Comments thus accumulate a score, which is additionally bounded to the range of -1 to 5 points. When viewing the site, a threshold can be chosen from the same scale, and only posts meeting or exceeding that threshold will be displayed. This system is further refined by the concept of
karma
Karma (; sa, कर्म}, ; pi, kamma, italic=yes) in Sanskrit means an action, work, or deed, and its effect or consequences. In Indian religions, the term more specifically refers to a principle of cause and effect, often descriptivel ...
—the ratings assigned to a user's' previous contributions can bias the initial rating of contributions he or she makes.
On sufficiently specialized websites, user moderation will often lead to
groupthink
Groupthink is a psychological phenomenon that occurs within a group of people in which the desire for harmony or conformity in the group results in an irrational or dysfunctional decision-making outcome. Cohesiveness, or the desire for cohesiveness ...
, in which any opinion that is in disagreement with the website's established principles (no matter how sound or well-phrased) will very likely be "modded down" and censored, leading to the perpetuation of the groupthink mentality. This is often confused with
trolling
In slang, a troll is a person who posts or makes inflammatory, insincere, digressive, extraneous, or off-topic messages online (such as in social media, a newsgroup, a forum, a chat room, a online video game), or in real life, with the i ...
.
User moderation can also be characterized by reactive moderation. This type of moderation depends on users of a platform or site to report content that is inappropriate and breaches
community standards. In this process, when users are faced with an image or video they deem unfit, they can click the report button. The complaint is filed and queued for moderators to look at.
[Grimes-Viort, Blaise (December 7, 2010). "6 types of content moderation you need to know about". ''Social Media Today''.]
Spontaneous moderation
Spontaneous moderation is what occurs when no official moderation scheme exists. Without any ability to moderate comments, users will spontaneously moderate their peers through posting their own comments about others' comments. Because spontaneous moderation exists, no system that allows users to submit their own content can ever go completely without any kind of moderation.
See also
*
Like button
A like button, like option, or recommend button, is a feature in communication software such as social networking services, Internet forums, news websites and blogs where the user can express that they like, enjoy or support certain content. In ...
*
Meta-moderation system
*
Recommender system
A recommender system, or a recommendation system (sometimes replacing 'system' with a synonym such as platform or engine), is a subclass of information filtering system that provide suggestions for items that are most pertinent to a particular u ...
*
Trust metric
In psychology and sociology, a trust metric is a measurement or metric of the degree to which one social actor (an individual or a group) trusts another social actor. Trust metrics may be abstracted in a manner that can be implemented on compu ...
* ''
We Had to Remove This Post''
References
{{Reflist, 35em
External links
Slashdot– A definitive example of user moderation
Fundamental Basics of Content Moderation
* Cliff Lampe and Paul Resnick
Slash (dot) and burn: distributed moderation in a large online conversation spaceProceedings of the SIGCHI conference on Human factors in computing systems table of contents, Vienna, Austria 2005, 543–550.
*Hamed Alhoori, Omar Alvarez, Richard Furuta, Miguel Muñiz, Eduardo Urbina
Supporting the Creation of Scholarly Bibliographies by Communities through Online Reputation Based Social Collaboration.ECDL 2009: 180–191
Internet forum terminology
Internet culture
Reputation management
*