CGI Script

In computing, Common Gateway Interface (CGI) is an interface specification that enables web servers to execute an external program, typically to process user requests.

Such programs are often written in a scripting language and are commonly referred to as ''CGI scripts'', but they may include compiled programs.

A typical use case occurs when a web user submits a web form on a web page that uses CGI. The form's data is sent to the web server within an HTTP request with a URL denoting a CGI script. The web server then launches the CGI script in a new computer process, passing the form data to it. The output of the CGI script, usually in the form of HTML, is returned by the script to the Web server, and the server relays it back to the browser as its response to the browser's request.

Developed in the early 1990s, CGI was the earliest common method available that allowed a web page to be interactive.

History

In 1993, the National Center for Supercomputing Applications (NCSA) team wrote the specification for calling command line executables on the www-talk mailing list. The other Web server developers adopted it, and it has been a standard for Web servers ever since. A work group chaired by Ken Coar started in November 1997 to get the NCSA definition of CGI more formally defined. This work resulted in RFC 3875, which specified CGI Version 1.1. Specifically mentioned in the RFC are the following contributors:
* Rob McCool (author of the NCSA HTTPd Web server)
* John Franks (author of the GN Web server)
* Ari Luotonen (the developer of the CERN httpd Web server)
* Tony Sanders (author of the Plexus Web server)
* George Phillips (Web server maintainer at the University of British Columbia)

Historically CGI programs were often written using the C programming language . RFC 3875 "The Common Gateway Interface (CGI)" partially defines CGI using C, in saying that environment variables "are accessed by the C library routine getenv() or variable environ".

The name CGI comes from the early days of the Web, where '' webmasters'' wanted to connect legacy information systems such as databases to their Web servers. The CGI program was executed by the server that provided a common "gateway" between the Web server and the legacy information system.

Purpose of the CGI specification

Each Web server runs HTTP server software, which responds to requests from web browsers. Generally, the HTTP server has a directory (folder), which is designated as a document collection – files that can be sent to Web browsers connected to this server. For example, if the Web server has the domain name example.com, and its document collection is stored at /usr/local/apache/htdocs/ in the local file system, then the Web server will respond to a request for http://example.com/index.html by sending to the browser the (pre-written) file /usr/local/apache/htdocs/index.html.

For pages constructed on the fly, the server software may defer requests to separate programs and relay the results to the requesting client (usually, a Web browser that displays the page to the end user). In the early days of the Web, such programs were usually small and written in a scripting language; hence, they were known as ''scripts''.

Such programs usually require some additional information to be specified with the request. For instance, if Wikipedia were implemented as a script, one thing the script would need to know is whether the user is logged in and, if logged in, under which name. The content at the top of a Wikipedia page depends on this information.

HTTP provides ways for browsers to pass such information to the Web server, e.g. as part of the URL. The server software must then pass this information through to the script somehow.

Conversely, upon returning, the script must provide all the information required by HTTP for a response to the request: the HTTP status of the request, the document content (if available), the document type (e.g. HTML, PDF, or plain text), et cetera.

Initially, different server software would use different ways to exchange this information with scripts. As a result, it wasn't possible to write scripts that would work unmodified for different server software, even though the information being exchanged was the same. Therefore, it was decided to specify a way for exchanging this information: CGI (the ''Common Gateway Interface'', as it defines a common way for server software to interface with scripts).
Webpage generating programs invoked by server software that operate according to the CGI specification are known as ''CGI scripts''.

This specification was quickly adopted and is still supported by all well-known server software, such as Apache, IIS, and (with an extension) node.js-based servers.

An early use of CGI scripts was to process forms. In the beginning of HTML, HTML forms typically had an "action" attribute and a button designated as the "submit" button. When the submit button is pushed the URI specified in the "action" attribute would be sent to the server with the data from the form sent as a query string. If the "action" specifies a CGI script then the CGI script would be executed and it then produces an HTML page.

Using CGI scripts

A Web server allows its owner to configure which URLs shall be handled by which CGI scripts.

This is usually done by marking a new directory within the document collection as containing CGI scripts – its name is often cgi-bin. For example, /usr/local/apache/htdocs/cgi-bin could be designated as a CGI directory on the Web server. When a Web browser requests a URL that points to a file within the CGI directory (e.g., http://example.com/cgi-bin/printenv.pl/with/additional/path?and=a&query=string), then, instead of simply sending that file (/usr/local/apache/htdocs/cgi-bin/printenv.pl) to the Web browser, the HTTP server runs the specified script and passes the output of the script to the Web browser. That is, anything that the script sends to standard output is passed to the Web client instead of being shown on-screen in a terminal window.

As remarked above, the CGI specification defines how additional information passed with the request is passed to the script.
For instance, if a slash and additional directory name(s) are appended to the URL immediately after the name of the script (in this example, /with/additional/path), then that path is stored in the PATH_INFO environment variable before the script is called. If parameters are sent to the script via an HTTP GET request (a question mark appended to the URL, followed by param=value pairs; in the example, ?and=a&query=string), then those parameters are stored in the QUERY_STRING environment variable before the script is called. If parameters are sent to the script via an HTTP POST request, they are passed to the script's standard input. The script can then read these environment variables or data from standard input and adapt to the Web browser's request.

Example

The following Perl program shows all the environment variables passed by the Web server:

#!/usr/bin/env perl

=head1 DESCRIPTION

printenv — a CGI program that just prints its environment

=cut
print "Content-Type: text/plain\n\n";

foreach ( sort keys %ENV )

If a Web browser issues a request for the environment variables at http://example.com/cgi-bin/printenv.pl/foo/bar?var1=value1&var2=with%20percent%20encoding, a 64-bit Windows 7 Web server running cygwin returns the following information:

Some, but not all, of these variables are defined by the CGI standard.
Some, such as PATH_INFO, QUERY_STRING, and the ones starting with HTTP_, pass information along from the HTTP request.

From the environment, it can be seen that the Web browser is Firefox running on a Windows 7 PC, the Web server is Apache running on a system that emulates Unix, and the CGI script is named cgi-bin/printenv.pl.

The program could then generate any content, write that to standard output, and the Web server will transmit it to the browser.

The following are environment variables passed to CGI programs:

* Server specific variables:
** SERVER_SOFTWARE: name/version of HTTP server.
** SERVER_NAME: host name of the server, may be dot-decimal IP address.
** GATEWAY_INTERFACE: CGI/version.
* Request specific variables:
** SERVER_PROTOCOL: HTTP/version.
** SERVER_PORT: TCP port (decimal).
** REQUEST_METHOD: name of HTTP method (see above).
** PATH_INFO: path suffix, if appended to URL after program name and a slash.
** PATH_TRANSLATED: corresponding full path as supposed by server, if PATH_INFO is present.
** SCRIPT_NAME: relative path to the program, like /cgi-bin/script.cgi.
** QUERY_STRING: the part of URL after ? character. The query string may be composed of *name=value pairs separated with ampersands (such as var1=val1&var2=val2...) when used to submit form data transferred via GET method as defined by HTML application/x-www-form-urlencoded.
** REMOTE_HOST: host name of the client, unset if server did not perform such lookup.
** REMOTE_ADDR: IP address of the client (dot-decimal).
** AUTH_TYPE: identification type, if applicable.
** REMOTE_USER used for certain AUTH_TYPEs.
** REMOTE_IDENT: see ident, only if server performed such lookup.
** CONTENT_TYPE: Internet media type of input data if PUT or POST method are used, as provided via HTTP header.
** CONTENT_LENGTH: similarly, size of input data (decimal, in octets) if provided via HTTP header.
** Variables passed by user agent (HTTP_ACCEPT, HTTP_ACCEPT_LANGUAGE, HTTP_USER_AGENT, HTTP_COOKIE and possibly others) contain values of corresponding HTTP headers and therefore have the same sense.

The program returns the result to the Web server in the form of standard output, beginning with a header and a blank line.

The header is encoded in the same way as an HTTP header and must include the MIME type of the document returned. The headers, supplemented by the Web server, are generally forwarded with the response back to the user.

Here is a simple CGI program written in Python 3 along with the HTML that handles a simple addition problem.

add.html:

Enter two numbers to add

First Number:

Second Number:

Add

add.cgi:

#!/usr/bin/env python3

import cgi, cgitb
cgitb.enable()

input_data = cgi.FieldStorage()

print('Content-Type: text/html') # HTML is following
print('') # Leave a blank line
print('

Addition Results

')
try:
num1 = int(input_data num1"value)
num2 = int(input_data num2"value)
except:
print('

Sorry, the script cannot turn your inputs into numbers (integers).

')
raise SystemExit(1)
print('

+ =

'.format(num1, num2, num1 + num2))
This Python 3 CGI program gets the inputs from the HTML and adds the two numbers together.

Deployment

A Web server that supports CGI can be configured to interpret a URL that it serves as a reference to a CGI script. A common convention is to have a cgi-bin/ directory at the base of the directory tree and treat all executable files within this directory (and no other, for security) as CGI scripts. Another popular convention is to use filename extensions; for instance, if CGI scripts are consistently given the extension .cgi, the Web server can be configured to interpret all such files as CGI scripts. While convenient, and required by many prepackaged scripts, it opens the server to attack if a remote user can upload executable code with the proper extension.

In the case of HTTP PUT or POSTs, the user-submitted data are provided to the program via the standard input. The Web server creates a subset of the environment variables passed to it and adds details pertinent to the HTTP environment.

Uses

CGI is often used to process input information from the user and produce the appropriate output. An example of a CGI program is one implementing a wiki. If the user agent requests the name of an entry, the Web server executes the CGI program. The CGI program retrieves the source of that entry's page (if one exists), transforms it into HTML, and prints the result. The Web server receives the output from the CGI program and transmits it to the user agent. Then if the user agent clicks the "Edit page" button, the CGI program populates an HTML textarea or other editing control with the page's contents. Finally if the user agent clicks the "Publish page" button, the CGI program transforms the updated HTML into the source of that entry's page and saves it.

Security

CGI programs run, by default, in the security context of the Web server. When first introduced a number of example scripts were provided with the reference distributions of the NCSA, Apache and CERN Web servers to show how shell scripts or C programs could be coded to make use of the new CGI. One such example script was a CGI program called PHF that implemented a simple phone book.

In common with a number of other scripts at the time, this script made use of a function: escape_shell_cmd(). The function was supposed to sanitize its argument, which came from user input and then pass the input to the Unix shell, to be run in the security context of the Web server. The script did not correctly sanitize all input and allowed new lines to be passed to the shell, which effectively allowed multiple commands to be run. The results of these commands were then displayed on the Web server. If the security context of the Web server allowed it, malicious commands could be executed by attackers.

This was the first widespread example of a new type of Web based attack, where unsanitized data from Web users could lead to execution of code on a Web server. Because the example code was installed by default, attacks were widespread and led to a number of security advisories in early 1996.

Alternatives

For each incoming HTTP request, a Web server creates a new CGI process for handling it and destroys the CGI process after the HTTP request has been handled. Creating and destroying a process can consume much more CPU and memory than the actual work of generating the output of the process, especially when the CGI program still needs to be interpreted by a virtual machine. For a high number of HTTP requests, the resulting workload can quickly overwhelm the Web server.

The overhead involved in CGI process creation and destruction can be reduced by the following techniques:

* CGI programs precompiled to machine code, e.g. precompiled from C or C++ programs, rather than CGI programs interpreted by a virtual machine, e.g. Perl, PHP or Python programs.
* Web server extensions such as Apache modules (e.g. mod_perl, mod_php, mod_python), NSAPI plugins, and ISAPI plugins which allow long-running application processes handling more than one request and hosted within the Web server. Web 2.0 allows to transfer data from the client to the server without using HTML forms and without the user noticing.
* FastCGI, SCGI, and AJP which allow long-running application processes handling more than one request to be hosted externally; i.e., separately from the Web server. Each application process listens on a socket; the Web server handles an HTTP request and sends it via another protocol (FastCGI, SCGI or AJP) to the socket only for dynamic content, while static content is usually handled directly by the Web server. This approach needs fewer application processes so consumes less memory than the Web server extension approach. And unlike converting an application program to a Web server extension, FastCGI, SCGI, and AJP application programs remain independent of the Web server.
* Jakarta EE runs Jakarta Servlet applications in a Web container to serve dynamic content and optionally static content which replaces the overhead of creating and destroying processes with the much lower overhead of creating and destroying threads. It also exposes the programmer to the library that comes with Java SE on which the version of Jakarta EE in use is based.

The optimal configuration for any Web application depends on application-specific details, amount of traffic, and complexity of the transaction; these trade-offs need to be analyzed to determine the best implementation for a given task and time budget. Web frameworks offer an alternative to using CGI scripts to interact with user agents.

See also

* CGI.pm
* DOS Gateway Interface (DGI)
* FastCGI
* Perl Web Server Gateway Interface
* Rack (web server interface)
* Server Side Includes
* Web Server Gateway Interface

References

External links

GNU cgicc
a C++ class library for writing CGI applications

CGI
a standard Perl module for CGI request parsing and HTML response generation

CGI Programming 101: Learn CGI Today!
a CGI tutorial

The Invention of CGI

{{Authority control

Servers (computing)
Web 1.0
Web technology
Network protocols
Articles with example Python (programming language) code