The theory of statistics provides a basis for the whole range of techniques, in both
study design
Clinical study design is the formulation of trials and experiments, as well as observational studies in medical, clinical and other types of research (e.g., epidemiological) involving human beings. The goal of a clinical study is to assess the saf ...
and
data analysis
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making. Data analysis has multiple facets and approaches, enco ...
, that are used within applications of
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
.
[
] The theory covers approaches to
statistical-decision problems and to
statistical inference
Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution, distribution of probability.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical ...
, and the actions and deductions that satisfy the basic principles stated for these different approaches. Within a given approach, statistical theory gives ways of comparing statistical procedures; it can find a best possible procedure within a given context for given statistical problems, or can provide guidance on the choice between alternative procedures.
Apart from philosophical considerations about how to make statistical inferences and decisions, much of statistical theory consists of
mathematical statistics
Mathematical statistics is the application of probability theory, a branch of mathematics, to statistics, as opposed to techniques for collecting statistical data. Specific mathematical techniques which are used for this include mathematical an ...
, and is closely linked to
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
, to
utility theory
As a topic of economics, utility is used to model worth or value. Its usage has evolved significantly over time. The term was introduced initially as a measure of pleasure or happiness as part of the theory of utilitarianism by moral philosopher ...
, and to
optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
.
Scope
Statistical theory provides an underlying rationale and provides a consistent basis for the choice of methodology used in
applied statistics
Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industri ...
.
Modelling
Statistical model
A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of Sample (statistics), sample data (and similar data from a larger Statistical population, population). A statistical model repres ...
s describe the sources of data and can have different types of formulation corresponding to these sources and to the problem being studied. Such problems can be of various kinds:
*
Sampling from a finite population
*Measuring
observational error
Observational error (or measurement error) is the difference between a measured value of a quantity and its true value.Dodge, Y. (2003) ''The Oxford Dictionary of Statistical Terms'', OUP. In statistics, an error is not necessarily a " mistake ...
and refining procedures
*Studying statistical
relations
Statistical models, once specified, can be tested to see whether they provide useful inferences for new data sets.
Data collection
Statistical theory provides a guide to comparing methods of
data collection
Data collection or data gathering is the process of gathering and measuring information on targeted variables in an established system, which then enables one to answer relevant questions and evaluate outcomes. Data collection is a research com ...
, where the problem is to generate informative data using
optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
and
randomization Randomization is the process of making something random. Randomization is not haphazard; instead, a random process is a sequence of random variables describing a process whose outcomes do not follow a deterministic pattern, but follow an evolution d ...
while measuring and controlling for
observational error
Observational error (or measurement error) is the difference between a measured value of a quantity and its true value.Dodge, Y. (2003) ''The Oxford Dictionary of Statistical Terms'', OUP. In statistics, an error is not necessarily a " mistake ...
. Optimization of data collection reduces the cost of data while satisfying statistical goals,
[Atkinson et al. (2007)] while
randomization Randomization is the process of making something random. Randomization is not haphazard; instead, a random process is a sequence of random variables describing a process whose outcomes do not follow a deterministic pattern, but follow an evolution d ...
allows reliable inferences. Statistical theory provides a basis for good data collection and the structuring of investigations in the topics of:
*
Design of experiments
The design of experiments (DOE, DOX, or experimental design) is the design of any task that aims to describe and explain the variation of information under conditions that are hypothesized to reflect the variation. The term is generally associ ...
to estimate treatment effects, to test hypotheses, and to optimize responses.
*
Survey sampling In statistics, survey sampling describes the process of selecting a sample of elements from a target population to conduct a survey.
The term " survey" may refer to many different types or techniques of observation. In survey sampling it most ofte ...
to describe
populations
Population typically refers to the number of people in a single area, whether it be a city or town, region, country, continent, or the world. Governments typically quantify the size of the resident population within their jurisdiction using a ...
[Särndal et al. (1992)]
Summarising data
The task of summarising statistical data in conventional forms (also known as
descriptive statistics
A descriptive statistic (in the count noun sense) is a summary statistic that quantitatively describes or summarizes features from a collection of information, while descriptive statistics (in the mass noun sense) is the process of using and an ...
) is considered in theoretical statistics as a problem of defining what aspects of statistical samples need to be described and how well they can be described from a typically limited sample of data. Thus the problems theoretical statistics considers include:
*Choosing
summary statistics
In descriptive statistics, summary statistics are used to summarize a set of observations, in order to communicate the largest amount of information as simply as possible. Statisticians commonly try to describe the observations in
* a measure of ...
to describe a sample
*Summarising
probability distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
s of sample data while making limited assumptions about the form of distribution that may be met
*Summarising the relationships between different quantities measured on the same items with a sample
Interpreting data
Besides the philosophy underlying
statistical inference
Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution, distribution of probability.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical ...
, statistical theory has the task of considering the types of questions that
data analyst
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making. Data analysis has multiple facets and approaches, enc ...
s might want to ask about the problems they are studying and of providing data analytic techniques for answering them. Some of these tasks are:
*Summarising populations in the form of a fitted distribution or
probability density function
In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
*Summarising the relationship between variables using some type of
regression analysis
In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one ...
*Providing ways of predicting the outcome of a random quantity given other related variables
*Examining the possibility of reducing the number of variables being considered within a problem (the task of
Dimension reduction
Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally ...
)
When a statistical procedure has been specified in the study protocol, then statistical theory provides well-defined probability statements for the method when applied to all populations that could have arisen from the randomization used to generate the data. This provides an objective way of estimating parameters, estimating confidence intervals, testing hypotheses, and selecting the best. Even for observational data, statistical theory provides a way of calculating a value that can be used to interpret a sample of data from a population, it can provide a means of indicating how well that value is determined by the sample, and thus a means of saying corresponding values derived for different populations are as different as they might seem; however, the reliability of inferences from post-hoc observational data is often worse than for planned randomized generation of data.
Applied statistical inference
Statistical theory provides the basis for a number of data-analytic approaches that are common across scientific and social research.
Interpreting data is done with one of the following approaches:
*
Estimating parameters
*Providing a
range of values instead of a
point estimate
In statistics, point estimation involves the use of sample data to calculate a single value (known as a point estimate since it identifies a point in some parameter space) which is to serve as a "best guess" or "best estimate" of an unknown popu ...
*
Testing statistical hypotheses
Many of the standard methods for those approaches rely on certain
statistical assumptions (made in the derivation of the methodology) actually holding in practice. Statistical theory studies the consequences of departures from these assumptions. In addition it provides a range of
robust statistical techniques that are less dependent on assumptions, and it provides methods checking whether particular assumptions are reasonable for a given data set.
See also
*
List of statistical topics
0–9
* 1.96
*2SLS (two-stage least squares) redirects to instrumental variable
*3SLS – see three-stage least squares
*68–95–99.7 rule
*100-year flood
A
*A priori probability
*Abductive reasoning
*Absolute deviation
*Absolute risk red ...
*
Foundations of statistics The foundations of statistics concern the epistemological debate in statistics over how one should conduct inductive inference from data. Among the issues considered in statistical inference are the question of Bayesian inference versus frequentist ...
References
Citations
Sources
*
* Pre-publication chapters are available on-line.
*
* Cox, D.R., Hinkley, D.V. (1974) ''Theoretical Statistics'',
Chapman & Hall
Chapman & Hall is an imprint owned by CRC Press, originally founded as a British publishing house in London in the first half of the 19th century by Edward Chapman and William Hall. Chapman & Hall were publishers for Charles Dickens (from 1840 ...
.
*
*
*
Kish, L. (1965), ''Survey Sampling'',
John Wiley & Sons
John Wiley & Sons, Inc., commonly known as Wiley (), is an American multinational publishing company founded in 1807 that focuses on academic publishing and instructional materials. The company produces books, journals, and encyclopedias, in p ...
.
* .
*
Further reading
*
Peirce, C. S.
** (1876), "Note on the Theory of the Economy of Research" in ''Coast Survey Report'', pp. 197–201 (Appendix No. 14)
NOAA PDF Eprint Reprinted 1958 in ''Collected Papers of Charles Sanders Peirce'' 7, paragraphs 139–157 and in 1967 in
Operations Research' 15 (4): pp. 643–648
Abstract from JSTOR
** (1967)
** (1877–1878), "
Illustrations of the Logic of Science"
** (1883), "
A Theory of Probable Inference"
** and
Jastrow, Joseph (1885), "On Small Differences in Sensation" in ''Memoirs of the National Academy of Sciences'' 3: pp. 73–83
Eprint
*
*Davison, A.C. (2003) ''Statistical Models''. Cambridge University Press.
*
*
External links
*
{{DEFAULTSORT:Statistical Theory