A funnel plot is a graph designed to check for the existence of

publication bias In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance o ...

; funnel plots are commonly used in

systematic review A systematic review is a Literature review, scholarly synthesis of the evidence on a clearly presented topic using critical methods to identify, define and assess research on the topic. A systematic review extracts and interprets data from publ ...

s and

meta-analyses A meta-analysis is a statistical analysis that combines the results of multiple scientific studies. Meta-analyses can be performed when there are multiple scientific studies addressing the same question, with each individual study reporting me ...

. In the absence of publication bias, it assumes that studies with high precision will be plotted near the average, and studies with low precision will be spread evenly on both sides of the average, creating a roughly

funnel A funnel is a tube or pipe that is wide at the top and narrow at the bottom, used for guiding liquid or powder into a small opening. Funnels are usually made of stainless steel, aluminium, glass, or plastic. The material used in its construct ...

-shaped distribution. Deviation from this shape can indicate publication bias.

Quotation

Funnel plots, introduced by Light and Pillemer in 1984 and discussed in detail by

Matthias Egger Matthias Egger is professor of epidemiology and public health at the University of Bern in Switzerland, as well as professor of clinical epidemiology at the University of Bristol in the United Kingdom. Education and career Egger completed his cli ...

and colleagues, are useful adjuncts to meta-analyses. A funnel plot is a

scatterplot A scatter plot (also called a scatterplot, scatter graph, scatter chart, scattergram, or scatter diagram) is a type of plot or mathematical diagram using Cartesian coordinates to display values for typically two variables for a set of data. ...

of treatment effect against a measure of study precision. It is used primarily as a visual aid for detecting bias or systematic

heterogeneity Homogeneity and heterogeneity are concepts often used in the sciences and statistics relating to the uniformity of a substance or organism. A material or image that is homogeneous is uniform in composition or character (i.e. color, shape, siz ...

. A

symmetric Symmetry (from grc, συμμετρία "agreement in dimensions, due proportion, arrangement") in everyday language refers to a sense of harmonious and beautiful proportion and balance. In mathematics, "symmetry" has a more precise definiti ...

inverted funnel shape arises from a ‘well-behaved’ data set, in which publication bias is unlikely. An asymmetric funnel indicates a relationship between treatment effect estimate and study precision. This suggests the possibility of either

or a systematic difference between studies of higher and lower precision (typically ‘small study effects’). Asymmetry can also arise from use of an inappropriate effect measure. Whatever the cause, an asymmetric funnel plot leads to doubts over the appropriateness of a simple meta-analysis and suggests that there needs to be investigation of possible causes. A variety of choices of measures of ‘study precision’ is available, including total sample size,

standard error The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error ...

of the treatment effect, and inverse

variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...

of the treatment effect (

weight In science and engineering, the weight of an object is the force acting on the object due to gravity. Some standard textbooks define weight as a Euclidean vector, vector quantity, the gravitational force acting on the object. Others define weigh ...

). Sterne and Egger have compared these with others, and conclude that the standard error is to be recommended. When the standard error is used, straight lines may be drawn to define a region within which 95% of points might lie in the absence of both

and publication bias. In common with

confidence interval In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...

plots, funnel plots are conventionally drawn with the treatment effect measure on the

horizontal axis A Cartesian coordinate system (, ) in a plane is a coordinate system that specifies each point uniquely by a pair of numerical coordinates, which are the signed distances to the point from two fixed perpendicular oriented lines, measured in ...

, so that study precision appears on the vertical axis, breaking with the general rule. Since funnel plots are principally visual aids for detecting asymmetry along the treatment effect axis, this makes them considerably easier to interpret.

Criticism

The funnel plot is not without problems. If high precision studies are different from low precision studies with respect to

effect size In statistics, an effect size is a value measuring the strength of the relationship between two variables in a population, or a sample-based estimate of that quantity. It can refer to the value of a statistic calculated from a sample of data, the ...

(e.g., due to different populations examined) a funnel plot may give a wrong impression of publication bias. The appearance of the funnel plot can change quite dramatically depending on the scale on the y-axis — whether it is the inverse square error or the trial size. Researchers have a poor ability to visually discern publication bias from funnel plots.

Quotation

Criticism

See also

References

Further reading