Survey methodology is "the study of
survey
Survey may refer to:
Statistics and human research
* Statistical survey, a method for collecting quantitative information about items in a population
* Survey (human research), including opinion polls
Spatial measurement
* Surveying, the techniq ...
methods".
As a field of
applied statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
concentrating on
human-research surveys, survey methodology studies the
sampling of individual units from a
population
Population typically refers to the number of people in a single area, whether it be a city or town, region, country, continent, or the world. Governments typically quantify the size of the resident population within their jurisdiction using a ...
and associated techniques of
survey data collection With the application of probability sampling in the 1930s, surveys became a standard tool for empirical research in social sciences, marketing, and official statistics.
The methods involved in survey data collection are any of a number of ways in ...
, such as
questionnaire construction
Questionnaire construction refers to the design of a questionnaire to gather statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subj ...
and methods for improving the number and accuracy of responses to surveys. Survey methodology targets instruments or procedures that ask one or more questions that may or may not be answered.
Researchers carry out statistical surveys with a view towards making
statistical inference
Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution, distribution of probability.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical ...
s about the population being studied; such inferences depend strongly on the survey questions used.
Polls about
public opinion
Public opinion is the collective opinion on a specific topic or voting intention relevant to a society. It is the people's views on matters affecting them.
Etymology
The term "public opinion" was derived from the French ', which was first use ...
, public-health surveys,
market-research surveys, government surveys and
census
A census is the procedure of systematically acquiring, recording and calculating information about the members of a given population. This term is used mostly in connection with national population and housing censuses; other common censuses incl ...
es all exemplify
quantitative research
Quantitative research is a research strategy that focuses on quantifying the collection and analysis of data. It is formed from a deductive approach where emphasis is placed on the testing of theory, shaped by empiricist and positivist philosop ...
that uses survey methodology to answer questions about a population. Although censuses do not include a "sample", they do include other aspects of survey methodology, like questionnaires, interviewers, and non-response follow-up techniques. Surveys provide important information for all kinds of
public-information and research fields, such as
marketing
Marketing is the process of exploring, creating, and delivering value to meet the needs of a target market in terms of goods and services; potentially including selection of a target audience; selection of certain attributes or themes to emph ...
research,
psychology
Psychology is the scientific study of mind and behavior. Psychology includes the study of conscious and unconscious phenomena, including feelings and thoughts. It is an academic discipline of immense scope, crossing the boundaries betwe ...
,
health-care provision and
sociology
Sociology is a social science that focuses on society, human social behavior, patterns of Interpersonal ties, social relationships, social interaction, and aspects of culture associated with everyday life. It uses various methods of Empirical ...
.
Overview
A single survey is made of at least a sample (or full population in the case of a census), a method of data collection (e.g., a questionnaire) and individual questions or items that become data that can be analyzed statistically. A single survey may focus on different types of topics such as preferences (e.g., for a presidential candidate), opinions (e.g., should abortion be legal?), behavior (smoking and alcohol use), or factual information (e.g., income), depending on its purpose. Since
survey research
In research of human subjects, a survey is a list of questions aimed for extracting specific data from a particular group of people. Surveys may be conducted by phone, mail, via the internet, and also at street corners or in malls. Surveys are us ...
is almost always based on a sample of the population, the success of the research is dependent on the representativeness of the sample with respect to a target population of interest to the researcher. That target population can range from the general population of a given country to specific groups of people within that country, to a membership list of a professional organization, or list of students enrolled in a school system (see also
sampling (statistics)
In statistics, quality assurance, and survey methodology, sampling is the selection of a subset (a statistical sample) of individuals from within a statistical population to estimate characteristics of the whole population. Statisticians attempt ...
and
survey sampling In statistics, survey sampling describes the process of selecting a sample of elements from a target population to conduct a survey.
The term "survey" may refer to many different types or techniques of observation. In survey sampling it most often ...
). The persons replying to a survey are called
respondent
{{unreferenced, date=February 2012
A respondent is a person who is called upon to issue a response to a communication made by another. The term is used in legal contexts, in survey methodology, and in psychological conditioning.
Legal usage
In ...
s, and depending on the questions asked their answers may represent themselves as individuals, their households, employers, or other organization they represent.
Survey methodology as a scientific field seeks to identify principles about the sample design, data collection instruments, statistical adjustment of data, and data processing, and final data analysis that can create systematic and random survey errors. Survey errors are sometimes analyzed in connection with survey cost. Cost constraints are sometimes framed as improving quality within cost constraints, or alternatively, reducing costs for a fixed level of quality. Survey methodology is both a scientific field and a profession, meaning that some professionals in the field focus on survey errors empirically and others design surveys to reduce them. For survey designers, the task involves making a large set of decisions about thousands of individual features of a survey in order to improve it.
[
]
The most important methodological challenges of a survey methodologist include making decisions on how to:
* Identify and select potential sample members.
* Contact sampled individuals and collect data from those who are hard to reach (or reluctant to respond)
* Evaluate and test questions.
* Select the mode for posing questions and collecting responses.
* Train and supervise interviewers (if they are involved).
* Check data files for accuracy and internal consistency.
* Adjust survey estimates to correct for identified errors.
Selecting samples
The sample is chosen from the sampling frame, which consists of a list of all members of the population of interest.
The goal of a survey is not to describe the sample, but the larger population. This generalizing ability is dependent on the representativeness of the sample, as stated above. Each member of the population is termed an element. There are frequent difficulties one encounters while choosing a representative sample. One common error that results is
selection bias
Selection bias is the bias introduced by the selection of individuals, groups, or data for analysis in such a way that proper randomization is not achieved, thereby failing to ensure that the sample obtained is representative of the population int ...
. Selection bias results when the procedures used to select a sample result in over representation or under representation of some significant aspect of the population. For instance, if the population of interest consists of 75% females, and 25% males, and the sample consists of 40% females and 60% males, females are under represented while males are overrepresented. In order to minimize selection biases,
stratified random sampling
In statistics, stratified sampling is a method of sampling from a population which can be partitioned into subpopulations.
In statistical surveys, when subpopulations within an overall population vary, it could be advantageous to sample each s ...
is often used. This is when the population is divided into sub-populations called strata, and random samples are drawn from each of the strata, or elements are drawn for the sample on a proportional basis.
Modes of data collection
There are several ways of administering a survey. The choice between administration modes is influenced by several factors, including
# costs,
# coverage of the target population,
# flexibility of asking questions,
# respondents' willingness to participate and
# response accuracy.
Different methods create
mode effect
Mode effect is a broad term referring to a phenomenon where a particular survey administration mode causes different data to be collected. For example, when asking a question using two different modes (''e.g.'' paper and telephone), responses to ...
s that change how respondents answer, and different methods have different advantages. The most common modes of administration can be summarized as:
[ Mellenbergh, G.J. (2008). Chapter 9: Surveys. In ]H.J. Adèr
HJ may refer to:
Science, technology, and mathematics
* Hall–Janko group, a mathematical group
* U.S. code for a cryptographic key change; see cryptoperiod
Other uses
* List of Latin-script digraphs#H, , a two-letter combination used in ...
& G.J. Mellenbergh
Gideon Jan (Don) Mellenbergh (9 August 1938 – 27 March 2021) was a Dutch psychologist, who was Professor of Psychological methods at the University of Amsterdam, known for his contribution in the field of psychometrics, and Social Research Me ...
(Eds.) (with contributions by D.J. Hand), Advising on Research Methods: A consultant's companion (pp. 183–209). Huizen, The Netherlands: Johannes van Kessel Publishing.
*Telephone
*Mail (post)
*Online surveys
*Personal in-home surveys
*Personal mall or street intercept survey
*Hybrids of the above.
Research designs
There are several different designs, or overall structures, that can be used in survey research. The three general types are cross-sectional, successive independent samples, and longitudinal studies.
Cross-sectional studies
In cross-sectional studies, a sample (or samples) is drawn from the relevant population and studied once.
A cross-sectional study describes characteristics of that population at one time, but cannot give any insight as to the causes of population characteristics because it is a predictive, correlational design.
Successive independent samples studies
A successive independent samples design draws multiple random samples from a population at one or more times.
This design can study changes within a population, but not changes within individuals because the same individuals are not surveyed more than once. Such studies cannot, therefore, identify the causes of change over time necessarily. For successive independent samples designs to be effective, the samples must be drawn from the same population, and must be equally representative of it. If the samples are not comparable, the changes between samples may be due to demographic characteristics rather than time. In addition, the questions must be asked in the same way so that responses can be compared directly.
Longitudinal studies
Longitudinal studies take measure of the same random sample at multiple time points.
Unlike with a successive independent samples design, this design measures the differences in individual participants’ responses over time. This means that a researcher can potentially assess the reasons for response changes by assessing the differences in respondents’ experiences. Longitudinal studies are the easiest way to assess the effect of a naturally occurring event, such as divorce that cannot be tested experimentally.
However, longitudinal studies are both expensive and difficult to do. It's harder to find a sample that will commit to a months- or years-long study than a 15-minute interview, and participants frequently leave the study before the final assessment. In addition, such studies sometimes require data collection to be confidential or anonymous, which creates additional difficulty in linking participants' responses over time. One potential solution is the use of a self-generated identification code (SGIC). These codes usually are created from elements like 'month of birth' and 'first letter of the mother's middle name.' Some recent anonymous SGIC approaches have also attempted to minimize use of personalized data even further, instead using questions like 'name of your first pet. Depending on the approach used, the ability to match some portion of the sample can be lost.
In addition, the overall attrition of participants is not random, so samples can become less representative with successive assessments. To account for this, a researcher can compare the respondents who left the survey to those that did not, to see if they are statistically different populations. Respondents may also try to be self-consistent in spite of changes to survey answers.
Questionnaires
![Questionaire in Thai](https://upload.wikimedia.org/wikipedia/commons/d/d0/Questionaire_in_Thai.png)
Questionnaires are the most commonly used tool in survey research. However, the results of a particular survey are worthless if the questionnaire is written inadequately.
Questionnaires should produce valid and reliable demographic variable measures and should yield valid and reliable individual disparities that self-report scales generate.
Questionnaires as tools
A variable category that is often measured in survey research are demographic variables, which are used to depict the characteristics of the people surveyed in the sample.
Demographic variables include such measures as ethnicity, socioeconomic status, race, and age.
Surveys often assess the preferences and attitudes of individuals, and many employ self-report scales to measure people's opinions and judgements about different items presented on a scale.
Self-report scales are also used to examine the disparities among people on scale items.
These self-report scales, which are usually presented in questionnaire form, are one of the most used instruments in psychology, and thus it is important that the measures be constructed carefully, while also being reliable and valid.
Reliability and validity of self-report measures
Reliable measures of self-report are defined by their consistency.
Thus, a reliable self-report measure produces consistent results every time it is executed.
A test's reliability can be measured a few ways.
First, one can calculate a test-retest reliability.
A test-retest reliability entails conducting the same questionnaire to a large sample at two different times.
For the questionnaire to be considered reliable, people in the sample do not have to score identically on each test, but rather their position in the score distribution should be similar for both the test and the retest.
Self-report measures will generally be more reliable when they have many items measuring a construct.
Furthermore, measurements will be more reliable when the factor being measured has greater variability among the individuals in the sample that are being tested.
Finally, there will be greater reliability when instructions for the completion of the questionnaire are clear and when there are limited distractions in the testing environment.
Contrastingly, a questionnaire is valid if what it measures is what it had originally planned to measure.
Construct validity of a measure is the degree to which it measures the theoretical construct that it was originally supposed to measure.
Composing a questionnaire
Six steps can be employed to construct a questionnaire that will produce reliable and valid results.
First, one must decide what kind of information should be collected.
Second, one must decide how to conduct the questionnaire.
Thirdly, one must construct a first draft of the questionnaire.
Fourth, the questionnaire should be revised.
Next, the questionnaire should be pretested.
Finally, the questionnaire should be edited and the procedures for its use should be specified.
Guidelines for the effective wording of questions
The way that a question is phrased can have a large impact on how a research participant will answer the question.
Thus, survey researchers must be conscious of their wording when writing survey questions.
It is important for researchers to keep in mind that different individuals, cultures, and subcultures can interpret certain words and phrases differently from one another.
There are two different types of questions that survey researchers use when writing a questionnaire: free response questions and closed questions.
Free response questions are open-ended, whereas closed questions are usually multiple choice.
Free response questions are beneficial because they allow the responder greater flexibility, but they are also very difficult to record and score, requiring extensive coding.
Contrastingly, closed questions can be scored and coded more easily, but they diminish expressivity and spontaneity of the responder.
In general, the vocabulary of the questions should be very simple and direct, and most should be less than twenty words.
Each question should be edited for "readability" and should avoid leading or loaded questions.
Finally, if multiple items are being used to measure one construct, the wording of some of the items should be worded in the opposite direction to evade response bias.
A respondent's answer to an open-ended question can be coded into a response scale afterwards,
or analysed using more qualitative methods.
Order of questions
Survey researchers should carefully construct the order of questions in a questionnaire.
For questionnaires that are self-administered, the most interesting questions should be at the beginning of the questionnaire to catch the respondent's attention, while demographic questions should be near the end.
Contrastingly, if a survey is being administered over the telephone or in person, demographic questions should be administered at the beginning of the interview to boost the respondent's confidence.
Another reason to be mindful of question order may cause a
survey response effect in which one question may affect how people respond to subsequent questions as a result of
priming
Priming may refer to:
* Priming (agriculture), a form of seed planting preparation, in which seeds are soaked before planting
* Priming (immunology), a process occurring when a specific antigen is presented to naive lymphocytes causing them to di ...
.
Nonresponse reduction
The following ways have been recommended for reducing nonresponse
[Lynn, P. (2008) "The problem of non-response", chapter 3, 35-55, in ''International Handbook of Survey Methodology'' (ed.s ]Edith de Leeuw
Edith Desiree de Leeuw (born April 12, 1962) is a Dutch psychologist, statistician, research methodologist, and professor in survey methodology and survey quality, at the University of Utrecht. She is known for her work in the field of survey r ...
, Joop Hox
Josephus Johannes Cornelis Maria (Joop) Hox (born 1949) is a Dutch psychologist and Professor of Social Science Methodology at the Utrecht University, known for his work in the field of social research method such as survey research and multilev ...
& Don A. Dillman
Don, don or DON and variants may refer to:
Places
*County Donegal, Ireland, Chapman code DON
* Don (river), a river in European Russia
* Don River (disambiguation), several other rivers with the name
*Don, Benin, a town in Benin
* Don, Dang, a v ...
). Erlbaum. in telephone and face-to-face surveys:
[Dillman, D.A. (1978) ''Mail and telephone surveys: The total design method''. Wiley. ]
* Advance letter. A short letter is sent in advance to inform the sampled respondents about the upcoming survey. The style of the letter should be personalized but not overdone. First, it announces that a phone call will be made, or an interviewer wants to make an appointment to do the survey face-to-face. Second, the research topic will be described. Last, it allows both an expression of the surveyor's appreciation of cooperation and an opening to ask questions on the survey.
* Training. The interviewers are thoroughly trained in how to ask respondents questions, how to work with computers and making schedules for callbacks to respondents who were not reached.
* Short introduction. The interviewer should always start with a short introduction about him or herself. She/he should give her name, the institute she is working for, the length of the interview and goal of the interview. Also it can be useful to make clear that you are not selling anything: this has been shown to lead to a slightly higher responding rate.
* Respondent-friendly survey questionnaire. The questions asked must be clear, non-offensive and easy to respond to for the subjects under study.
Brevity is also often cited as increasing response rate. A 1996 literature review found mixed evidence to support this claim for both written and verbal surveys, concluding that other factors may often be more important.
A 2010 study looking at 100,000 online surveys found response rate dropped by about 3% at 10 questions and about 6% at 20 questions, with drop-off slowing (for example, only 10% reduction at 40 questions).
Other studies showed that quality of response degraded toward the end of long surveys.
Some researchers have also discussed the recipient's role or profession as a potential factor affecting how nonresponse is managed. For example, faxes are not commonly used to distribute surveys, but in a recent study were sometimes preferred by pharmacists, since they frequently receive faxed prescriptions at work but may not always have access to a generally-addressed piece of mail.
Interviewer effects
Survey methodologists have devoted much effort to determining the extent to which interviewee responses are affected by physical characteristics of the interviewer. Main interviewer traits that have been demonstrated to influence survey responses are race, gender,
and relative body weight (BMI).
These interviewer effects are particularly operant when questions are related to the interviewer trait. Hence, race of interviewer has been shown to affect responses to measures regarding racial attitudes,
interviewer sex responses to questions involving gender issues,
and interviewer BMI answers to eating and dieting-related questions.
While interviewer effects have been investigated mainly for face-to-face surveys, they have also been shown to exist for interview modes with no visual contact, such as telephone surveys and in video-enhanced web surveys. The explanation typically provided for interviewer effects is
social desirability bias
In social science research, social-desirability bias is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting "good behavio ...
: survey participants may attempt to project a positive self-image in an effort to conform to the norms they attribute to the interviewer asking questions. Interviewer effects are one example
survey response effects Survey response effects are variations in survey responses that result from seemingly inconsequential aspects of survey design and administration. Susceptibility to these effects varies depending on the stability of one's beliefs. Those without a st ...
.
See also
*
Data Documentation Initiative
The Data Documentation Initiative (also known as DDI) is an international standard for describing surveys, questionnaires, statistical data files, and social sciences study-level information. This information is described as metadata by the st ...
*
Enterprise feedback management (EFM)
*
Likert scale
A Likert scale ( , commonly mispronounced as ) is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term (or more fully the ...
*
Official statistics
Official statistics are statistics published by government agencies or other public bodies such as international organizations as a public good. They provide quantitative or qualitative information on all major areas of citizens' lives, such as e ...
*
Paid survey
A paid or incentivized survey is a type of statistical survey where the participants/members are rewarded through an incentive program, generally entry into a sweepstakes program or a small cash reward, for completing one or more surveys.
Detai ...
*
Quantitative marketing research
Quantitative marketing research is the application of quantitative research techniques to the field of marketing research. It has roots in both the positivist view of the world, and the modern marketing viewpoint that marketing is an interactive ...
*
Questionnaire construction
Questionnaire construction refers to the design of a questionnaire to gather statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subj ...
*
Ratio estimator The ratio estimator is a statistical parameter and is defined to be the ratio of means of two random variables. Ratio estimates are biased and corrections must be made when they are used in experimental or survey work. The ratio estimates are asymme ...
*
Social research
Social research is a research conducted by social scientists following a systematic plan. Social research methodologies can be classified as quantitative and qualitative.
* Quantitative designs approach social phenomena through quantifiable ...
*
Total survey error
In survey sampling, total survey error includes all forms of survey error including sampling variability, interviewer effects, frame errors, response bias, and non-response bias. Total survey error is discussed in detail in many sources including ...
References
Further reading
* Abramson, J. J. and Abramson, Z. H. (1999). ''Survey Methods in Community Medicine: Epidemiological Research, Programme Evaluation, Clinical Trials'' (5th edition). London: Churchill Livingstone/Elsevier Health Sciences
*
Adèr, H. J.,
Mellenbergh, G. J., and Hand, D. J. (2008). ''Advising on research methods: A consultant's companion''. Huizen, The Netherlands: Johannes van Kessel Publishing.
* Andres, Lesley (2012)
"Designing and Doing Survey Research" London: Sage.
* Dillman, D.A. (1978) ''Mail and telephone surveys: The total design method''. New York: Wiley.
* Engel. U., Jann, B., Lynn, P., Scherpenzeel, A. and Sturgis, P. (2014). ''Improving Survey Methods: Lessons from Recent Research''. New York: Routledge.
* Groves, R.M. (1989). ''Survey Errors and Survey Costs'' Wiley.
* Griffith, James. (2014) "Survey Research in Military Settings." i
Routledge Handbook of Research Methods in Military Studiesedited by Joseph Soeters, Patricia Shields and Sebastiaan Rietjens.pp. 179–193. New York: Routledge.
* Leung, Wai-Ching (2001
in
Student BMJ
''Student BMJ'' is a monthly, international medical journal for medical students and junior doctors. It is published by the BMJ Group.
''Student BMJ'' was launched as a print journal in 1992 with the aim of publishing articles for medical student ...
, (
British Medical Journal
''The BMJ'' is a weekly peer-reviewed medical trade journal, published by the trade union the British Medical Association (BMA). ''The BMJ'' has editorial freedom from the BMA. It is one of the world's oldest general medical journals. Origi ...
, Student Edition), May 2001
* Ornstein, M.D. (1998). "Survey Research." ''Current Sociology'' 46(4): iii-136.
* Prince, S. a, Adamo, K. B., Hamel, M., Hardt, J., Connor Gorber, S., & Tremblay, M. (2008). A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. International Journal of Behavioral Nutrition and Physical Activity, 5(1), 56. http://doi.org/10.1186/1479-5868-5-56
* Shaughnessy, J. J., Zechmeister, E. B., & Zechmeister, J. S. (2006). ''Research Methods in Psychology (Seventh Edition ed.)''. McGraw–Hill Higher Education. (pp. 143–192)
* Singh, S. (2003). Advanced Sampling Theory with Applications: How Michael Selected Amy. Kluwer Academic Publishers, The Netherlands.
* Soeters, Joseph; Shields, Patricia and Rietjens, Sebastiaan.(2014)
Routledge Handbook of Research Methods in Military StudiesNew York: Routledge.
*
* Shackman, G
What is Program Evaluation? A Beginners Guide2018
External links
*
{{Authority control
Psychometrics
Quantitative research
Product testing