In this chapter…
This chapter brings the handbook decisions together and shows how classical and contemporary methods fit one defensible survey strategy.
By the end of this chapter, you should be able to…
- connect the research question, population, measurement, fieldwork and processing decisions
- explain why digital, AI and web-data tools still require piloting and validation
- document limitations and justify the evidence a survey can support
It is with good reason that Dillman’s (1978) classic text on postal and telephone surveys was subtitled The Total Design Method. The preceding sections of this handbook should have convinced the reader that the component aspects of the design and conduct of questionnaire surveys - for example, sampling or question wording - cannot and should not be considered in isolation. Excellent execution cannot save a survey if the design does not fit the aims; excellent design cannot save a survey if there are planning, resource or management deficiencies.
Dillman’s Total Design Method has two components. Firstly, it is important to identify how each aspect of the survey process may affect the quantity and quality of the data collected. The survey researcher must carefully consider and manipulate each aspect to obtain the best possible responses, while operating within the constraints imposed by scarce resources of time, money, personnel and equipment. Procedures and practices should be based on sound theory and empirical evidence from previous surveys. Secondly, care must be taken that the actual conduct of the survey is in line with the design intentions.
Throughout the survey process, the survey researcher should keep the aims and objectives of the research study at the forefront of their mind. At each step, the potential for bias and error to creep in should be considered and every effort should be made to minimise these threats.
Survey practice in a changing data environment
Digital, AI-assisted and web-data methods can extend survey practice, but they do not remove its foundations. A defensible study still begins with a clear research question and a defined population; it still requires coverage or sampling that supports the intended inference, valid and reliable measurement, ethical and legal governance, inclusive design, transparent documentation, human accountability, piloting and empirical validation.
Chapter 4: Digital methods of collecting survey data shows how web, mobile, messaging and mixed-mode collection can improve reach and timeliness. Those benefits must be assessed alongside mode effects, accessibility and digital exclusion. A digital questionnaire is not merely a paper form on a screen, and a mixed-mode design needs evidence that its measurements remain comparable.
Chapter 7: AI-assisted survey instrument design treats AI as a bounded tool under human control. AI-generated draft items are not validated instruments. They require source checking, construct review, bias and accessibility assessment, cognitive interviewing, piloting and documented human sign-off.
Chapter 14: Big data and web-scraped data for survey research explains how APIs, scraping and digital traces can complement survey evidence. Platform or web-scraped data are not representative survey evidence simply because they are numerous. Coverage, self-selection, bots, missingness, temporal change, rights, privacy and provenance determine what those data can support.
The practical test is therefore unchanged: can another informed reader understand why the method fits the question, how people and measures entered the data, what errors remain and what claims the evidence can justify? The contemporary chapters and their chapter-specific Further reading and References provide the detailed guidance for answering that question.
Further reading
- There are a number of good texts on survey research available. They tend to cover much the same material, so choosing between them is as much a matter of availability and preferred style. The list below includes a number of the relevant texts.
Czaja R and Blair J. Designing surveys: a guide to decisions and procedures. Thousand Oaks: Pine Forge Press, 1996. (A reasonable introduction, though less specifically oriented to health surveys)
de Vaus DA. Surveys in social research (4th edition). London: UCL Press, 1996. (A relatively recent book, regularly updated, providing a good overview of the whole survey research process)
Dillman DA. Mail and telephone surveys: The total design method. New York: John Wiley and Sons, Inc, 1978. (A classic, though somewhat dogmatic and, in parts, outdated)
Dillman DA. Mail and internet surveys: the tailored design method. 2nd ed. New York: John Wiley and Sons, Inc., 2000. (An update on the Dillman classic of 1978)
Fink A (editor). The survey kit. Thousand Oaks: Sage Publications, 1995. (A set of seven books, somewhat simplistic but a good introduction; volumes 1, 3 and 4 are particularly useful)
Fowler FJ Junior. Survey research methods (2nd edition). Newbury Park: Sage Publications (Applied Social Research Methods Series - Volume 1), 1993. (Fairly comprehensive and up-to-date)
Litwin MS. How to measure survey reliability and validity. Thousand Oaks: Sage Publications, 1995.
Mangione TW. Mail surveys - improving the quality. Thousand Oaks: Sage Publications (Applied Social Research Methods Series, Volume 40), 1995. (Similar to Fowler, but focusing on postal surveys)
Moser CA and Kalton G. Survey methods in social investigation. London: Gower, 1971. (A classic, though now a bit dated)
Oppenheim AN. Questionnaire design, interviewing and attitude measurement (2nd edition). London: Pinter Publications, 1992. (Another classic, but in an updated version)
Streiner DL and Norman GR. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press, 1989.
Sudman S and Bradburn NM. Asking questions - a practical guide to questionnaire design. San Francisco: Jossey-Bass Publishers, 1982. (A very comprehensive presentation of the wording of questions and response categories, and of questionnaire layout; little on other aspects of the survey process).
Further practical guidance
- AAPOR’s Best Practices for Survey Research provides current practical guidance from survey planning through reporting.
Current guidance and methodological literature for 2026–27:
- UK Statistics Authority. Code of Practice for Statistics 3.0. 2025.
- American Association for Public Opinion Research. Transparency Initiative disclosure elements. current guidance.
- American Association for Public Opinion Research. Responsible AI Integration in Survey Research. 2026.
- World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 2.2. 2023.
- Sen, I. et al.. A Total Error Framework for Digital Traces of Human Behavior on Online Platforms. 2021.
References
Dillman DA. Mail and telephone surveys: The total design method, New York: John Wiley and Sons, Inc, 1978.