01 The quantitative social survey method

In this chapter

By the end of this chapter, you should be able to:-

• define the key distinguishing features of quantitative social surveys;
• describe the kinds of information that may be gathered in quantitative social surveys and the contexts and settings in which surveys may be employed;
• identify the main steps in the survey process and the interactions between these steps; and
• list the skills required to conduct a successful quantitative social survey.

What is quantitative survey research?

The quantitative survey method may be defined as ‘a set of scientific procedures for collecting information and making quantitative inferences about populations’. A ‘social’ survey will be about some aspect of people in society, such as: their personal and household attributes; their living circumstances; their behaviour and lifestyle; their attitudes, beliefs and expectations. 

One principal tool used by the quantitative survey researcher to achieve their objectives is scientific sampling of populations. The other principal tool is the survey questionnaire. This has been defined as

“an instrument consisting of a series of questions and / or attitude-opinion statements designed to elicit responses which can be converted into measures of the variable under investigation”. (Franklin and Osborne, 1971)

Distinguishing features of the survey method

A number of features make the survey method different from other methods of social research.

• Surveys involve collecting new data, rather than being purely theoretical, or based on information already available (i.e. secondary data).
• There is careful prior definition of the information to be collected.

• The data collected and the results are quantitative (counts, rates etc.); ‘qualities’ (e.g. gender, strength of opinion) are assigned a numeric code so that they can be more readily manipulated in statistical analyses, and are analysed in terms of counts, percentages and similar quantifiable measures.
• Surveys involve collecting data about a population of units of some defined type.
• But surveys are usually based on samples, rather than complete censuses, of these units.
• The method for selecting the sample is fixed and objective; ideally, the method should be based on statistical probability theory.
• The sample of units for which data are collected is often large (hundreds or even thousands).
• The procedures for sampling these units and for collecting information are explicit, systematic and standardised (e.g. standardised questionnaires in contrast to the more fluid approaches such as topic guides used in the collection of qualitative data through focus groups or in-depth interviews). They are designed to achieve objective measurement and to yield comparable results across the entire sample.
• The results from the measurements made on the sample are summarised statistically. Conclusions about the population are drawn within confidence limits defined using sampling theory (i.e. inferences are made from the sample to the underlying population). Conclusions may be descriptive of the population or based on the testing of specified hypotheses.
• Surveys are conducted in the real world, under circumstances which cannot be fully controlled, rather than in the more rarefied laboratory setting.
• Surveys use a wide range of human skills and other resources and require much planning and teamwork.

Quantification in social surveys

Quantitative social surveys aim to convert the information they collect to meaningful numbers (e.g. counts, averages, rates etc.) relating to the population of interest.  It is not good enough simply to produce numbers, if their meaning is not clear, or to elicit information which does not yield useful numeric estimates. Because most surveys are sample surveys (rather than population censuses), some random sampling variability in the results obtained is inevitable (i.e. if the same questionnaire was administered to a different sample drawn from the same population, the findings from the two samples would not be identical).  For this reason alone, the results from sample surveys will always be estimates, surrounded by margins of error (in statistical terms, parameter estimates with associated confidence limits). One of the main reasons for favouring probability sampling methods is that they make it possible to estimate the margins of random sampling error. 

Key aims of optimised probability sampling are, therefore:-

• to minimise random sampling variability (variance);
• to avoid or minimise systematic sampling bias.

Meaningful numbers are produced by classifying, counting and scoring respondents’ answers to survey questions.  These procedures are followed at the analysis stage by summarising and estimating, whereby inferences are made from the sample on which data was collected to the underlying population.  The central challenge for the survey researcher is to devise classifications and scoring systems that, on the one hand, are practical in terms of the data collection process and, on the other hand, satisfy the requirements of data analysis and the making of population estimates.  For example, sample units / respondents can be classified and counted on the basis of their questionnaire responses, according to attributes such as their age, their diagnosis, their educational status, their occupation, their socio-economic class etc. Or they can be scored according to the amount of something they do (e.g. eating certain foods, engaging in leisure activities), the frequency of their symptoms, their quality of life measured by a set of questions, the direction and strength of their opinion on a topic, their expressed level of satisfaction etc.  Taken together, counts (which in turn can be converted to rates, percentages and so on) and scores (which can be summarised in terms of means, standard deviations, medians, inter-quartile range, ranges etc) provide summary measures of the sample as a whole.  From these sample values, estimates of these parameters in the underlying population can be made, using appropriate statistical techniques.  Parameters of interest may include age distribution, what proportion have engaged in some form of behaviour, the average rate of performing some behaviour, what proportion are satisfied with a service received and so on.

Types of information that may be gathered in a survey

Depending on the objectives of the data collection exercise, questionnaire surveys may be used to gather one or more of the following types of information (based on Dillman, 1978; p80):-

• Attributes and statuses – according  to Dillman, “what people are” (for example: personal characteristics, such as age; gender; marital status; personal and familial medical history; educational achievements etc.). Attributes may be seen as something that are an intrinsic and relatively stable (at least at a given point in time) part of the respondent, as opposed to something s/he does. In general terms, temporary and potentially changeable statuses (e.g. marital status) can be distinguished from more permanent attributes (e.g. gender, ethnicity) but no sharp line can be drawn between the two.  Attributes and statuses are generally regarded as facts, but of course the reporting of facts in surveys may be subject to distortions.
• Behaviour and events – according to Dillman, “what people do” (for example: frequency of engaging in potentially risky behaviour such as smoking or alcohol consumption; mode of travel to work) or “what has happened in people’s lives” (for example: having a particular acute disease; experiencing a bereavement; experiencing redundancy).  Questions about behaviour may refer to past, current or (intended) future behaviour; questions about events usually refer to the past.  Events – for example, leaving school, getting a job, getting married – can be viewed in their own right, or can be seen as a shift or point of transition between one status and another – for example, single to married.  Periods during which an individual is in a particular status are sometimes termed ‘episodes’. 
• Beliefs  / knowledge – according to Dillman, “ what people think is true”  (for instance: beliefs and understanding regarding the causes of illness; assessments of the likelihood of being the victim of different types of crime).  Assessing beliefs means assessing what respondents believe to be true or false, correct or incorrect. There is no implied value judgement about what is ‘good’ or ‘bad’.  Belief questions include those which test a respondent’s knowledge of facts (e.g. who is the Prime Minister, whether HIV can be transmitted by blood donation), as well as those which tap into issues for which there is no agreed ‘correct’ answer (e.g. whether homeopathy works).

• Attitudes / opinions / reasons for doing things etc. – according to Dillman, “what people say they want”, “how people feel about something” (for example: satisfaction with health care services; tolerance towards people of different ethnic origins, motivation for career choice).  Attitudes and opinions are essentially evaluative, reflecting respondents’ value judgements about what is good or bad, effective or ineffective, desirable or undesirable and so on.  Measuring attitudes involves making assumptions about how people structure their perceptual world. For example, it is no use asking questions to elicit people’s attitudes to ‘services provided by X county council’ if they have no concept of what the county council is or does.  Sometimes, questions may be posed in such a way that they tap a mixture of attitudes and beliefs, but the distinction is still usefully made (Fishbein, 1967; Dillman, 1978).

It is important to bear these distinctions in mind when designing questionnaires and determining question wording.  Unless the researcher is clear about what s/he is trying to measure, the information yielded by the question may not be what is actually required.

Applications of quantitative social survey research

There are many types of survey. Social surveys are a method of studying human beings and relations and interactions between them.  The social survey method is used in:-

• demography (studies of births, deaths, family formation, structure of populations etc.);
• sociology (studies of social organisations, networks etc.);
• social psychology (studies of perceptions, values, attitudes, reactions, individual and group behaviour etc.);
• political science (studies of voting intention and behaviour, political allegiance etc.)
• economics (studies of economic behaviour, distribution of income, wealth, spending power, expenditure etc.);
• social medicine and epidemiology (studies of health needs, health-related behaviour, illness, disability, experience of and satisfaction with health services etc.);

• market or consumer research (studies of customer demands, perceptions, choice of products and services, satisfaction with services etc.);
• government research (studies of demography, employment, education, health, income, expenditure, housing, travel, crime, leisure, social attitudes etc.).

Many surveys are carried for instrumental reasons (e.g. consumer satisfaction surveys as part of a quality assurance initiative; needs assessment surveys to plan provision of health and social services).  Others are carried out to expand basic scientific knowledge. Data collection using questionnaires may also be used in the context of other research designs, particularly in evaluations of policies, services and other interventions.  For example, in randomised controlled trials of health care technologies, it is increasingly common to gather information on patient outcomes using self-completion questionnaires.

Of course, there are other methods of eliciting social information, which have some parallels with surveys, but differ from surveys in other respects.  It is important, therefore,  to realise that a survey is not always the best way to gather accurate and unbiased information on ‘social’ issues.  For example, people may not be adequately informed about certain aspects of their health status, such as whether their blood pressure or cholesterol levels are normal or raised; in this situation, a more accurate picture would be obtained from scrutiny of medical records (i.e. use of secondary data) or by direct measurement (i.e. primary data collection).  Likewise, self-reports of performance in exams or attendance at classes may be distorted; more accurate data might be obtained from school or university records.  Turnstile counts and ticket sales are efficient, unobtrusive means of gathering data about the number of attendances at concerts, sporting events, exhibitions and other public attractions.

Stages in the survey process

The key stages in carrying out a survey are summarised in Box 1.  In this handbook, the main focus is on steps 6 to 10. For further details on other aspects of the survey research process, see one of the many comprehensive texts available (identified in the Further Reading section at the end of this handbook). 

It is important to realise that the survey process defined by the steps shown in Box 1 is not a rigid sequence; rather the process involves iteration and integration – pilot work may indicate the need for changes to questionnaires or to sampling procedures; decisions about study design will influence choice of data collection method and sample design and so on. Brilliant ideas about survey content can be ruined by: lack of clear definitions; incorrect statistical design; sloppy measurement procedures. Brilliant questionnaire design can be ruined by: poor sampling; inadequate fieldwork design and operations; inappropriate data analysis. Good statistical design can be ruined by poor communication, lack of psychological and sociological insight, and inadequate project management.  It is therefore important to keep the ‘big picture’ constantly in mind, looking back to the aims and design, and anticipating the analysis and interpretation of findings. 

Box 1 Steps in a survey (after Oppenheim, 1992)

Skills needed in survey research

Designing a sample survey and analysing the findings requires statistical skills to:-

• define the population to be sampled;
• work out the required sample size and optimum sampling design;
• understand concepts such as bias, variance, validity, reliability, and their relevance and application to surveys;
• work out what data analysis and estimation methods should be used;
• apply and interpret the selected methods correctly;
• identify and interpret
• which statistical results are statistically reliable;
• what the statistical results mean in practical terms;
• which results are practically important;
• which sources of response and measurement error we should be concerned about.

However, the successful conduct of a quantitative sample survey is dependent on successful communication between sets of human beings:-

• between survey designers and survey sponsors (otherwise the survey will not properly address the issues in which the sponsor is interested);
• between survey designers and respondents (because the latter must understand and be able to answer the questions and follow the instructions provided);
• between project managers and all other staff and organisations working on the study (otherwise Murphy’s Law – anything that can go wrong will go wrong – will triumph);
• between survey designers and data users (otherwise the findings may be interpreted and applied incorrectly).

Psychological and sociological insight, creativity and knowledge are needed to:-

• operationally define  those aspects of people’s lives that we wish to study such as
• personal attributes and living circumstances;
• behaviour;
• attitudes;
• devise ways of quantifying these definitions through questions and response categories;
• design the interface between researcher and respondent
• questionnaire design;
• contact procedures;
• interview procedures (where appropriate);
• contribute to the interpretation of the quantitative results of the study
• what do the statistical results mean in practical terms?
• which statistical results are practically important?
• what sources and effects of response error should worry us?

Finally, conducting a survey requires planning and management of complex, interactive operations and requires managerial skills to:-

• estimate and procure the resources needed;
• draw up a budget and financial plan, and monitor expenditure against this plan;
• time-table activities and identify the ‘critical path’;
• monitor progress against plans and take remedial action if necessary.

Large scale quantitative surveys

The features described above apply to all social surveys, regardless of scale (from small-scale surveys conducted in fulfilment of a degree dissertation to large-scale national surveys).  However, there are some particular features of larger-scale surveys, particularly those with multiple stake-holders, that warrant consideration.  Such large-scale surveys require agreements, commitments and understandings between the different parties involved in the survey:-

• between sponsoring organisations (the ‘client’ or ‘sponsor’) and research organisations;
• between researchers and support staff (secretaries, interviewers, data coders, data entry personnel);
• between research organisations and the subjects of the research (the survey respondents).

The issues about which agreement, promises and understandings need to be in place include:-

• the obligations of the different parties to help with or co-operate in the survey;
• commercial or financial rights and obligations;
• publication rights and ownership of data;
• whether participation in the survey by respondents is voluntary or compulsory;
• what counts as informed consent of survey subjects to participate in the survey;
• rules and guarantees of confidentiality and use of information about individuals.

A proper ethical and legal basis for surveys is therefore essential.  Those concerned must have a clear understanding of the relevant legislative framework (e.g. General Data Protection Regulations) and of ethical codes.

Summary of key points

• Quantitative social surveys involve a scientific approach to collecting quantitative data (counts, rates etc.) about some aspect of people in society, and to making statistical inferences about the underlying population.  Data are collected in a real-life setting.
• Quantitative social surveys are characterised by the collection of new data, using structured instruments and standardised procedures. Data are collected from a (large) sample, using fixed and objective methods, and are summarised statistically. 
• Many types of information can be gathered through quantitative social surveys – attributes and statuses; behaviour and events; beliefs and knowledge; attitudes and opinions.

• Quantitative social surveys have a wide range of applications, spanning many disciplines and settings.
• The survey process involves several activities, from problem definition through design of data collection methods and sampling protocols, to data gathering, analysis and interpretation.  The process is integrative and iterative.
• Skills needed to conduct a successful quantitative social survey include statistical expertise, good communication skills, psychological and sociological insight and project management skills.

Some examples of surveys on health-related topics

• Arising from the White Paper The new NHS – modern, dependable, the UK Department of Health has embarked on a series of surveys of patients’ experiences – see http://www.nhssurveys.org/surveys. 
• The Advice Centre for the NHS Surveys has a website at http://www.nhssurveys.org/
• The Department of Health also carries out regular health surveys of the general population of the United Kingdom, or of subsets of that population (often, the design and conduct of these surveys is contracted out to survey research organisations such as the National Centre for Survey Research.  The national surveys often include collection of anthropometric and clinical data, as well as questionnaire data on self-reported health and health-related behaviour.  The reports of these studies are strong on the methods employed – overall survey design, sampling strategy, ethical issues etc. – as well as on the substantive findings.  Links to the 2019 report from the Health Survey for England may be found at: https://digital.nhs.uk/data-and-information/publications/statistical/health-survey-for-england/2019   with links to other years’ surveys at https://digital.nhs.uk/data-and-information/publications/statistical/health-survey-for-england; those where there are links to the full report are particularly valuable, since they include the methodological aspects.

• Links to other Department of Health surveys may be found at https://www.health-ni.gov.uk/topics/doh-statistics-and-research/doh-commissioned-surveys while data from Welsh and Scottish health surveys may be found at http://ukdataservice.ac.uk/get-data/key-data.aspx
• The National Audit Office reports archive provides current access to reports on public services, including studies that use patient and provider surveys.
•  

Further reading

• A good general resource for researchers is the Research Methods Knowledge Database, an electronic textbook on many aspects of the research process.  It may be found at http://www.socialresearchmethods.net/kb/index.htm  . 
• The American Association for Public Opinion Research has produced a useful guide to best practice in survey and public opinion research, elaborating on issues raised in this chapter and elsewhere in this handbook.  It is available at https://aapor.org/standards-and-ethics/best-practices/

• Colorado State University has a nice website on survey methods at https://writing.colostate.edu/guides/pdfs/guide68.pdf .
• The Australian Bureau of Statistics has an on-line guide to basic survey design at http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Basic+Survey+Design+-+Questionnaire+Design

Current guidance and methodological literature for 2026–27:

• American Association for Public Opinion Research. Best Practices for Survey Research. current guidance.
• European Social Survey. Methodology overview. current Round 12 framework.
• European Social Survey. Round 12 Survey Specification. 2025.
• Cross-Cultural Survey Guidelines. Full survey-lifecycle guidelines. 2016, maintained online.

Some useful web sites

The major survey organisations, both in the United Kingdom and the United States, have their own web sites.  These often provide access to electronic versions of the organisation’s publications, or order forms for hard copy versions of those publications.  Many of the sites also include useful links to other web based resources.  All are well worth a browse, though inclusion of a link does not imply our whole-hearted endorsement of everything you will find by pursuing it – do not suspend your critical faculties!

• The National Centre for Social Research (NatCen) is the largest independent social research institute in Britain. It conducts social research among members of the public to provide information on a range of social policy issues in Britain. For more than 30 years NatCen has conducted the design and fieldwork for important, policy informing, UK social surveys including The British Crime Survey, British Social Attitudes, Family Resources Survey, The Health Survey for England, and many other major quantitative and qualitative surveys. NatCen’s web site is at http://www.natcen.ac.uk/ and details of their research may be found at http://www.natcen.ac.uk/our-research
• An excellent guide to using survey data, from surveys such as those listed above, has been produced by the UK Data Service – see https://www.ukdataservice.ac.uk/media/455510/using-survey-data.pdf (if this link does not take you to the guide, go to https://www.ukdataservice.ac.uk/, select site search and in the search box enter the phrase using survey data).
• The Cathie Marsh Centre for Census and Survey is an interdisciplinary research centre in the Faculty of Social Sciences and Law at the University of Manchester.  The work of CCSR includes methodological research relating to census and survey data and its web site is at https://www.cmi.manchester.ac.uk/
• The UK Office for National Statistics (ONS) has a group specialising in survey methodology and quality, including a Survey Methodology Bulletin – see https://www.ons.gov.uk/methodology
• The US Bureau of the Census is the United States’ equivalent of our Office for National Statistics.  Its web site is very comprehensive – see https://www.census.gov/en.html .  Of particular interest are the Statistical Research Division’s series of research and study reports on many aspects of survey methods, which may be downloaded in pdf format from https://www.census.gov/library/working-papers.html (click on ‘view working papers by series’, select research and then research reports – methodology, or survey measurement)
• The U.S. National Center for Health Statistics publishes current surveys and data-collection systems for epidemiological and health-services research; the NCHS surveys overview links to individual programmes and documentation.
• The National Opinion Research Centre (NORC) in the United States is a leading centre for research and practice in survey methods – see http://www.norc.org/Research/Capabilities/Pages/study-design-and-survey-methodology.aspx
• Dillman, Smyth and Christian’s Internet, Phone, Mail, and Mixed-Mode Surveys provides a current publisher record for the Tailored Design Method.
• William Trochim’s Research Methods Knowledge Base (2^nd edition), at http://www.socialresearchmethods.net/kb/, has a good section on survey methods at http://www.socialresearchmethods.net/kb/survey.htm

In addition to these organisation- and person-specific sites, there are a number of useful portals, providing access to other sites and resources (note however, that some of the onward links are no longer functional).

• Realities is part of the National Centre for Research Methods, which aims to improve research methods across the UK social science community. They are based in the Morgan Centre for the Study of Relationships and Personal Life at the University of Manchester and have some useful on-line and downloadable material, including on different types of interview, albeit with an emphasis on qualitative methods  – see
• https://www.socialsciences.manchester.ac.uk/morgan-centre/resources/toolkits/
• The website Electronic Resources for Research Methods at https://web.archive.org/web/20170606053132/http://www.informationr.net/rm/ has sections on survey methods, questionnaires and other topics relevant to survey research

• The International Consortium for the Advancement of Academic Publication also maintains a general portal entitled Resources for Methods in Evaluation and Social Research at https://gsociology.icaap.org/methods/index.html , which lists free resources for methods in evaluation and social research.  The focus is on "how-to" do evaluation research and the methods used: surveys, focus groups, sampling, interviews, and other methods.  The section on survey methods is at http://gsociology.icaap.org/methods/surveys.htm
• Links to useful survey methods resources may also be found at https://sites.google.com/site/gsocialchange/surveys
• See https://ccsg.isr.umich.edu/ for cross-cultural survey guidelines

References

Dillman DA.  Mail and telephone surveys: The total design method,  New York:  John Wiley and Sons, Inc, 1978.

Fishbein M. A consideration of beliefs, and their role in attitude measurement. In Fishbein M (Ed.). Readings in attitude theory and measurement. New York: John Wiley and Sons, 1967.

Franklin B and Osborne H.  Research methods: issues and insights,  Belmont:  Wadsworth Publishing Company Incorporated, 1971.

Oppenheim AN.  Questionnaire design, interviewing and attitude measurement,  2nd edition.  London:  Pinter Publishers, 1992.