What a survey panel is
A survey panel is a pool of people who have agreed, in advance, to answer questionnaires from time to time. Instead of finding fresh respondents for each study, a research organisation draws invitations from this pool. That saves time and money, and it makes some designs possible that one-off surveys cannot manage, such as asking the same people the same questions across several years to see how they change.
Panels are used widely in market research, academic work and public opinion polling. Understanding how they are made helps explain what they show well and where they can mislead.
How panels are built
There are two broad routes, and the difference matters a great deal.
Probability-based recruitment. Members are selected by chance from a list that is meant to cover most of the population, such as a register of addresses. Everyone on the list has a known chance of being invited. People without internet access may be given a device or offered another way to take part, such as a phone or paper questionnaire. This approach is slower and more expensive, but it gives a firmer basis for describing the wider population.
Opt-in recruitment. People join voluntarily after seeing an advertisement, a website offer, a loyalty scheme message or a referral. Because anyone can volunteer, no one can say what chance a given person had of being included. These panels can be large and cheap, but they depend heavily on who chooses to sign up. The contrast between this approach and chance selection is covered in more detail under opt-in polls.
In both cases, joining usually involves answering a profile questionnaire on age, location, education and similar traits. That profile information lets the organisation pick people matching a target group or adjust results later.
How panels are maintained
A panel is not built once. It is managed continuously:
- Refreshing. Members drop out, so new people are recruited to replace them.
- Rotation. Members are often invited only a limited number of times to avoid over-surveying them.
- Incentives. Small rewards, points or entries into prize draws are common. Incentives encourage participation but can also attract people mainly interested in the reward.
- Quality checks. Organisations typically screen for duplicate accounts, inattentive answering and responses that look automated or implausible. How strict and effective these checks are varies, and it is rarely visible to readers.
Who tends to be missing
No panel is a miniature copy of the public. Several groups are typically under-represented, to varying degrees depending on the design.
- People who are offline or hardly online. Online panels cannot reach those without access, and these people often differ in age, income and location from those who are connected.
- People with little spare time. Those with demanding jobs or caring duties may not volunteer or may drop out quickly.
- People wary of surveys or data sharing. Willingness to join a research pool is itself a characteristic, and it may correlate with the very views being measured, such as trust in institutions.
- People who do not read the language of the survey well. Surveys offered in a limited number of languages leave out residents who are more comfortable in another.
- People who move often or lack a stable address. Panels built from address lists struggle to follow them.
- Those who dislike answering questions repeatedly. Panel fatigue can lead to drop-outs among exactly those who find the process burdensome.
The practical result is that panels are often better at representing people who are comfortable online, settled and cooperative. This is a form of non-response bias that arises both at the moment of joining and over time as people leave.
Attrition and conditioning
In a panel that follows the same people, some members leave between waves. If those who leave differ from those who stay, the remaining group slowly drifts away from the original mix. This is called attrition, and researchers often try to reduce it with reminders, incentives and statistical adjustment.
There is also a subtler issue sometimes called panel conditioning. Answering questions about a topic can change how a person thinks or behaves. Someone repeatedly asked about their exercise habits may start paying attention to them, or may learn how to answer faster. If so, long-standing panel members may differ from fresh respondents for reasons unrelated to the wider trend.
Why weighting helps but does not solve it
Organisations usually adjust panel results so that the mix of respondents matches known figures for the population, such as the shares in each age group or region. This process, discussed under survey weighting, can correct for visible imbalances. It rests on an assumption: that within each group, the people who joined resemble those who did not. If, for example, the online older adults who join a panel hold different views from those who are offline, weighting by age cannot repair the difference.
Weighting can also increase uncertainty, because a few respondents may be counted heavily. This matters when interpreting a margin of error, which for opt-in panels cannot be interpreted in the usual way because the chance of selection is unknown.
What to look for in a panel-based claim
- Was the panel recruited by chance or by volunteering?
- Is there a statement of how the sample was adjusted?
- For change over time, were the same people followed, or were new samples drawn each time?
- Does the claim concern a group the panel is likely to cover well, or one it tends to miss?
Panels are powerful tools for studying change and for reaching people quickly. They are most reliable when the claim being made matches the population the panel can plausibly represent, and least reliable when a headline stretches the result to everyone.