Receiving a “don’t know” answer to a survey question can be frustrating for survey researchers. The response is difficult to interpret and provides little value for analysis.

More complete data is gathered using probing, allowing survey researchers to capture the opinions of those who would otherwise be excluded. Respondents can often require probing on a variety of topics including knowledge-based questions and attitudinal questions.

In online surveys, questionnaires are designed to probe by following up with respondents after a “don’t know” response. Respondents are asked “could you please give reasons for your opinion or response?” as an open-ended question after a multiple choice question. Probing gives the respondent an opportunity to explain themselves and provide meaningful insight to survey researchers.

So, what factors have an impact on the quality of probing?

Researchers from Leibniz Institute for the Social Sciences in Mannheim, Germany studied the impact of probing questions in online surveys. Their study used data from two online panels. One was a community panel where members can actively get involved, for example, by creating their own polls, and the other was a conventional panel where answering surveys is the member’s only activity.

The researchers hypothesized that community panel members would respond more favorably to probing than members of conventional panels. However, the study found that high community involvement did not increase the likelihood to answer probes or produce longer responses.

Context and frequency have an impact on the quality of probing. Providing context around probing makes a significant difference in the effectiveness of probing. Providing both the probed item and the respondent’s answer produces the highest number of meaningful answers.

Researchers should probe when additional insight is essential for analysis. Probing selectively also makes a difference in the effectiveness of probing. The likelihood of answering a probe decreases with the number of preceding probes. However, the word count of those who eventually answer the probes slightly increases with increasing number of probes.