Screening Participants

Recruiting the most suitable people for your study

Once you have established your selection criteria, you'll need to create a process that effectively qualifies people who fit your criteria and disqualifies those who do not. A screening questionnaire helps ensure that you are only talking to people that will give you meaningful information about your key research questions. Think about screener questions as a series of gates that gradually filter out candidates, leaving only the most qualified.

1. Screen for all requirements: Even if you've specified in your recruitment posting that you are looking for "working parents of young children", you should still build in questions that screen for those characteristics. You can't assume your respondents have meticulously read your posting or study description.

2. Avoid leading and binary questions : Remember, people are taking your screener because they want to participate in your study. If your questions are leading, it's extremely easy for someone who shouldn't qualify to get through your survey, leaving you with unusable data. Similarly, yes/no or binary questions should also be avoided.

3. Provide clear and distinct answers that are mutually exclusive: Good survey questions provide respondents with clear and complete options. The more you know about your target group and industry, the more exhaustive your screener questions will be, increasing the reliability of your data.

4. Test the screener: User testing the survey ensures your questions are clear and that any logic built in to reject or accept people works the way you intended. The screener should be tested by someone with fresh eyes who doesn't have direct knowledge of your project. Provide your tester with your screener criteria and have them take on the qualities of an ideal candidate. Then, have them run through your questions, noting any points of confusion. Next, have a new tester (who does not fit the desired criteria) take the screener to ensure they would be filtered out.

Example: Recruiting people who bike to work.

Leading question 👎

How often do you bike to work?

• Daily ACCEPT

• a few times /week ACCEPT

• a few times /month ACCEPT

• a few times /year REJECT

• I have never cycled to work REJECT

This question assumes that the respondent cycles to work, which may not be true!

Binary Question 👎

Do you bike to work?

• Yes ACCEPT

• No REJECT

We haven't assumed the respondent bikes, but it's not difficult to guess which answer the researchers are hoping for.

Better Question 👍

Which of the following transportation methods have you used in the past month while commuting? (select all that apply)

• Drove my own vehicle MAY SELECT

• Carpooled in another person's vehicle MAY SELECT

• Took public transit MAY SELECT

• Biked MUST SELECT

• Walked MAY SELECT

• Other, please specify: _________ MAY SELECT

This question does not assume any transportation method, nor does it give away the study's intent. Additionally, this structure allows for multiple response types. The respondent may select any of the options, but must select the cycling option to advance.

Be on the look out for screener hacking. If you are paying respondents to participate in your research, some applicants may look to bypass your screener by giving you the answers they think will get them selected. One way to combat this is by including trick questions, such as asking if they've used a product that doesn't exist (and disqualifying them if they answer yes).