Overview
Screening surveys are a way to accept or reject applicants that are invited to your test based on how they answer one or more questions. They are the first step of the application process to join a test.
The tester screening process works like this:
Testers must match the defined demographic and device criteria to be eligible to apply to a test
When you launch the test, we invite testers. A tester receives a private invite to a test via email, or clicks to apply to a test that is available publicly.
The tester is presented with the screening survey
If they answer any screening questions incorrectly, their application to join the test will be rejected
Most screening surveys are created with multiple-choice questions, with the answers to each question having a 'thumbs up' or 'thumbs down' option. Testers have to pass all of the questions in order to participate, so it is best to limit your questions to your most important questions.
On the BetaTesting platform, there are screening options for Automatic Selection and Manual Review, which you can read more about below.
Automatic Selection: Testers that successfully answer all questions are automatically accepted into your test and immediately receive instructions and survey questions based on your test design.
Manual Review: Multiple choice questions must be passed and text questions can also be asked to learn more about testers before they join your test. You will receive a list of Pending testers who passed your screening requirements and you can choose one-by-one who you want to accept into your test based on their demographics and screening answers.
Automatic Selection
When you choose Automatic Screening during the recruiting process, any tester who passes ALL of your questions will automatically be accepted into the test, until the total number of testers you selected is reached.
If a tester is rejected for any question, they will not be accepted into your test.
If a tester passes a screening survey with "Automatic Selection" enabled, they will immediately receive any instructions you provided, as well as any surveys that are set to go out.
Note: For questions that include checkboxes (i.e. allow applicants to select multiple answers), a testers will pass a question if they select at least one acceptable answer.
Manual Review
Manual Review gives you the ability to hand pick testers for your test based on their responses to your screening questions or their demographic and device criteria.
You can include multiple choice questions that testers must pass, plus text questions that you can review before you accept a tester into your test. Like Automatic Selection, testers still have to pass any multiple choice screening questions you created.
How Manual Review Works:
When your test launches, we continue to invite participants until approximately 2X the number of desired participants have passed your demographic and screening criteria. You will see a notification on your test dashboard that indicates you have participants pending acceptance, and you can also see this list within the Testers->Recruiting Results section.
In the Testers->Recruiting Results tab, you can review and choose the testers you want to accept. Scrolling horizontally will allow you to review each tester's answers to all your screening questions (in addition to their demographic and device details).
Click the checkbox next to each applicant, and click the "Accept to Test" button.
3. You can accept extra participants if you want, and will be charged for any completed test results at the end of the test.
4. Once you accept a tester, they are immediately given access to your test instructions. For Moderated Interview tests, they will now have access to your link to book your interview.
5. For moderated interviews, if some participants haven't booked a call after 24 hours, you may want to accept more/replacement testers into the test. This is normal and usually takes a few rounds of accepting people to fill your test.
NDA, Terms, and Closed Betas
For clients that subscribe to the Recruit Premium, Professional, or Enterprise plans, you have access to use several additional screening question types.
Document Signing
The "Document Signing" screening survey item allows you to distribute an NDA or other agreement terms via PDF. Testers must then view/download and digitally sign (by typing their name) your agreement. Learn more about Document Signing and various ways to distribute NDAs and project terms here:
Collect Emails for Closed Betas
"Collect Emails" for closed betas is a screening survey item that allows you to collect emails from applicants during the screening survey in the event that you need to manually whitelist testers to provide access to your app for closed betas. You will have access to see each tester's email address after they are accepted into your test.
If you chose Automatic Selection, then you will have access to each tester's email immediately after they are accepted. If you chose Manual Review, you will see each tester's email after you manually accept them into the test.
Where to see each tester's email for whitelisting:
Emails are viewable within the Testers -> Recruiting Results section. You can view emails online or click the button to Export all your testers and view your tester data in a spreadsheet.
Question Types
Multiple choice questions are the most common type of screening question. You ask a question, and the answers to each multiple-choice question have a 'thumbs up' or 'thumbs down' option to choose acceptable and rejected answers.
You can also turn a multiple-choice question into a checkbox question to allow multiple answers to be chosen. A tester will only be rejected if they select rejected answers ONLY. If a tester selects 1 acceptable answer and 2 rejected answers, they will still get in.
Testers have to pass all of the questions in order to participate, so it is best to limit your questions to your most important questions. Testers will be accepted automatically if they pass your screening requirements under Automatic Selection.
With Manual Selection, you can also choose from text questions and file uploads before you select testers:
Short Text: Add a question for users to answer quickly
Long Text: Add a question for users to answer in more detail
Shipping Address: Have users share their address for hardware tests
Accept File Upload: Have users upload a picture or video that you can review before you accept testers.
Common Types of Questions
Here are some common types of questions asked in screening surveys:
Lifestyle Questions:
How often do you workout?
Do you own a home?
How many flights do you book each year?
What age are your children?
Participation Questions:
This app requires you to share your location to participate. Would you still like to participate?
This test requires you to share a post on Facebook. Would you still like to participate?
Technical Questions:
Which model of iOS device do you own?
Are you familiar with VPN services and products?
Have you ever traded cryptocurrencies?
Product / Brand Questions:
Which of the following project management apps have you used before?
Which of the following streaming services do you currently have?
Creating too many questions may result in a tester pool that is too small to get the number of completed tests you are requesting. To get the best results, try to create open-ended questions that don't lead testers to answer a certain way. It's best to limit each screening survey to 3-6 questions.
Question Types that are Not Allowed:
Make sure you avoid questions that might violate our rules or other internet privacy rules, such as:
Health related questions that violate HIPAA privacy laws
Questions that ask about other sensitive topics that may be offensive to have to answer
Questions that reveal personal info that a user may not want to share or have associated with their online profile.
💡Want to learn more? Book a call/demo with our team or get in touch through our contact form.