Our Bug Hunt test helps you quickly find bugs and technical issues in your product, across a range of real-world devices and use cases. By having a wide mix of testers go through your various features, you will discover issues across devices, operating systems, browsers, and more, that can't be found with in-house testing.
Our bug reports capture device information, and testers can add screenshots and videos to understand how to replicate the bug. You can also message testers within each bug report to ask questions and get more information.
Choose Bug Hunt from the test type section when you create a new test.
Test Details
Test Title
Your survey title is shown publicly to potential applicants during recruiting. Make it something relevant to your test, for example: "Bug test for a video editing app".
Brief description of your test
In the description, you can describe a bit about the test process or who you're targeting. This is also shown to applicants during the recruiting phase
How are my test title and description shown during recruiting?
When you launch your test, testers will receive an email invite with your Test Title as the email subject and Test Description in the email body. It will look similar to the screenshot below and will ask testers to apply to participate.
In the below email "We want your feedback on a new packaging design" is the Test Title, and "Help us understand what resonates..." is the Brief Description of Your Test.
Test Instructions (Bug Hunt)
Be sure to include:
Link to access your product (e.g. link to your website, TestFlight public beta URL, Play Store, APK, Firebase, prototype URL, etc).
Details on what you'd like testers to do. A Bug Hunt can be more "Functional Testing" with step-by-step tasks, or "Exploratory Testing" with high-level guidance.
Optionally attach any other document for instructions. Make sure it's clear and easy to understand, or you'll get poor results for your test.
If you upload a file (e.g. a spreadsheet with tasks) make sure any instructions you provide to testers are clear. It should not be a guessing game to understand what you're asking. Do not ask that testers update a spreadsheet that you provide with their test results. The output of a Bug Hunt are the bugs that are filed through BetaTesting.
Don't include:
Do not ask testers to update a spreadsheet or upload a document with their test results. The output of a Bug Hunt are the bugs that are filed through BetaTesting.
Don't put questions directly in the instructions. Instead, you need to include any questions you want to have answered directly in a survey itself.
Don't expect a tester to purchase something or go through a process that requires sensitive personal or financial information
If you have a complex functional test or a test process that requires multiple usage sessions instead, use the Multi-Day Test with the Advanced Test Workflow mode.
Don't provide a test process that can't be completed within 12 hours.
Timing & Pricing
How long will it take to complete your test?
Provide an estimate for how long the test will take each tester from start to finish. Be sure to include a reasonable estimate for time required to download/access your product, engage with your product, and complete your survey.
How quickly do you want results?
With the Standard option, testers have 12 hours to finish your test, regardless of the expected time required. You should get the full results within ~24 hours if you're running a test with Automatic acceptance (rather than Manual Review). If a tester shows as "Active" during your test, it's because their due date has not yet passed.
The Expedited option costs $199 / test and allows you to get results within a few hours. If you have a contract with BetaTesting, we can discuss providing this feature for all your tests at no additional cost.
The No Rush option allows testers up to 24 hours to submit their results. If your test might require a tester to set aside some unique time during the day, this option would be helpful (e.g. an app that someone can use during a walk or for meal planning).
Price / Tester
BetaTesting uses credits for running tests. You can get volume discounts by purchasing subscriptions or larger credit packages.
The Bug Hunt Test costs 1 credit per tester for the standard test time requirements (30 minutes). Credit costs are inclusive of payments to testers (which means you don't pay anything extra for incentives).
How does my target audience impact pricing?
Targeting business professionals with employment targeting criteria, or defining very niche consumer requirements increase the credit cost by 2X. This is because business professionals (and other difficult-to-reach audiences) demand higher incentives to participate and higher rewards leads to better results.
To see our packages for credits, see our pricing page. We also have a credit calculator to see how many credits you need for various test configurations.
Custom incentives Feature
For clients that have a need to customize the incentive amount that you provide to each participant, get in touch to discuss our custom incentive feature. This is useful if you are giving testers free products as part of the test process (e.g. a free TV), or testing with your own customers/employees, or need to offer higher incentives for difficult tests.
When using this feature, our system recommends average incentive amounts based on your test type, time expectations, and your target audience. You can learn more about choosing how much to incentivize in this article.
Recruiting & Screening
You can choose your recruitment criteria and screening questions like you normally would on any test. Learn how tester recruiting works here.
Survey Design
Choose what survey you'd like testers to complete a survey after they finish your bug hunt test.
Standard Bug Hunt Survey
Our standard Bug Hunt survey includes a series of questions that allow you to analyze the quality and stability of your product (e.g. number of bugs found, technical stability, speed, ease of use, etc). This is a safe bet for any bug test, and you can use as-is or edit as you desire!
Building your own survey
You can build your own survey by using any of our base question types, or our core question bank. Learn about the various survey question types here and how to use show/hide logic here.
 | 
|
Participation Verification Survey: Linking feedback to your database usage data
BetaTesting does not integrate directly with your product. A Participation Verification survey is useful in the following cases:
If you want to cross reference your user feedback to exact usage data in your database
If you want testers to prove they participated fully and completed your tasks and instructions
The Participant Verification Survey is a simple 3 question standalone survey that asks testers for the following info:
Username / email / or phone number used to access your product, so you can cross-reference the dat
Screenshot to validate that they used the product
A few sentences about the testing experience.
You also have the option to add "participation verification" questions directly to any survey that you create. Choose "Verify Participation" within the Core Question Bank, and it will add these 3 questions directly to any survey.
Bug Report Results
After your test launches, you will see bug reports on your Bugs tab in your test. Each bug report includes self-reported data from the user that submitted the bug, such as:
Priority Level
Status
Title
Expected Result vs actual result
Bug Description
Attachments: screenshots or videos
Date Submitted
From this page, you can change statuses, delete bugs or mark them as complete, combine bugs as duplicates, search for bugs, and much more.
💡Want to learn more? Book a call/demo with our team or get in touch through our contact form.