Our Bug Hunt test helps you quickly find bugs and technical issues in your product, across a range of real-world devices and use cases. By having a wide mix of testers go through your various features, you will discover issues across devices, operating systems, browsers, and more, that can't be found with in-house testing.
Our bug reports capture device information, and testers can add screenshots and videos to understand how to replicate the bug. You can also message testers within each bug report to ask questions and get more information.
Choose Bug Hunt from the test type section when you create a new test.
Your survey title is shown publicly to potential applicants during recruiting. Make it something relevant to your test, for example: "Bug test for a video editing app".
Brief description of your test
In the description, you can describe a bit about the test process or who you're targeting. This is also shown to applicants during the recruiting phase
How are my test title and description shown during recruiting?
When you launch your test, testers will receive an email invite with your Test Title as the email subject and Test Description in the email body. It will look similar to the screenshot below and will ask testers to apply to participate.
In the below email "We want your feedback on a new packaging design" is the Test Title, and "Help us understand what resonates..." is the Brief Description of Your Test.
Test Instructions (Bug Hunt)
Be sure to include:
Link to access your product (e.g. link to your website, TestFlight public beta URL, Play Store, APK, Firebase, prototype URL, etc).
Details on what you'd like testers to do. A Bug Hunt can be more "Functional Testing" with step-by-step tasks, or "Exploratory Testing" with high-level guidance.
Optionally attach any other document for instructions. Make sure it's clear and easy to understand, or you'll get poor results for your test.
Do not ask testers to update a spreadsheet or upload a document with their test results. The output of a Bug Hunt are the bugs that are filed through BetaTesting.
Don't put questions directly in the instructions. Instead, you need to include any questions you want to have answered directly in a survey itself.
Don't expect a tester to purchase something or go through a process that requires sensitive personal or financial information
If you have a complex functional test or a test process that requires multiple usage sessions instead, use the Multi-Day Test with the Advanced Test Workflow mode.
Don't provide a test process that can't be completed within 24 hours.
Timing & Incentives
How long do you want users to test
Provide a realistic estimate for how long your test process will take. You can give testers up to 60 minutes for a bug hunt. This should take into account the time required to get your product, go through features, and complete bug reports.
The Base Reward is the reward that all testers will get for following any tasks you provide and attempting to find bugs. For the base reward, our system automatically recommends average incentive amounts based on your test type, time expectations, and your target audience. You can learn more about choosing how much to incentivize in this article.
Define a Bonus reward to indicate that you will decide to provide bonuses at the end of the test to the testers that did a great job. You will have the opportunity at the end to choose which testers earned the bonus.
Recruiting & Screening
You can choose your recruitment criteria and screening questions like you normally would on any test. Learn how tester recruiting works here.
Choose what survey you'd like testers to complete a survey after they finish your bug hunt test.
Standard Bug Hunt Survey
Our standard Bug Hunt survey includes a series of questions that allow you to analyze the quality and stability of your product (e.g. number of bugs found, technical stability, speed, ease of use, etc). This is a safe bet for any bug test, and you can use as-is or edit as you desire!
Building your own survey
Participation Verification Survey: Linking feedback to your database usage data
BetaTesting does not integrate directly with your product. A Participation Verification survey is useful in the following cases:
If you want to cross reference your user feedback to exact usage data in your database
If you want testers to prove they participated fully and completed your tasks and instructions
The Participant Verification Survey is a simple 3 question standalone survey that asks testers for the following info:
Username / email / or phone number used to access your product, so you can cross-reference the dat
Screenshot to validate that they used the product
A few sentences about the testing experience.
Bug Report Results
After your test launches, you will see bug reports on your Bugs tab in your test. Each bug report includes self-reported data from the user that submitted the bug, such as:
Expected Result vs actual result
Attachments: screenshots or videos
From this page, you can change statuses, delete bugs or mark them as complete, combine bugs as duplicates, search for bugs, and much more.