Based on the years of extensive experience running beta and usability tests, we generally recommend 50 - 100 testers per test cycle. We recruit more than the target number to guarantee high quality engagement and feedback from all testers.
Beyond that number, feedback and bugs often become repetitive and it becomes more difficult to analyze the data. It often is more effective to run multiple test cycles and iterate with new test designs and feedback.
However, depending on your goals and budget, we can run tests at any scale and we’ve helped clients run tests with thousands of testers in the past. This is better suited to load testing, crowdsourcing data, and other test types.