Summary: Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.
There’s two points of possible confusion. First, the intersection between quality standards and the breath of your user base. For example, for our existing apps, we’re reaching for a high quality bar, but our user base also has two distinct groups: dancers and instructors. We’re skating by at the moment, only use instructors as our testers, and asking them to proxy on behalf of dancers. That’s going to scale for a while longer for us, but one needs to be sensitive when you go out onto the thin ice like that.
Second, it’s odd to even have to say this, but there’s no requirement that your testers actually work for your company. A vast majority of the *best* UX testers I’ve ever “worked” with have been volunteers. So, yeah… One less excuse of the doubters to use.
Love the iteration aspect. Once you loop in external users, this ties nicely with Steve Blank’s “talk to your users” mantra. Gives you something of value to interact with them about.