Starting Up Wrong: 6 Product Testing Mistakes You Need to Avoid

The potential of any “great idea” is limited by how you execute it. Successful execution happens due to one of two reasons: luck and proper testing and iterations.

In other words, product-market fit is found through hard work or blind luck while minimizing mistakes along the way. After a few years, several other apps and dozens of iterations, our app, After School, a private social network for teens, was downloaded 100,000 times in the two weeks after launching the test version of it. If not for our process of testing ideas, we would’ve never been able to achieve this early milestone, and certainly would not have gone on to secure millions of downloads and active users.

Before creating your process for testing your business or app idea, you need to know what mistakes to avoid. From our own tests and working with other startups and founders, here are what I see as the most common testing mistakes when launching a product or service.

1. Letting bias taint the process

As an entrepreneur, you’re confident in your abilities. This confidence will help you overcome obstacles that others would give up before rising above, and it’s also likely to lead you to rely on your own ideas and gut instincts. This is where bias can taint the testing process. It’s easy to let our passion for an idea and our belief in our abilities override information and common sense. When testing your product or service, always test your assumptions and beware that your own bias could lead you toward the wrong decision.

2. Data problems

As long as you’re not solely relying on your own bias and gut instincts to make decisions, data drives actions. Despite its importance, data collection mistakes are just as common as they are costly. Here are the three most common types of data-based mistakes to avoid:

Small sample size: When doing focus groups or testing specific aspects of your product or service, get enough of a sample size that you’re confident their reviews would reflect the same opinions of a group 100 times their size.

Collecting the wrong data: Know what you’re measuring when testing and focus your testing efforts on relevant pieces of information to your specific product or service.

Narrow data: If you’re a skilled engineer and have a large network of engineers, only getting feedback from your network, even if it’s thousands of unique opinions, won’t reflect what your actual users want unless they, too, are skilled engineers.

3. Relying on technology to test

An app, hot dog stand or cleaning service for apartment complexes are all types of businesses, and none of them need technology in order to test. If your app connects people in a geographic location who love dogs, start doing it without the app. You’ll learn about your target market and gain important insight into what should and should be part of your future product.

4. Ignoring the competition

Many young entrepreneurs treat competitors in their space like kryptonite instead of a source of valuable information. If what you’re testing has been done before in some capacity, you now can set benchmarks and test your product against your competitors. If you’re not searching for what’s on the market and using this information in creating, testing and marketing your product, you’re missing a significant opportunity to create competitive advantages that you can utilize the second you launch that your competitors won’t see coming.

5. Waiting to 'sell' until it’s built

When launching the test version of After School, we knew there would be dozens of future changes we would need to implement based on how teens used our app. The launch took off, and we learned more in two weeks than through months of market research, focus groups and internal testing. When we re-launched, we had a user base to quickly provide feedback on features, which helped us grow even further. You never want to launch a bad product, but if it’s designed extremely well and gives users something unique, get it in users’ hands as quickly as possible because this is the testing that really matters, and it will determine whether you’re successful or not.

6. Leaving assumptions untested

Before launching the test version of our app, we decided to test the names of features, keywords we planned on using and several other content-related aspects of the app. Although we felt strongly about many of them, we decided to test anyway and it turned out that our target market had very different opinions than we did. Testing needs to be about better serving your target market. Test everything you can!