Building a new product is nerve-racking, especially if you’re taking a shot at something that’s never been tried before.
Will it end up being a YouTube or a Quibi? An iPod or a Zune? If it fails, will you ever be able to bounce back?
Luckily, you can build confidence by reducing risk. That’s where product experimentation comes in.
In this post, we'll discuss Cornell Tech Professor Karan Girotra’s De-Risking Framework, a three-step approach to testing your idea before launch.
Here’s how it works.
Identify what to test
When you have a big idea, it’s easy to feel overwhelmed by everything that could go wrong.
To make the process more manageable, break those concerns down into individual risks. Then break those down even more. Then keep going. And going.
These smaller risks fall into three buckets:
1. Demand
Do enough people want the product? How do you market to them? Will they come back for more or is it a one-time purchase?
2. Supply
Do you have the technology to make it work? If so, do you have the operational resources to carry it out? Can you find suppliers?
3. Value
How much can you charge? Will that exceed the costs? What about additional revenue streams? And what share of the market can you capture?
Challenge yourself to find at least 20-30 risks that address everything down to the elemental level. This will turn abstract implicit risks into explicit risks that you’ll be able to start testing against.
Identify how to test
Once you know what to test, the big question is “how?”
The best technique varies from product to product, but the overarching goal remains the same: Get the most amount of information with the least amount of money and effort.
Here are a few proven ways:
1. Build a prototype
A barebones prototype can address a singular variable. This can either be a physical object to test ergonomics or a stripped down landing display to measure user experience.
For example, the founders of B2B company Zapier first created a prototype that only included the front-end of the product - the part of the app users saw. The backend didn't exist until they tested user interest for different integrations.
2. Launch a pilot
If your idea has many interconnected factors that impact success, use a pilot to measure multiple variables. If you’re launching a meal service, a pop-up kiosk at a farmer’s market will help you understand demand. If it’s a new rideshare, test it within a neighborhood to figure out operations before you go national.
3. Market before you build
Market the product before building it to gauge interest or determine how much your audience is willing to pay. This approach fuels Kickstarter campaigns, which often depict the ideal version of the product well before the kinks have been ironed out.
4. Use non-scaleable hacks
When you're starting out, you can get scrappy by testing concepts yourself, even if you won't be handling all the work down the road.
The founders of DoorDash delivered their first meals on their own, and Airbnb’s co-founders hosted their first guests on an air mattress in their living room. This allows you to observe and address the big, initial concerns firsthand before you start hiring people to scale the operation.
Identify when to test
Not all risks are created equal. Some can create minor inconveniences, while others can upend your entire product.
Once you’ve determined how to test your risks, the next step is figuring out which to examine first.
Create a benefit-to-cost ratio to guide your decisions. The higher the ratio, the higher the priority it is to test.
1. Start by quantifying your risk
Create a table that can chart both importance and doubt. Use a 1-5 scale to think about this more clearly.
So let’s say your product is relying on new technology that hasn’t been fully developed yet.
If it’s key to the product’s success, you’ll rank importance at a 5. And if you’re not completely sure it’ll work, rank doubt at a 3.
When you multiply these two numbers, you’ll have a risk factor of 15 points.
2. Determine the benefit of an experiment
Let's use our example from before. If the new tech works once you test it, the importance to your product will remain a 5, but doubt will drop to 1.
This will leave you with a new risk factor of 5. The 10 points it dropped represents the benefit of the experiment. So you can list the benefit as a 10.
3. Look at costs
Quantify how much you expect to spend on a test. If it's thousands of dollars, $1,000 cost can be ranked as a 1, while $2,000 can be a 2.
Let’s say you’re building a functional prototype that’ll cost $1,000.
You’ll have a benefit-to-cost ratio of 10:1.
4. Compare risks by benefit-to-cost
Put every risk through the same method. If you test something just as important but with more doubt, you might end up with a new benefit-to-cost ratio that’s higher – say, 15:1.
Start with the risk that gives you a 15:1 benefit-to-cost ratio, then work your way back to the 10.
When you show that you’ve assessed and managed risks, your product becomes more appealing to potential investors. Join Karan Girotra’s Product Experimentation Sprint to fine-tune your big idea.