Idea Ranking

Once you’ve generated your ideas the next step is idea ranking to determine the potential to become innovations. So, how do you decide which ideas to work on? The typical innovation process leaves that decision to the senior-level managers, which seems like a logical choice. They’re senior, so they should have the wisdom to make the best choices, right? Unfortunately, they often aren’t involved in the process of creating and selecting the best ideas. Because of this, the ideas they like may well be heavily influenced by their personal preferences and biases. If this is the case, the chance of the ideas selected becoming killer innovations will be low. A properly defined ranking system, on the other hand, helps people to set aside their biases and look at ideas from a bigger-picture perspective.

Many believe that the idea ranking process must be some kind of complex set of analytics. It is not. I’ve developed a very simple and elegant process. The idea ranking system uses questions to determine which ideas not only will have significant results but are also aligned with your core capabilities and expertise. It’s based on having the team score five questions for every idea generated in the workshop. The final scores rank your ideas from best to worst, which allows you to clearly see which ones you should pursue further.

The Five Questions For Idea Ranking

Score your answers to each of the first three questions from 0 to 5. A 0 means that the idea being considered doesn’t move the needle on this question (or, in other words, is a no-go for the time being), while a 5 is a resounding yes. You should ask these five questions for each of the ideas being considered. You’re looking for both an initial yes/no response, and an eventual score for the ideas that get the initial yes.

• Does this idea improve the customer’s experience and/or expectation?

Does this idea solve a problem? Will it give your customer something they need and want? Will it solve their problems in a unique and different way?

• Does this idea fundamentally change how you’re positioned competitively in the market?

Will this idea put you ahead of your competition? Will it disrupt the market’s idea about your organization?

• Does this idea radically change the economic structure of the industry?

Will this idea disrupt the way value is created and monetized so that your organization benefits?

If you can’t answer yes to one or more of these first three questions, then you’re doing the same thing as everyone else, so why bother.

The last two questions are about whether the idea is a strategic fit for your organization. To have a killer idea, you must be able to answer yes to both of them. Score them the same way you did for the first three.

• Do you have a contribution to make?

Do you have the experience and/or expertise to bring to the problem?

• Will this idea generate sufficient margin?

Are the potential returns enough to justify the cost of pursuing the idea?

Just because you can do something, it doesn’t mean you should. When I ask you if you can make a contribution, I’m asking if your organization brings some capability to the idea. HP has the R&D budget to explore just about any avenue of innovation, but we’re not going to work on an idea that is too far outside of our industry or our core strengths. There needs to be some foundation to leverage from.

Just because you can do something, it doesn’t mean you should.

Finally, you want to figure out if this idea is lucrative enough to be worth pursuing. This is often a matter of gut instinct, since for a truly great idea, you won’t have a precedent to determine its value. Think about it. If an idea is solid enough to get a yes on the first three questions, it should make money. In most organizations, return on investment (ROI) is the first filter that an idea has to pass through if it’s ever going to get funded. Setting ROI as your primary filter means that you will discard a ton of potentially great ideas, because there is no way to prove (to the satisfaction of the CFO and CEO) in advance exactly what the return will be. The determination of how much it will return will be the output of the stage-gate funding process. Don’t get too hung up on how much but more on its potential.

Keep All Of The Ideas

Once you’ve completed the idea ranking, keep the top ideas to use now, but don’t discard the rest. Instead, hold on to the master list of all ideas, and revisit in a year or so, when the timing might be better for them. I don’t specify how many ideas we should end with because it varies depending on what the objective is for any given group. And always remember, just because an idea comes out with a top score, it doesn’t mean it’s the best idea to pitch. Once everyone is finished totaling their scores for each idea you can gather the totals for all the scores and determine which ones are the best to explore further.

Avoiding Bias By Anonymity

One of the most interesting aspects of developing the ranking process was realizing how careful we needed to be to weed out bias and influence in the voting phase. I quickly found that we’d get completely different results when people knew their vote was anonymous versus if they were voting in a public forum with a bunch of senior execs in the room. I’ve been thrown out of workshops by my own team leaders because of the concern that “people will want to make Phil happy.” I have to constantly be on guard not to have that kind of an influence.

Originally I had a dot system that was quick to use and simple to tally. The people in the workshop had red, green, and yellow dots that were respectively worth one, two, and three points. At the end of a workshop participants walked up and put their dots on the ideas they liked. Seems simple, right? Unfortunately I quickly realized that people were waiting to see what the alpha-dog executive voted for, then immediately rallied behind his or her votes. The votes were aggregated and lost any usefulness because they essentially reflected only one person’s opinion. Anonymity significantly changes the group dynamics, so it’s critical to keep people unaware of how the other participants are voting.