Scoring criteria your team will actually agree on
Every scoring method is only as good as its criteria, and most teams copy the list from a template without agreeing what the words mean. Ask five people to score "strategic fit" and you get five different questions answered, averaged to one decimal place. The ranking looks precise and is noise. Here is what a usable criterion looks like, a starter set, and a 20-minute routine to agree yours.
Why the criteria matter more than the method
Whether you use a weighted score, RICE or a simple matrix, the method only multiplies and adds. If the inputs mean different things to different people, no arithmetic repairs that. The symptom: scores cluster between 3 and 4 and the room argues about the criterion, not the idea.
What a good criterion looks like
- One dimension. "Value and feasibility" is two criteria in one box. A valuable idea that is hard to build gets a meaningless 3.
- Observable. Someone outside the room could check it. "Removes a manual step in order handling" can be checked. "Fits our DNA" cannot.
- Anchored. A written sentence for a 1 and one for a 5. Without them, one person's 4 is another's 2 and nobody notices.
- Relevant to this decision. Not the company values, not last year's strategy slide. A criterion that would rank the ideas the same way for any decision does no work.
- Few. Three to five. With ten, every idea ends up "fairly good overall".
A starter set and when each fits
| Criterion | What it measures | A 1 reads like | A 5 reads like |
|---|---|---|---|
| Customer value | Gain for a named customer group | Nobody has asked for this | Customers raise the problem unprompted |
| Strategic fit | Contribution to one named goal of this year | Does not touch the goal | Moves the goal visibly |
| Feasibility | Can we build it with what we have | Needs skills or partners we lack | Current team, current tools, this quarter |
| Cost or effort (inverse) | People and money until a first usable version | A quarter of the team for months | A week for one person |
| Time to first result | How soon we learn whether it works | A year or more | Within four weeks |
| Risk (inverse) | What breaks if it fails | Harm to customers, revenue or compliance | Only the time spent |
| Learning value | What we know afterwards that we do not know now | Confirms what we know | Answers the question the portfolio depends on |
Pick by decision. Unproven ideas: learning value and time to first result. A budget round: cost, risk and strategic fit. All seven at once: a flat ranking.
How to agree them in 20 minutes
- Name the decision. Say it out loud: "We pick three ideas to test next quarter with two people each." The criteria follow from that sentence.
- Each person writes the three things that would make an idea a yes. Silently, one per card, three minutes.
- Cluster to 3 to 5 criteria. Group the cards on the wall and name each cluster with a noun, not a slogan.
- Write the 1 and the 5 for each on the wall. Full sentences, visible to everyone, before any scoring. Most of the time goes here.
- Decide weights by asking which criterion you would drop first. The last one standing gets the highest weight. Two or three levels are enough.
- Run two test ideas through them. One obvious winner, one obvious loser. If the criteria do not separate them, adjust the anchors.
- Freeze them before anyone sees the real list. Type criteria, anchors and weights into the sheet. Tools like IdeaHone store criteria, weights and scale next to the scores, so a ranking from March still reads in September.
Write the anchors as sentences about the idea, not as adjectives. "Feasibility 5: we can build it with the current team this quarter" can be tested. "Feasibility 5: very feasible" cannot.
A criterion that nobody scores below 3 is not a criterion. It adds the same number to every idea and changes nothing. Drop it or sharpen the anchors.
Criteria that rig or flatten the ranking
- The criterion that encodes the sponsor's favourite. "Uses our new platform" is not a criterion, it is a pre-selected winner. If an idea is already decided, say so and score the rest.
- "Innovation" as a criterion. Every idea in the room is innovative by definition, so people end up scoring how much they like it.
- Changing criteria after seeing the scores. Once the ranking is visible and someone says "feasibility should weigh more", the exercise is over.
- Ten criteria to keep everyone happy. Every idea scores a comfortable 3,4 and the top ten are separated by rounding.
- No anchors, so the scale drifts. The first person's scores set the standard and everyone else calibrates to them. The ranking then measures speaking order.
Score and prioritise ideas with your team
IdeaHone scores ideas against your criteria, shows them on a matrix and keeps the reasons, so the decision is clear and traceable.
What criteria should you use to evaluate ideas?
Three to five from customer value, strategic fit, feasibility, cost or effort, time to first result, risk and learning value, depending on the decision. Early ideas need learning value and speed; funding decisions need cost, risk and strategic fit.
How many criteria for scoring ideas?
Three to five. Fewer than three and one criterion dominates; more than five and the scores average out until the ranking is flat.
How do you define a scoring scale for ideas?
Use 1 to 5 and write both ends as full sentences about the idea, for example "Feasibility 5: we can build it with the current team this quarter." Test the anchors on one obvious winner and one obvious loser, then freeze them before scoring.
