Evaluating decision quality to make better decisions

When have we made a good decision? On first examination, if we meet or exceed the desired or projected outcome, we might expect that we have made a good choice. However, what if the results have nothing to do with the decision that was made? We could be left with the false belief that our choice led to the desired outcome. In reality, our choice and supporting action may have had no influence, or possibly worked against the intended result.

An easy example would be that I choose heads in a coin flip and it lands as a heads. Most would agree that my choice had no influence on the outcome of the flip. If this happened a few times in a row, the added correlation might increase my belief that I can successfully choose the outcome of the coin flip. Sadly, a larger number of tosses would quickly demonstrate that my choice actually had no influence.

Problems with evaluating decision quality

The coin toss problem points to just one of the difficulties in evaluating decision effectiveness. We expect good choices to lead to good results, forgetting that decisions are developing a projection of something that will happen in the future. Accurate future predictions would require perfect knowledge of all causes and all possible results. Here are some of the difficulties in evaluating the quality of a decision:

• Imperfect information and knowledge - To check quality we need to establish the level of knowledge for which we should be held accountable.
• Relevance and causality - Evidence should clearly demonstrate a specific choice logically leads to or influences a desired outcome. For complex decisions, the number of interrelated factors can make this difficult.
• Timeliness - Decision timing can be difficult to determine because the loss in value associated with delay may not be known.
• Measurement of outcome - Strategic decisions may have an extended time between the decision and the anticipated outcome. Execution of these decisions requires maintaining commitment, action and accountability for long time periods.
• Data integrity - Distortion of facts, misrepresentations, unsupported interpretations, irrelevant information and lies can completely corrupt the decision making process.
• Unintended consequences - Evaluating all possibilities of what might go wrong is untenable, but there should be some accountability for knowable negative outcomes. Risk management helps address this problem.

Possible requirements for a good decision

In light of the difficulties above, it is still important to define or choose what makes for a good decision. Since the goal of a decision is to generate a preferred outcome, quality could be defined in terms of the outcome as:

• Minimally meeting specific needs
• Maximizing fulfillment of relevant goals or desires
• Realization of the outcomes with appropriate timing (hopefully more specific than as soon as possible)
• Minimal or no negative consequences (avoids harm)
• Demonstrated causality (Did choice and action influence the outcome, at least in part? This might require experiments to confirm)

However, we must still contend with all the uncertainties indentified in the problems above. We know that despite our best efforts, the preferred outcome may not occur.

Here is where a decision making process can help. If a process is shown to deliver the desired outcome with a specified likelihood, we could now expect that the decision outcome will be realized at the success rate of the process, assuming it is reliably followed. The outcome is not guaranteed for a single decision, but the process will enable a defined success rate (hopefully better) over a large number of decisions.

In this case, quality can measured at the time a decision is made by measuring fulfillment of the process. Making good decisions means that an effective process was followed that has a higher likelihood of generating the desired results. Quality might be defined in terms of the process as:

• Reasonable (or minimum) effort that does not exceed value of the outcome
• Clear logic that will lead to the same choice for the same information
• Relevant and reliable data with established credibility
• Need, desire, and/or goal clarity that enables assessment
• Adequate alternatives that can deliver on required outcomes
• Inspires commitment for needed action

Making good decisions more likely

When considering what makes for a good decision, it is easy to focus on the outcome. In reality every decision contains some amount of uncertainty. Outcomes are not guaranteed. Achieving a good or bad decision outcome may not reflect directly on the quality of the decision. However, if there is consistent failure to achieve projected results, it is probably time to improve the quality of the decision process.

Here are some ideas that can help increase decision quality:

1. Capture the information and your thought process. Our memory of how we decided can be heavily influenced by emotions. Hindsight makes events appear more predictable than they really were. Consider using decision support software or other decision making tools.
2. Learn from others. Find someone who has made this decision (or a similar one) in the past and use their experience as a starting point.
3. Reflect and review the process after the results. What worked and what didn't? Consider how the process could be improved. What would you do differently?
4. Monitor progress. Results for high value decisions usually take time. Look for evidence that supports or undermines the desired outcome. New evidence could justify a new decision.

Assessing quality can provide a number of advantages for group decision making. Here are some additional process tips that can support making good decisions.

Return from Making Good Decisions to Decision Making Process