Select Evaluation Methods

Methods for evaluating alternative solutions against established criteria can range from simulations to the use of probabilistic models and decision theory. These methods need to be carefully selected. The level of detail of a method should be commensurate with cost, schedule, performance, and risk impacts. While many problems may need only one evaluation method, some problems may require multiple methods. For instance, simulations may augment a trade study to determine which design alternative best meets a given criterion.

• Select the methods based on the purpose for analyzing a decision and on the availability of the information used to support the method. For example, the methods used for evaluating a technical solution when requirements are weakly defined may be different from the methods used when the requirements are well defined. Typical evaluation methods include:
  • Simulations
  • Engineering studies
  • Manufacturing studies
  • Cost studies
  • Business opportunity studies
  • Surveys
  • Extrapolations based on field experience and prototypes
  • User review and comment
  • Testing
• Select evaluation methods based on their ability to focus on the issues at hand without being overly influenced by side issues. Results of simulations can be skewed by random activities in the solution that are not directly related to the issues at hand.
• Determine the measures needed to support the evaluation method. Consider the impact on cost, schedule, performance, and risks.