An interesting article from Sloan about How to Manage Risk (After Risk Management Has Failed). The article is talking about risk management in financial companies, but probably applies even more to R&D.
Yet at these companies, and at others with comparable “cultures,” risk management apparently performed quite dismally. How could this be? We contend that the answer lies in the concepts and practices of traditional risk management, which tend to look for risk in all the wrong places. That is, failure did not stem from merely paying lip service to risk management or from applying it poorly, as some have suggested. Instead, collapse resulted from taking on overly large risks under the seeming security of a risk-management approach that was in fact flawed.
emphasis added
Overall message from the article is pretty straight forward:
- The traditional “frequentist” approach is based entirely on the historical record.
- The alternative “Bayesian” approach incorporates judgments to complement historical data.
- The Bayesian perspective provides more powerful and accurate results.
It is surprising that even many advanced R&D organizations delegate risk management to the quality control organization. Many quality engineers are removed from detailed R&D activities and identify risks mainly based on past history of similar products. So, as the article points out, R&D risks are identified based on historical failures (frequentist approach). However, new products often bring new failure modes that are not anticipated from older products. For example,
electronic accelerators in Toyota resulted in uncontrolled acceleration. Quality engineers had failure modes and effects only for legacy mechanical accelerators. Hence, traditional risk management would not be able to identify these new failure modes.
A better way to address this situation is the integrate
risk assessment into the R&D process directly. Engineers can then identify failure modes that they are concerned about and estimate risks based on their judgement (expert opinion). Results from testing can then be used to validate and enhance risk judgments using Bayesian statistics. This concurrent risk assessment is much better than increasing on quality control
after the design is complete. Hence, this approach (unlike
Toyota’s Devil’s Advocate Process) will improve quality AND reduce costs.
