Metrics that measure effectiveness of ongoing R&D have been difficult to obtain. Four types of R&D metrics have traditionally been used:
- R&D Investments / Expenses
- Total R&D Headcount
- Total R&D Expense
- R&D Expense as % of Revenue
- R&D Expense increase/decrease from prior year
- R&D Expense compared with peers/industry average
- …
- Project execution status
- Performance relative to plans (costs and schedule)
- Concept to Market Time
- Number of Projects in the Pipeline
- …
- Historic results-based Metrics
- Fraction of Revenues from New Products
- Number of Patents generated
- Number of Papers Published
- Customer satisfaction with new products
- ROI-based metrics
- Return on Innovation Investment
- Target NPV for each new product
- …
The problem with most of these metrics is that they are not actionable. As we discussed earlier, R&D effectiveness needs to drive real business results. It is hard for managers to take concrete actions using these metrics to improve R&D effectiveness. All metrics need to be able to aggregate information across the R&D portfolio to help managers see trends and make decisions. Good metrics should allow segregation of management decisions into individual project level actions.
The first two types of metrics are not directly related to ongoing R&D and its projected results. For example, what would happen if we increased our headcount or reduced it? There is no easy way to understand the impact on R&D pipeline. Will it improve revenues? When? Are there key technologies that will take time to develop no matter how much money you invest? It is not easy to tie headcounts to R&D results in the future.
Historic results-based metrics such as number of patents generated are all approximate indicators of R&D performance several years in the past. Managers can gain little direct insight into ongoing R&D effectiveness from these metrics. Nothing a manager can do now will impact number of patents issued or number of papers published. More importantly, it might be better for
ROI-based metrics tend to work better with product development projects near delivery, but they are very hard to use for early stage development. Furthermore, it is hard (if not impossible) to develop ROI on technology development effort that might impact a feature in several different products (think a new type of metal that can be used in different types of cars). ROI computation becomes even more difficult for disruptive innovation.
Over the next few weeks, we will discuss new kinds of metrics that can help managers improve R&D effectiveness…