Toyota’s quality improvement changes aren’t enough

Toyota sales have taken a beating after the quality problems of 2010. May 2011 sales declined by almost a third compared to May 2010. May 2010 sales were barely better than those during the depth of the great recession in May 2009 (a 45% drop from 2008). In comparison, Ford’s sales have have actually increased by a third since 2009!

So, it might be worth looking into the root causes of Toyota’s quality problems. In the aftermath of all the recalls, Mr. Akio Toyoda laid out a four step plan to fix Toyota’s image. All four steps dealt with either improving the actual quality control or safety checks. The conventional wisdom seems to maintain that if there are quality problems, increase quality control or safety checks (Toyota advisory panel says safety management changes aren’t enough: “

Automotive News reports that management changes made by the Japanese automaker haven’t gone far enough to fix all that ails Toyota. For example, the panel, which is led by former Transportation Secretary Rodney Slater, feels that Toyota decision-making is too centralized to Japan, which could mean that individual regions don’t have the flexibility to act on issues in a timely fashion. Further, the group found that even with recent management changes, it’s still too difficult to identify a clear chain of command in the Toyota safety department. The panel reportedly also referenced ‘skepticism and defensiveness’ towards outside safety complains as a reason issues weren’t solved sooner.”

Clearly, large organizational problems, such as authority to make decisions etc need to be fixed. However, these problems have not changed for decades. Every time I have worked with large Japanese organizations with large US presence, I have always found significant cultural conflicts. Why did the quality problems start now as opposed to 20 years ago? Is it because of quality control?’ Clearly, Toyota thinks so. In fact, Toyota made a big deal about the new devil’s advocate policy that puts in an additional round of quality control AFTER the car has been fully designed and tested.

The ‘devil’s advocate’ approach to vehicle design is a key element of the new Toyota. Under the plan, the company gives engineers four extra weeks to tear down and evaluate new vehicles.
The goal is to use the car in ways the owner’s manual doesn’t even consider. That’s because Toyota found out the hard way last year that customers use cars in unpredictable ways. It traced some unintended acceleration cases to gas pedals being jammed by stacked floor mats — an ill-advised practice for which Toyota engineers didn’t plan.

As we remarked earlier, the problem does not seem to be arising from a lack of quality control. In fact, Toyota is legendary in its Toyota. If unpredicted customer behavior can be found after the design is completed, why can it not be found before? It would be much easier to fix problems during design rather than after the production has started! As the article above pointed out, the push to fix problems after the design is complete cost a lot more and are never very effective.

Toyota and the supplier switched to crisis mode. They designed a new pinchless wiper and the Yaris still made the scheduled start of production in November.
“We really had to push hard,” recalled Katsutoshi Sakata, Toyota Motor Corp.’s lead executive for quality research and development.

Toyota has a very thorough manufacturing process (the Toyota Way and the Toyota Production System). So, the likely problem lies with design… As Knowledge@Wharton from the Wharton School of Business pointed out, the key challenge is in the ability to manage increased complexity of new automobiles.

MacDuffie: And that creates tremendous demands on the designers, right?
Fujimoto: Right, it’s a nightmare for the designers. You have to take on all these constraints. It’s like solving gigantic simultaneous equations involving structures and functions. For example, with the Prius recall, the problem resulted because Toyota tried to improve fuel efficiency and safety and quietness at the same time through a nice combination of very powerful regenerating brakes, plus the latest antilock brake system, plus the hydraulic braking system.
But the relationship between the three kinds of brakes changed with the new design, and then drivers could have an uneasy experience when there was switching between the different brakes a little bit…. Toyota failed to see this problem in the right way, at least in the beginning.

The Toyota design organization seems to be based on the traditional model of apprenticeship. Where engineers go through years to learn about the design ecosystem (suppliers, their capabilities, integration into overall design, etc.). However, the rate of technology change has increased tremendously – especially when one considers electronics and computers. Since cars are increasingly computerized, waiting for years to learn the system just does not work. Hence, the quality problems are most likely design and R&D culture problems. Toyota recognizes the problems, and has taken steps to reorganize its R&D departments (See Behind the scenes at Toyota’s R&D center Part I and Part II). As the the advisory panel rightly pointed out, Toyota needs to find ways to bring the Toyota Way to R&D.

In addition, the company should apply its vaunted Toyota Production System and Toyota Way principles outside manufacturing, the report said.
Toyota’s manufacturing error detection, based in part on going to the source of the problem to understand root causes, is “unhelpfully narrow,” the report said.
This manufacturing approach is “not applied rigorously enough” in vehicle design, corporate governance, customer feedback and regulatory affairs, the report said.

FYI – This article is not meant to be a criticism of Toyota or their products. In fact, I am a proud owner of a Toyota vehicle. The article is intended to help us learn about challenges in R&D management…

Article first published as Toyota’s !uality Improvement Changes Aren’t Enough on Technorati.

3 V’s for a successful new Venture

I have seen many a new ventures flounder because of a lack of focus, a clear plan and a cohesive culture. In the battle to get revenues and become cash flow positives, many entrepreneurs get far away from what they intended to build.

Here are three V’s to keep ourselves on track and help us succeed:

Vision: Define your vision of where the venture will go.
Vantage: Define your beliefs and view points of what will make the vision a reality.
Values: Define your organization’s culture and values that will help you get to the vision.

Vision:

Is a realistic, credible, attractive future for [an] organization. A clear vision has many advantages (described below). You can change the vision as many times as you like – but not having one will mean you will chase every opportunity however irrelevant to your vision or how much it might take you away from your goal. The most critical commodity in a new venture is the founder’s time and energy. A clear vision will help you ensure that resource is used wisely.

Here are some of the benefits of having a clear vision:

It attracts commitment and energizes people.
A vision allows people to feel like they are part of a greater whole, and hence provides meaning for their work.
It establishes a standard of excellence.
It bridges the present and the future.

A great example of a leader with a vision is Steve Jobs. His vision was that there would be convergence between phones and music players. It took him seven years from the development of iPod to get to iPhone. However, the vision guided product development iterations and experimentation till the right solution was developed.

Vantage:
Having a vision is necessary, but not sufficient. There are many ways to get to the desired vision. It is important to define a few guiding principles that you are going to follow to achieve your vision. These guiding principles will help you decide what not to do – probably even more important than deciding what to do. These guiding principles will help you prioritize between conflicting priorities.

Research shows that vantage and mindset defines how an organization develops its products. It is best to think about vantage from an example. Apple’s vantage is user centric design and on exquisite industrial design. So when it comes deciding on how to achieve the vision of a converged device, it clear to everyone on the team that the device has to have the absolute best user interface possible. In fact, the focus is so sharp if there is a conflict between user experience and industrial design, user experience wins. It is important to remember that whether you define it or not, there is always a set of beliefs that are guiding your actions. Defining them helps the entire organization drive coherently towards the vision.
Choosing the right vantage is extremely important. As the recent article in Arstechica pointed out, Apple’s focus on user experience drove them towards total control of the product solution (hardware, software (OS), and service (iTunes)). This worked well for most of the hardware centric products, but not so well for the cloud-centric services (such as iCloud). Google on the other hand, is focused primarily on the internet and may have a better chance of succeeding with cloud services.

Values:
Once you have a set of guiding principles, define what cultural values you will inculcate in your organization. Think about how will your vantage transfer from you into the entire organization.

There are no right or wrong values or cultures. It is important to decide on what values are important to you and follow those consistently. Organizations often take on the persona of their leaders. It wills save you and everyone else a lot of trouble if you are aware of the values you are cherish. Fun organizations have become a buzz word lately. Should you spend your scarce resources to make your organization fun? Are you going to want to control all aspects of the organization or do you prefer to be hands off? How will the team be rewarded – for individual achievement or for team success? A few minutes of thoughts along this will ensure you hire the right kinds of people to support you.

Article first published as 3 V’s for a successful new Venture on Technorati.

Too Big to Succeed?

A quick note about an article with some interesting data inf CFO.com (Too Big to Succeed?). The overall conclusion is pretty interesting:

Research on nonfinancial companies finds that larger companies typically grow more slowly and earn lower returns on capital.

The author has done a pretty extensive analysis:

Our capital-market research on the 1,000 largest nonfinancial U.S. companies, excluding those that were not public for the full decade of the 2000s (net sample size: 748 companies), indicates that size does indeed matter — but more as a shortcoming than an advantage.

Here are the detailed results (should be useful for any benchmarking):

The overall lessons are quite intuitive:

Why do large companies tend to underperform smaller companies? The specific reasons vary greatly, but there are a number of common themes:
• Organizational distance from executives to the people running each business inhibits use of full and objective information in strategic decision-making at the top and tends to slow down the decision processes at the bottom.
• Managerial reliance on performance against budgets lessens the intensity for delivering true continuous improvement at the front line and introduces managerial stumbling blocks such as “sandbagging,” “hockey-stick plans,” and “spend it or lose it.”

Nokia’s troubles arise from mismanagement not lack of innovation

Nokia has been in the news quite a bit lately. Nokia’s new CEO Elop was recently quoted in the WSJ saying:

Rising competition in China and Europe has forced Nokia to cut its prices, contributing to the second-quarter sales miss, Mr. Elop said. He singled out Google’s Android operating system as a major source of Nokia’s current troubles in both regions

In fact, Elop announced on May 31 that company’s second quarter of 2011 will come in substantially below expectations. The outlook is so dismal that Nokia disavowed its forecasts for the rest of the year. We have speculated in the past that Nokia’s management bureaucracy stifled innovation. The new article in Business week (Stephen Elop’s Nokia Adventure) has a lot more empirical evidence:

For a moment, Elop, 47, lays into the complacency he sees settling over the company. When he asks how many people in the crowd use an iPhone or Android device, few hands go up. ‘That upsets me—not because some of you are using iPhones, but because only a small number of people are using iPhones. I’d rather people have the intellectual curiosity to understand what we’re up against.’ Finally, after emphasizing that he believes mismanagement—not a lack of innovation—is what ails the company“

From hi-fi sounds to water proof phones, interesting innovations abound at Nokia:

On his visit to Salo, Elop was shown a hi-fi speaker that encloses a phone, giving a richer sound. Another engineer handed him a phone and asked him to toss it into a tank of water. When the engineer dialed its number, the device, still submerged, rang. A nanoscale coating makes electronic parts water-resistant. “

However, few of the innovations came to market because of management:

This kind of stuff has been sitting around people’s desks, because it’s too hard to get anything done around here,” Elop says. “If we can get some of this to market—that’s what gives me confidence.

Mismanagement seemed to start from senior leadership (Ex-CEO):

Recent history has hardened employees to the opportunities of a new era. “Under OPK, you could work on something for four years” before a decision was made to halt it, says Tuomas Artman, a former employee and Nokia contractor. OPK is Olli-Pekka Kallasvuo, the former CEO frequently accused by ex-Nokians of running a politicized, indecisive organization.

Even in well managed organizations, it may be necessary to halt ongoing R&D projects if the market realities or competitive pressures change. However, the leadership should be able to communicate these changes to employees and need to be held accountable for their decisions. This was definitely not the case at Nokia:

On Sept. 21, his first day, Elop sent an e-mail to every employee asking what they thought he should change, what should be left alone, and what they feared he wouldn’t understand. There were more than 2,000 responses, mostly about accountability. (One of Elop’s favorites: “At Nokia, everybody and nobody is accountable for nothing.”) Elop personally responded to each one, and word got around that the new boss was serious about addressing their concerns.

An interesting side node about Elop’s interesting change management style: He clearly has engaged the employee base, shown a sense of urgency and personal commitment. Many of the traits we have been discussing about effective organizational change. The real challenge for Elop is to instill accountability in the organization (more on it below). Back to the R&D management, Elop quickly recognized that Symbian was a significant problem:

Most of these problems could be traced back to Symbian. Never beloved by users, it became hopelessly buggy as Nokia tried to make the 10-year-old dog pull off iPhone-like tricks.

Nor did Nokia have a coherent effort to develop a developer community for Symbian:

And while Apple and Google have created software tools that help outside developers to easily create apps, Nokia’s equivalent tools gave developers fits. “Developing for Symbian,” says Artman, the former Nokian, “could make you want to slice your wrists.”

The root cause of this seems to be a lack of a single driving vision (like that of Steve Jobs) to help prioritize what features to focus on:

Until last month, the company hadn’t delivered a single new smartphone on time or without major software glitches since 2009, in part because of delays as scores of different hardware teams lobbied to get their pet capability—a new camera, say—built into Symbian.

Another root cause is the lack of focus on user experience. The amazing factory and ability to crank out cutting edge hardware seemed to treat software as just one component. This lack of integration allowed Apple and Google to gain market share:

And while Apple and Google focus on making one operating system to power a wide variety of devices, software at Nokia had been seen as just one more “component” to enable hardware teams to craft their latest models. “The terminology shows the mindset,” says Mark Wilcox, a former Nokia engineer. “The focus was on the phone, because Nokia had this amazing factory that could crank out 100 million units a year if you got a hit.”

The last root cause (at least for this post) seems to be a lack of R&D portfolio management processes. As the link shows, the lack of results was despite large R&D investments. In fact, Nokia invested almost 6.2B Euro in Symbian in 2010 – more than 10 times the total R&D budget at Apple. The fact that the company had no visibility into the product pipeline or the R&D portfolio is even more shocking:

Elop drew out what he knew about the plans for MeeGo on a whiteboard, with a different color marker for the products being developed, their target date for introduction, and the current levels of bugs in each product. Soon the whiteboard was filled with color, and the news was not good: At its current pace, Nokia was on track to introduce only three MeeGo-driven models before 2014—far too slow to keep the company in the game.

So, instead of having a live dashboard to look at status of the entire R&D pipeline, the CEO had to collect information through interviews and phone call. I wonder how accurate that information was! As if that is not enough, the chief development officer was also unaware of the development status:

When they finally spoke late on Jan. 4, “It was truly an oh-s–t moment—and really, really painful to realize where we were,” says Oistämö. Months later, Oistämö still struggles to hold back tears. “MeeGo had been the collective hope of the company,” he says, “and we’d come to the conclusion that the emperor had no clothes. It’s not a nice thing.”

I wonder who is going to be held accountable for a complete lack of R&D portfolio management! May be that can be the first order of business for the new CEO… May be we can suggest our integrated system for R&D management? 😉 Back to R&D management: clearly, Nokia needed to remove unprofitable projects from their portfolio and that is what they decided to do (illustration via engadget):

Overall, this looks like a good R&D strategy from Elop: Eliminate low-return R&D and focus on core business of low cost phones:

Windows-based smartphones are the first stage of Elop’s three-part comeback plan. One huge incentive for dumping Symbian was to cut the company’s bloated costs. With an estimated $1.4 billion annual savings from discontinuing Symbian, he says he will invest more to protect and build Nokia’s massive low-end phone business in emerging and yet-to-emerge nations in Asia and Africa, which brought in 33 percent of Nokia’s sales in 2010.

The third priority seems to be investing in disruptive innovations by setting up skunkworks:

It’s a fully sanctioned skunkworks, with teams in Helsinki and Silicon Valley, staffed by top technical talent from the discontinued Symbian and MeeGo efforts, especially MeeGo. That initiative began when Nokia hired a crew of inventive open source evangelists in 2009 with orders to dream up entirely new devices. A few months later they were reassigned to develop a replacement for Symbian. The goal, as Elop told a group of engineers in Berlin on Feb. 29, is once again to “find that next big thing that blows away Apple, Android, and everything we’re doing with Microsoft right now and makes it irrelevant—all of it. So go for it, without having to worry about saving Nokia’s rear end in the next 12 months. I’ve taken off the handcuffs.

The key challenge here is going to be the same – effective R&D portfolio management processes. Even if there are disruptive innovations in skunkworks, industry’s record of successfully integrating them in developed products is weak at best. Hopefully, Elop’s superpower will help him through this challenge:

“It was classic Stephen,” says Myerson, who worked for Elop at Microsoft. “His superpower isn’t his great intuitive judgment. It’s his amazing ability to create a transparent, fast process that reasonable people can feel good about.”

Article first published as Nokia’s Troubles Arise From Mismanagement Not Lack of Innovation on Technorati.

R&D Implications of Microsoft’s Takeover of Skype

Summary: For Microsoft to benefit from the Skype acquisition, they will have to integrate Skype and its R&D into Windows Phone OS. Microsoft has a history of successfully integrating acquisitions (e.g PowerPoint). However there have also been some missteps (such as Danger). One of the reasons for the failure of Microsoft Kin (a derivative of the Danger Sidekick) was failure to manage R&D. This is a highly competitive market place and Microsoft will have to be very vigilant to ensure success.

Microsoft recently bought Skype for $8.5B for Skype. A lot has been said about how the price paid and the strategic fit such as (See ArsTechnica):

When the Wall Street Journal reported last night that Microsoft was going to buy Skype, the response was puzzlement. Though Skype has some value, the estimated $7 billion-8 billion valuation was unfathomable. Microsoft has now confirmed the purchase and held a press conference to announce the takeover. The morning after the night before, is it making any more sense?”

There has been some speculation that the pressure to purchase Skype came from Bill Gates (see DailyTech):

Well it turns out a lot of the pressure to buy Skype originated from Microsoft founder and tech icon Bill Gates. We were the first to note Mr. Gate’s ties to Silver Lake Partners, one of the principle groups that profited from the Skype acquisition.

Knowledge@Wharton has a different perspective on this acquisition now that it is complete (What’s Behind Microsoft’s US$8.5 Billion Takeover of Skype?): First a bit of the background. Microsoft can hope to leverage Skype’s technology to get an edge in the smart phone market:

This one makes profound sense for Microsoft. Of the three major players in cell phones, they are the third. They were the first in, and some people would say they’re now the third, after Apple and Google. If they’re going to succeed, they need to have something that offers some unique value.

Skype’s technology can provide an edge, if Microsoft can execute:

This [acquisition] may do this. If they can effortlessly merge Skype with the phone, then they have the best videoconferencing phone in the business. If they can effortlessly merge [Skype’s] tech services with their own instant messaging service, then they have the best instant messaging system in the business.

However, as eBay can point out, integrating Skype is not easy. Skype is peer-to-peer communications, but most large corporations (eBay and Microsoft) have a centralized infrastructure. This change is not going to be easy:

Skype is a peer-to-peer instant messaging system. Peer-to-peer allows a buyer and a seller to completely bypass eBay. If eBay wanted Skype as a commercial infrastructure system, it would have had to be modified. If they acquired it without the appropriate intellectual property development team, it was of no use to them whatsoever. Again, I can’t back this up. If it’s true, I’m sure there are people who can back this up.

As the article rightly pointed out, integration of acquisitions is risky:

Acquiring high-tech companies, especially when there’s a great difference in culture or technology, can be a very risky play.

Although Microsoft has stepped up its M&A activity, the results have been mixed – especially with large acquisitions such as aQuantive ($6.3B) where integration is likely to be more difficult. Furthermore, Skype is not a profitable company despite having more than 100M subscribers. So, integration Skype to generate previously untapped value is going to be key to success. What can R&D management can do to extract value from this acquisition?

Define a clear R&D vision and strategy for merged group. Unless different cultures are forced to collaborate to achieve difficult goals, they will have trouble working together. As Steve Jobs and Apple have pointed out, there is no substitute to a comprehensive vision.
Setup clear portfolio balancing processes that can overcome bureaucracies and distribute resources to “new” entities. One of the key problems that plagued Kin is the product portfolio management process that depended too much on executive sponsorship.
Align strategy with a clear product management across entities. As we have discussed in the past, corporate mindset has a significant impact on how products get managed and launched. It is likely that Skype’s culture is quite different from Microsoft.
Use concrete metrics and a strong project management process to ensure the project remains on track. Both will be critical to getting over the not-invented-here problem that plagues all acquisitions.

Please see the Microsoft Kin case study for more details.

Article first published as R&D Implications of Microsoft’s Takeover of Skype on Technorati.

The Collaborative Organization: How to Make Employee Networks Really Work

I normally like articles in the MIT Sloan Management Review. Here is an exception – at least as far as R&D management is concerned – The Collaborative Organization: How to Make Employee Networks Really Work. However, I think it teaches us a few things anyway. The overall proposition is quite good – collaboration between employees is good for innovation:

The traditional methods for driving operational excellence in global organizations are not enough. The most effective organizations make smart use of employee networks to reduce costs, improve efficiency and spur innovation.

We have learned that project networks that leverage informal networks between employees are beneficial to productivity. We have also learned that there are many barriers to R&D productivity originating from networks and connections between R&D employees. These are especially important as R&D teams becomes increasingly multi-organizational and virtual. However, the article seems to zero in on network analysis as a way to improve collaboration:

Executives should analyze employee collaboration networks to discover how high-performing individuals and teams connect. Networks should be designed to optimize the flow of good ideas across function, distance and technical specialty. Network analysis can show where too much connectivity slows decision making.

I have nothing against network analysis. However there are many lower hanging fruits to boost R&D productivity and innovation. Network analysis is at best a snapshot in time. R&D networks are built over long term. Even if one can decipher networks across multi-location R&D teams, how does one change them? Force R&D employees to work with new people? What do you think…

Does Strategic Planning Impede Innovation?

Executives are increasingly focusing on innovation in response to the great recession. Executives also depend on strategic planning to navigate through tumultuous times. In fact, there has been very influential research showing that planning can actually help organizations learn and improve. So what is the impact of strategic planning on innovation? The article Does Strategic Planning Enhance or Impede Innovation and Firm Performance? from the Journal of Product Innovation Management has some timely insights:

Does strategic planning enhance or impede innovation and firm performance? The current literature provides contradictory views. This study extends the resource-advantage theory to examine the conditions in which strategic planning increases or decreases the number of new product development projects and firm performance. The authors test the theoretical model by collecting data from 227 firms.

The paper has empirical evidence suggesting that both strategic planning and innovation are good for organization. We should keep in mind, however, that the paper appears to use new product development (NPD) as innovation. That is probably not the accepted definition of innovation, but we can use it as an approximate stand-in…
The other interesting information is that the firms with organizational redundancy (larger firms), gain more from strategic planning than smaller one. However, strategic planning actually reduces NPD (or innovation). The evidence seems to support that improvisational organizations (as opposed to planned) are more innovative. Also, lack of strategic planning seems to encourage more outside-the-box thinking. May be Wall Street is right in thinking that larger firms can not be innovative… However, the article also shows that larger firms with large R&D budgets are able to overcome the impact of planning and still succeed.
So what are the lessons for R&D managers:

Make strategic planning flexible so that innovation and improvisations is allowed.
Develop resource allocation processes that clearly communicate to employees that innovation is important.
Encourage employees to access innovation from the outside (in form of technology or user experience).

Here is the overall message:

Finally, managers must understand that managing strategic planning and generating NPD project ideas are beneficial to the ultimate outcome of firm performance despite the adverse relationship between strategic planning and the number of NPD projects.

Article first published as Does Strategic Planning Impede Innovation? on Technorati

Sense of Urgency Critical to Driving Change

I have seen many a process improvement and organizational change projects fail. In fact, studies have shown that 90% of cost cuts are reversed within 3 years. In fact, 60% of the organizations are dissatisfied with their return on process improvement tools such as six sigma. Here is another interesting insight from Forbes about change management (The Biggest Mistake I See: Strategy First, Urgency Second) :

Some people think a “burning platform” gets people urgent and committed to change. Other leaders think that they can announce the change and then leave it to everyone else to make it happen. Both are mistakes. But the biggest one I’ve run into is when someone says, “We’ve done our research, we have our strategy, we’re ready to get everyone on board.” In the video below I talk about why this approach just doesn’t work for large-scale change, and what you need to do instead: start with a sense of urgency.

The message is clear, a strategy is not enough. Change requires a clear purpose, vision and urgency from senior management. Change also requires continued engagement from executives, leaders. Finally, change requires clear accountability, metrics and reinforcement systems to ensure it sticks.

Corporate Performance Management in volatile times

Here is an interesting article from Research & Technology Executive Council about consideration for improving and managing corporate performance:

Corporate performance outstripped many people’s expectations in 2010 and, as a consequence, senior executives have been given oversized 2011 growth goals that are consistent with a recovering business cycle. The problem is that we are not yet in the midst of a runaway recovery.

Although the article is focused on financial performance management, the ideas discussed are also valid for other disciplines (including R&D). Here is what I learned about robust performance management process:

Target setting: Use stretch targets judiciously. Take into account factors that the team can control and those they can not. For example, if the entire market declines by 20%, the stretch goal of increasing revenues by 20% might not make sense. Another approach would be to use competitive benchmarks to decide targets (Stretch goal is at least 10% better than my closest competitor).
Metric selection: Right metrics that support the targets are extremely important. As we have seen in the past, there is a strong temptation to fudge the metrics to get rewards. We should focus on developing dashboards that encourage an appropriate level of risk taking and reward good decision making.
Use of IT to provide visibility: Automated performance management systems to help teams focus on performance targets and metrics is useful. IT can help with dashboards as well. However, one has to balance those benefits with costs of setting up and feeding/maintaining such systems. Many times IT leads companies focus too much on the logistics instead of the actual results.
Creation of a ”performance management culture”: Leaders should help setup a culture that uses performance management to drive organization success and discover hidden sources of value. The process should probably incorporate non monetary / financial rewards.

Article first published as Corporate Performance Management in volatile times on Technorati.

How to Evaluate a Training Program

Training is a great way to improve the performance of R&D teams. However, the value delivered by training remains quite hard to measure. We have discussed some approaches to rigorously explore the value delivered by training programs. The Great Leadership blog has a useful checklist on how to evaluate the success of training programs using web-based surveys (How to Evaluate a Training Program):

Since the dawn of time, when early trainers were training their clan members how to improve their hunting and gathering skills, training organizations have struggled with how to measure the impact of their training programs.

Here are my takeaways:

Use the same web-based survey platform to evaluate all training courses
Ask questions about:

Course logistics (instructor, food, venue, etc);
Course material and learning
Course value and business results: Ask participants about the impact of training on concrete business results such as time-to-market, margins, cycle time, costs, etc

Ensure questions are consistent across all courses so the results can be aggregated
Administer survey immediately after the class is complete (but not in-class because people tend to feel pressure from trainers being present. Also, survey should be administered by someone other than the trainer to remove biases)
Administer survey section 4 90 days after the training to measure actual value delivered
Ask managers to validate section 4 results (through the same survey).