Making Data Work For You

19 May 2010 Sandeep Mehta

Here is an article from Forbes talking about how to make data work for you. The underlying presumption is that the data necessary to make management decisions is lying around in the company in different forms – the challenge is to mine it effectively and prepare the right kind of report.

The amount of data inside corporations is exploding. In fact, there has never been such a wealth of information available in the history of business, and there has never been the vast array of tools to dissect it.
CIOs are generating reports for business leaders that slice the data horizontally and vertically, project growth, calculate productivity and profitability, and compare all of this historically and competitively. They are even pulling out tidbits of data that may appear randomly and building models based upon recurrences that escaped notice by even the most astute teams of experts. But is all this stuff right?

The key challenge according to the article is to find the right kinds of skills that marry IT and business understanding to generate these reports. And that unfortunately, those skills are hard to come by.

You’re talking about creating a data model, but aren’t models inherently flawed? Look at what happened to Wall Street in 2007.
The problem with models is not so much that they’re difficult to create. It’s whether everyone involved agrees with the same semantics. If you want a revenue report or a profitability report, you need to figure out what should be included. Once you have clarity on that, the other steps are much easier.
Do these skills exist?
They’re not very common. We need a completely new skill set for the CIO and the IT department. They have to speak the language of requirements for the application as well as the business language of reports.

In my experience, getting hold of the right data to make effective R&D management decisions is a big challenge. The problem is not only that the data is fragmented across multiple tools and databases, but also the fact that the tools all use different jargon and structure. It is difficult, if not impossible to aggregate data across these fragments to generate meaningful reports.

For example, if one wants to generate project portfolio status reports automatically, one has to mine across project management, requirements management and financial management systems. Each one of these systems maintain their data at different, disconnected levels of abstraction. Even if it was possible to mine data from each one of them, effort needed to link them across each other would be cost prohibitive.

What do you think? Have you used automated data mining / report generation tools successfully? Are there any lessons learned that you can share?

Leave a Reply