Measuring Innovation

“You cannot manage what you don’t measure,” says a popular axiom in business. If you need to track the economy, [...]

February 5 2008 by Fayazuddin A Shirazi


“You cannot manage what you don’t measure,” says a popular axiom in business. If you need to track the economy, you need to measure the growth. Strongly guided by this notion, the US commerce secretary Carlos M Gutierrez recently released a report, calling for innovation measurement, which according to experts is an elusive driver of economic progress. The report known as €˜Innovation Measurement: Tracking the State of Innovation in the 21st Century Economy,’ outlines the initiatives to be taken in effectively measuring innovation.

Drafted by an expert advisory panel of business leaders including some of the stalwarts like Samuel J Palmisano of IBM, Steve Ballmer of Microsoft and George Buckley of 3M and few of the top US academics from Harvard and Carnegie Mellon universities, the report makes aggressive proposals to gauge and quantify innovation better. The US Secretary of Commerce commissioned the advisory committee in September 2006 for recommending ways to improve the measurement of innovation in the economy.

Most of the committee recommendations including comprehensive accounting of the effect of high-tech goods and services, measuring the increase in productivity due to increased investments in innovation, and expanding collection of data on innovation have been accepted in principle by the commerce department, except the major initiative on sharing statistical information and data from different statistical agencies for research purposes doesn’t seem to be getting a nod from the ministry for now.

Although Gutierrez noted that US today is more than 75 percent wealthier in terms of real GDP per capita than it was 30 years ago, largely attributable to productivity gains driven by innovation, he didn’t give any clear indication on how the agencies would share statistical information and data for further research, as required by the advisory panel.

A release from the Commerce department merely said that Secretary Gutierrez pledged to work with the Council of Economic Advisors, the Office of Management and Budget, the Departments of Treasury and Labor, and Congress to see whether a framework that meets everyone’s data confidentiality concerns is achievable, which experts believe is not a concrete assertion from the authorities.

The advisory panel encourages the statistical agencies such as Bureau of Economic Analysis (BEA), the Census Bureau and the Stock Exchange Commission (SEC) to pursue an agenda in support of the development of linkages between data sets both to improve data consistency and to provide a richer database for understanding and explaining innovation. Full implementation of such an agenda requires a new legislation, for which the advisory panel has sought support from the Commerce Secretary.

However, economists believe that the information can be effectively shared for research purposes without compromising on the confidentiality aspect. “There has been significant increase in our understanding of how to provide information without compromising confidentiality. On a technical front, I am confident we can do more by way of sharing information with researchers (not necessarily for agencies to share information with each other, which raises issues well beyond the scope of this discussion),” Asish Arora, a panel member and Professor of Economics and Public Policy, H. John Heinz 111 School of Public Policy and Management Carnegie Mellon University told CE Online.

Arora also feels that confidentiality concerns can be undermined and information crucial for studying innovation and its drivers can be accessed, if there was political will and motivation. “With political will, issues of confidential limitations and constraints can be resolved and the potential payoff thereby could be very large,” says Arora. Elaborating further, he says that many European countries- which are perfect examples for such inter-agency cooperation, have linked firm-based records with that of individual records, allowing researchers to study how people move across firms and how this is related to entrepreneurship and innovation. “And it is not very complicated to replicate it here,” he says.

Though there is no proper mechanism to quantify innovation, analysts have been mostly relying on the effects of innovation index known as Total Factor Productivity (TFP). Any growth in TFP levels is considered to be the growth in innovation. However, experts point out that the TFP levels might see a growth for variety of reasons, besides the increase in the innovation activity. “TFP may grow because of superior diffusion of the existing technologies and the measured TFP may also reflect other effects such as the vagaries of data collection. TFP though is touted as one of the most accurate tools to quantify innovation, it is basically a tool that measures the effects of innovation and not the innovation itself,” says Arora.

Besides the TFP approach, Arora says that patents are also used as a measure to track innovation over time across many firms and countries, even though this approach has several limitations. “Many firms use the fraction of sales method from products introduced fewer than five years ago, which does not give you any break-through,” he says.

Often, academics and economists have been using tools such as tracking the number of people engaged in R&D and the affiliated scientific research as a proxy for innovative output, though there are again limitations to this as well, say experts. “The current advisory panel did not opt to recommend an index, because there is no serious evidence on how different measures of innovation should be combined, either at the organizational level or at the aggregate national level,” Arora points out.

While all the talk on measuring innovation is seemingly very encouraging, quantifying it in actual terms is very difficult, says Arora. “Innovation measurement is a difficult task, particularly when one realizes that innovation is not merely technical advancement but it is also about improvements in organizational practices and new business models,” he says, adding: “Google is a perfect example of a new business model. It is not merely a story of better indexing and search algorithm; it is more so of a successful business model. I think the committee’s recommendations have to be tempered based on what is practically feasible and viable, to achieve the desired results.”

A great deal of research has to be undertaken on issues such as measuring the intangible in innovation, which according to Arora are the greatest challenges in innovation metrics. “Issues such as measuring transactions in intangibles, particularly technology licensing, patent licensing are very complicated, however, it can be done by studying the financial records of different firms. It is mostly a matter of getting this information and systematically organizing it,” he says. The outcome will be a huge increase in the ability to measure the extent to which the economy creates these intangible assets and how they are deployed, within the country and globally.