When it comes to settling disputes, a lawyer friend of mine likes to tell me, “bad facts trump no facts.”  In the business of economic policy research, that statement has a depressing ring of truth to it.  In fact, it even suggests a corollary.  Namely, if you want to hide the quality of your data, serve it up by the truckload.

The recent report by the Milken Institute on the biopharmaceutical industry, released in the same week as Eli Lilly and Company announcement that it will eliminate an estimated 575 positions, is a case in point.  The Institute has been at the forefront in researching what has come to be known loosely as the new economy – the wide spectrum of businesses that employ newly developed technology to produce a new generation goods and services that didn’t exist a few years ago.

There is no quibbling with the essential message of the report.  That is that biopharmaceuticals, or the production and development of medicines and other biological products, are an important source of high paying jobs in many states, including Indiana, both now and in the future.  It’s still a business, with risks and occasional failure, as Lilly’s recent announcement reminds us.  But it is a business that feeds off powerful trends in technology, affluence, and demographics that bode well for the future.

As a piece of research, however, the Milken report is much less impressive.  Its tables of state-by-state comparisons and neatly presented charts of trends are nothing more than an elaborate fiction.  It’s a fiction that is well crafted, quite plausible, and perhaps worthy of attention.  But it is fiction nonetheless.

Arguments about the details of research reports are things that only academics seem to get excited about.  But I’m picking on the Milken report to make a larger point about policy research as a resource for making decisions.  If research does not adequately distinguish between what we know, and what we don’t know, then it can fail us when we need it.

The Milken report certainly presents a wealth of detail on the economic footprint of the biopharmaceutical industry.  Its tables show employment, earnings, and output for two industry classifications, given by six and seven-digit classification codes, for every state for each of the last twenty years prior to 2003. 

That’s a pretty neat trick, when you consider that the newly established NAICS system of coding industries in the economy didn’t start producing data until 2001.  The older data, based on the now-retired SIC system, are not comparable, for a variety of reasons.  And, due to budget constraints, the federal statistical agencies that produce the data that we all consume have focused more on going forward with the new system, rather than recasting the old SIC data into the new NAICS format for the benefit of a few researchers.

The result has been a roadblock to any research that relies on long-term trends in the economy at any level of detail.  The Milken report isn’t fazed by that problem.  Its tables chart industries, defined as NAICS codes, going back to 1983, when no official data exist at any level of aggregation, national, state or local, exist prior to 1990.

Making up, or “refining,” data is not exactly unheard of in policy research, where the importance of the question at hand can often justify a little patching and cleaning of one’s dataset.  But in the case of the Milken report, the entire point of the study is made using a level of detail that the actual data do not contain.

That doesn’t exactly inspire outrage in the economic development community.  Indeed, a common reaction to research reports like the Milken Institute’s recent offering is an increased desire to have a similar report done for one’s own industry or community.  If you are comfortable making decisions based upon data created by computer algorithms, rather than by the economy outside your doors, then you are welcome to it.