By Tim Nadreau, Ph.D.
The Issue:
Many economic consulting firms use data to answer a question being posed to them, couching it in terms of “data driven decisions.” They provide you with some data, and a story or two that corroborates your preconceived, or not so preconceived, ideas. You lean into that data—or don’t—and make your decision, a “data driven decision”. However, there is a flaw in this approach: it is sterile and often misses the underlying cause of the data generating process. Looking at a trend in data, and even being able to tell a good story with it, might help move decisions from the “gut instinct” category into a more rational category, but let’s think of counterexamples to this type of decision-making process—stock traders. Nassim Taleb has a lot to say about this in his books.
Traders are highly sophisticated in their trading algorithms. They look at all the historic data—even incorporating real time data—and they still can’t “beat the market.” The rule of thumb is that 80% of traders lose money, 10% break even, and 10% make money consistently. Data driven decisions might be superior to some other types of decision making; but then again maybe not.
"Well-reasoned doubts are good for economics. Neither theory, nor data, nor mathematics can fully
resolve them. ...Economic behavior is more complex than our thoughts about it; our thoughts,
however, are more comprehensive than standard theory; and standard theory is more
comprehensive than mathematical economics. Each of these has its advantages. What is known
from all of them is nevertheless subject to doubts. Economics would be better if we would
substitute reasoned doubts for our parochial economic doctrines.”
-T. W. Schultz, 1986
The Theory:
This week’s theory section deviates from our traditional graphs and descriptions. Here we focus on some definitions. Intelligence gathering is broadly broken into two categories: surveillance and reconnaissance. These two types of information are synthesized, along with experience, to arrive at intelligence. The problem is, even if you have the intelligence that matters, you may not be the ultimate decision maker—the “boots on the ground” as it were. At this stage you have intelligence, but it is not yet actionable because it may be in the wrong hands. Operations and intelligence only benefit one another in so far as they communicate with one another. Faith without works is dead. Works without faith are but dirty rags.
Actionable Intelligence happens when a comprehensive understanding of a problem can be coupled with a policy (not necessarily a government policy) that addresses that problem with minimal fallout. Remember, we don’t believe in “solving problems,” as every action has an equal and opposite reaction. We believe in tradeoffs. The goal then is to reduce or eliminate a constraint or set of constraints without creating a bigger problem somewhere else. Green zones were a great way to prevent urban sprawl but the unintended consequences—the unforeseen tradeoff—was extremely costly. These green zone policies were the result of data driven decisions, possibly ideological driven decisions, but they were not based on actionable intelligence.
Let’s delve into the concepts of surveillance and reconnaissance for a moment. Surveillance reflects persistent monitoring of a variable. We do this with GDP, Inflation, oil prices, LMI, you name it. Even wheat production falls under the purview of surveillance. This is where the “data driven” folks live, usually pulling various data set from government databases and synthesizing them, unpacking some information to generate comprehensive data sets, and then making that information user friendly. This is critical work and with improvements in technology, more and more variables fall into this surveillance category every year. But go back to the stock traders. Something is missing from the equation. Often it may be as simple as economic theory is missing from the data, but more frequently it is the on the ground knowledge that is missing. Call it industry knowhow, or something that is just in the air, there is a validation of circumstance that the data is not capturing. Part of this is timeliness (discussed in other posts) and part of this is knowledge and information that isn’t and cannot be quantified. How close a researcher is to a revolutionary breakthrough in bioinformatics, how close the Recon Team is to a fully functional CGE model—these are things outside the purview of surveillance. They require reconnaissance. This typically means needing to embed researchers, analysts, etc. within the industry trying to address a problem.
Reconnaissance is on the ground information gathering. The Musk-Twitter debate centers squarely on this distinction right now. Because twitter didn’t have bot surveillance, they now have to do in-depth reconnaissance to ascertain the information about “real” accounts. The surveillance and reconnaissance will result in actionable intelligence—a sale or a failed sale of the company. When we embed with economic development teams or industry groups, we spend a considerable amount of time learning and understanding the unique attributes of the problem and industry of analysis. This is the most difficult and least profitable part of the job. It often requires us to learn new terminology, understand the historic and stochastic processes that operate within the industry’s culture. This slows the pace of analysis down. To alleviate some of this we lean on industry professionals and ask them to take us under their wing until we become knowledgeable enough to combine the economic theory we have with the surveillance and reconnaissance information to build actionable intelligence.
Our View:
Actionable Intelligence requires multiple dimensions of information and needs to extend beyond surveillance, particularly in the realm of economic development. A solid economic evaluation should have: 1) some operational dimension—often referred to as policy analysis, 2) solid surveillance AND reconnaissance information, and 3) an economic theory that combines information into intelligence. A fourth and often overlooked component of economics is ex post analysis. This is an area routinely ignored at every level. If we got the operational decisions “right” then we brag about it and maybe learn from it. But no one seems to want to learn from failures. This is where we at Recon thrive; we love studying the economics of historical policy failures. This is something we learned from papers at the Federal Reserve Bank of Minneapolis where models were used to advise policy, and afterwards the models were evaluated for accuracy.
Data driven decisions will help you move the needle from making good decisions 3% of the time to 10% of the time. These reports are easy to identify because they will drown you in data, while giving you confidence in their description of that data. Actionable Intelligence will help you move beyond that, improving profitability, policy design, and confidence in your staff and clients. Contact us to learn more.