Bacon's Rebellion

Democracy Thrives in Sunlight

How Good Is Chmura’s Economics Data?

—

by

Peter Galuszka
in , , , , , , , , , ,

chmuraBy Peter Galuszka

In the 40 months since Robert F. McDonnell has been in office, the launch of many of the governor’s policy initiatives seems to be accompanied by a press release touting the supportive findings of a small, Richmond-based research firm named Chmura Economics & Analytics.

When McDonnell was pushing his transportation plan to come up with $3.4 billion road funding by eliminating the gasoline tax and increasing the sales tax, the Chmura firm was hired to research the impacts. The results were glowing: McDonnell’s signature plan would eventual result in 13,058 new jobs and $9.5 billion investment.

When McDonnell and his Transportation Secretary Sean Connaughton wanted a $1.4 billion toll road linking Petersburg with Suffolk near U.S. 460 that not many other officials seemed too keen about, Chmura served up a report saying it would create 14,000 jobs, including more than 8,000 jobs from advanced manufacturing or automotive firms that would locate by the end of the decade at two “megasites” in Isle of Wight and Sussex counties. A little problem: the Isle of Wight site is just gearing up and the one in Sussex hasn’t been built yet.

There are other examples of questionable data in Chmura reports involving the Redskins moving its summer training center from Ashburn to Richmond and in the capital pitching a 2015 international bicycle race. The former involved considerable monetary incentives to the rather wealthy Redskins NFL club.

The economics firm is headed by Christine Chmura, an economics Ph.D. with impeccable credentials at a Richmond bank and the Federal Reserve. Fourteen years ago, she founded her firm and built it up in this state and in her native Ohio. She is a popular speaker on the economics and policy circuit. (Full disclosure, when I edited a business magazine about 10 years ago, I hired Chris several times for economic analysis and was pleased with the results).

There does seem to be something wrong and when I wrote a cover this week for Style Weekly, I detail some of the issues. Style filed Freedom of Information Act requests and we reviewed some of the Chmura contracts. The Virginia Department of Transportation some her firm’s payments, including one for the new toll road, under “advertising/public relations.”

Read more here.

Share this article

ADVERTISEMENT

(comments below)

ADVERTISEMENT

(comments below)

Comments

8 responses to “How Good Is Chmura’s Economics Data?”

  1. Ghost of Ted Dalton Avatar
    Ghost of Ted Dalton

    Great article.

    I’ve always thought the best idea I’ve ever heard is this: If an economist presents a paper to any public agency (local, state, federal), it must go through the same peer review process as an academic paper. If the economist continually refuses peer review, they could be stripped of their doctorate.

    “Economics” has become highly politicized in the past few years. I think you’d see a lot fewer “studies” and idiocy spewed on television and in public policy circles if these individuals had to actually submit their work to peer review. It would definitely be in the public interest to mandate peer review.

  2. James A. Bacon Avatar
    James A. Bacon

    I know Chris pretty well and I respect her work. She did some pro bono numbers crunching for me when I was writing “Boomergeddon,” for which I am very appreciative. I can’t imagine her slanting her findings for any client. (Let me be clear, Peter isn’t insinuating that she has. But a careless reader might leap to that conclusion.)

    That said, I think Peter is raising some legitimate questions. I’m not sure they are questions about Chris, however, as much as questions about the potential pitfalls of hiring anyone to do econometric studies.

    The big issue, to my mind, is how you frame the question. Take the U.S. 460 study, for example. If you spend $1.4 billion on a toll road to serve the ports of Virginia in Hampton Roads, yes, undoubtedly, there will be an economic impact. The Chmura study examined that impact. I scrutinized it at the time and found its conclusions to be at least plausible.

    As I recall, here’s what the Chmura study didn’t do: Examine the net economic impact. Extracting $1 billion in public funds from taxpayers has a negative impact, which offsets the positive impact of the toll road to some degree. The truly meaningful economic question is this: Do the benefits of spending $1 billion in public dollars on the toll road outweigh the costs?

    The McDonnell administration wasn’t interested in asking that question, so it didn’t pay Chmura to study it. The McDonnell team, as Peter discovered through his FOIA query, was looking for something with P.R. value. Chmura delivered a product. She didn’t control how the question was framed, and she didn’t control how the McDonnell team used the results.

    As I observed when the study was released, “While the Chmura study does quantify the economic impact of the project, it does not purport to answer the question, ‘Is this a good investment of state dollars?’”

    If Governor McDonnell and his people made more of the study than was warranted and no one called them on it, who do we blame? I would blame the McDonnell administration. But I would also point the finger at the press, the legislature and watchdog groups who gave McDonnell a pass.

    Was it Chmura’s job to write a more contextually honest report that the client didn’t want and wasn’t paying for? I’m not sure it was.

  3. larryg Avatar
    larryg

    I’m not comfortable with the proximity of the people doing the study with the proponents.

    I note that GMU Stephen Fuller hs been hired similarly by developers to address the issues of fiscal impacts from development – and he often comes up with results that essentially favor more rooftops.

    I’m just leery of these kinds of relationships in general and the folks that sell their services run the risk of being perceived as “hired guns” rather than truly credible experts than can be trusted to tell not only the good – but the bad and the ugly.

    1. James A. Bacon Avatar
      James A. Bacon

      Any time you see an economic study published by an advocacy group, it makes sense to give the study extra scrutiny. I reject the idea that such studies should be automatically dismissed as biased, but we should be open to the possibility that biased has crept into the findings. As I mentioned in the previous comment, the most obvious way in which such studies can be manipulated is to frame the question in such a way as to lead to an advantageous answer. But it’s also worthwhile to look closely at the various assumptions embedded in the study.

  4. TMT Avatar
    TMT

    Peer review is important, so long as the peers are honest. The FCC has started peer review on some of its internal staff economic papers. So far, there has been a fair level of criticism on parts of the analysis. Having another federal government economist find a few points upon which to disagree helps the credibility of the process. Rubber stamping, on the other hand, is worthless.

  5. larryg Avatar
    larryg

    I’d be way more comfortable when a company like Chimura does a study that they provide with the final product – a literature search of other studies on the same subject.

    I do not care for “custom” studies that are provide by paid folks if there is any connection at all between their study and where the money came from.

    I do not reject them out of hand – but I do not consider them credible either without supporting data.

    and I do not believe anything because someone who is aligned with my views recommends it. I do the opposite because confirmation bias is a major disease among people now days.

    1. TMT Avatar
      TMT

      Studies should state their assumptions clearly and, as appropriate, explain why they were made.

  6. DJRippert Avatar
    DJRippert

    The closest thing to honest and accurate assessments in the business world are the independent financial audits of public companies.

    This is nothing like that.

    Independent auditors follow a standardized approach to accounting rules, the practitioners must pass a grueling test (the CPA exam), the practitioners must get continuing education and the US legal system metes out huge financial penalties for overt mis-behavior or incompetence.

    Why should anyone believe there is any real level of independence in these analyses? And let me be clear – if I were Ms. Chmura I’d do exactly what she does.

    As a 30 year consultant in the private sector I was often asked to analyze pending decisions and investments for clients. The requests came in one of two ways – a private request or a public request. In a private request the executive making the request quietly explained what they wanted analyzed. They showed me how to obtain the necessary data and insisted that the results only be revealed to a small, named group. These were honest requests for help in understanding the implications of a decision. The “public” requests were made loudly and openly – usually after a project had been started. The task was often phrased as, “Fine me all the benefits associated with this action.”. These were essentially requests for help in marketing a decision that had already been made. If there were major flaws in the decision you had to go back and explain the flaws to the sponsor. Why? Because they would come out anyway and when the sponsor got fired his or her replacement would ask you how you could have possibly missed the obvious flaws. However, if it was a mediocre decision you had to find ways to make it look good. That was what you had been hired to do.

    I am very sure that Ms Chmura did the work she was hired to do. I am sure she performed ethically. Yet I have no reason to believe she acted with studied independence. In fact, I’d guess that she did not. As far as I know, unlike independent financial auditors, Ms. Chmura is under no legal obligation to maintain independence “in fact and in appearance”.

    I think of Ms. Chmura’s work more like the testimony of an expert witness in a trial. She knows who she is working for and she “testifies” accordingly. If there is an egregious problem she will upset her customer and call it out. But it has to be egregious.

    The problem is that a trial has two sides. Our elected officials lack the competition of “the other side”. So, they ask people to help them market decisions they have already made. They use the analysis to push their version of reality. Just like in a trial – only with no rebuttal.

    What Virginia needs is a not-for-profit watchdog organization that does its own primary analysis of the government’s decisions. Instead of asking Ms. Chmura to “find all the benefits” or “quantify the benefits” it would ask Ms. Chmura to develop a risk adjusted return on investment. Then, in a perfect world, the media would publish and analyze both the government’s analysis and the independent analysis.

Leave a Reply

Read More