Consultants
A few thoughts about expert consultants, as employed by both the government and the private sector.First, they are almost always actual experts, at least to some degree, with respect to the area for which they have been hired to consult.
Second, they are inclined to shape the data, whether consciously or subconsciously, to suit the outcomes and conclusions sought by the government / private sector actor that has engaged them.
Third, a consultant can make the data say whatever they want it to say, through a combination of selecting and / or favouring certain data points / sets over others, and in the all important part of interpreting that data and drawing conclusions based on those interpretations.
I’ll illustrate with a story.
Years ago, not long after I left the NSFDC, I was hired, along with two other former NSFDC staffers, as a consultant by a provincial government. They were considering whether or not to build a film sound stage there (everyone was considering whether or not to build a sound stage around 2000), and they hired various consultants to put a study together. There were technical and engineering consultants for the sound stage itself, but our job was to provide an expert analysis of the industry. We were eminently qualified.
Here’s the thing. We were told by the film development chief at the time that they wanted a sound stage. Nobody forced us to say that the film industry was booming, or that a sound stage (which was, after all, going to be a significant public expense) was an important part of continuing to build the industry there. But it was made clear to us that this was the conclusion that government was looking for… and they were the ones signing our cheques.
Of course, we took our job seriously. We made a couple of trips to the other province, and met with filmmakers there, and got their views. Key point? There was no consensus. The bigger producers and the ones who wanted to do service productions wanted a sound stage (as did the film bureaucrats who had commissioned us); the smaller, independent producers wanted the precious government resources to go elsewhere. Documentary makers in particular wanted nothing to do with a sound stage. So there was no firm answer there.
We examined the data supplied to us by government for the industry in that province (which, having compiled similar data during our time in government in Nova Scotia, we took with a grain of salt), and then we examined the data as it existed across the country, and in other jurisdictions. And in the end, privately, all three of us were convinced that a sound stage was a bad investment.
But that’s not what our report said (and it also wasn’t our job to make that call). We extolled the virtues of the film industry (which we really believed to be true, to a point, although we larded it up with a lot of rose-coloured fluff, as these things always do), and said that a sound stage was considered by key segments of the production community (i.e. the ones that government tends to listen to, which is to say the bigger companies) to be a necessary component of continued growth. We mentioned, in passing (probably a footnote, although I don’t recall), the opposition.
We didn’t tell them to build it, but we provided a measure of cover for them if they eventually chose to do so. One of the civil servants with whom we dealt later told me that folks further up the food chain weren’t terribly happy with our findings, in the sense that they weren’t quite as firmly pro-film (which really meant “pro sound stage”) as they had hoped. But none of us were going to sign our names to that kind of report, and in the end the one we did author was good enough – or at least we got paid. I never found out what they did with it. Being politicians, I suspect they cherry-picked the data and interpretations they liked from the data and interpretations we provided, which was to some degree already cherry-picked. But who knows?
I want to stress that absolutely none of the information that we provided was false. But it was carefully selected. And there is a difference. For example – and this is the easiest way to shift reports on economic development in one direction or another – we had to use a multiplier for the spin-off effects that the film industry had in the province. The problem was that there was no single multiplier for that kind of thing. We checked, and multiple government agencies and private sector groups used different multipliers for economic activity. Some were higher than the others, but as far as we could tell, most were just guesses. And in many cases, wildly speculative ones at that.
So what did we do? We just cobbled together a list of about a dozen different multipliers used across Canada in different sectors, added them together, divided by twelve, and took the average as the multiplier that we used (after rounding up, of course). It seemed like the most reasonable course of action, given that there was no real alternative. I can’t recall with certainty, but I’m pretty sure we loaded it up with a few caveats.
But the lesson for me was clear – never put your faith solely in expert reports that come from an interested party (and all parties who commission an expert report are “interested parties” to some degree). Trust claims of spin-offs less than everything else. At best, it’s like throwing a dart blindfolded.
Most important, always remember that most of these things are cases of the conclusions leading the research, and not the other way around. I’m sure if there was a group that was vehemently against a sound stage being built and it had commissioned a report, it would have gotten results along their line of thinking (or, if they didn’t, the report would never have seen the light of day).
Think of it like this – someone has a story that they want to write, and they know how they want it to end, but they’re not writers. So they hire a ghost writer to do the heavy lifting. In essence, that’s what most consultants are. I know that sounds cynical, but once you understand what’s really going on, you can approach expert reports better equipped to judge their worth / relevance.
And maybe that’s the best that can be hoped for – competing studies, put out in the open for all to see, from two sides in a debate / argument / policy dispute, so that citizens can look at both and decide for themselves which one makes more sense, or which parts of both seem right.
Kind of like politics.
Consultants