Should consultants get sued for errors in traffic forecasts?

As reported in LTT 8 Aug

As reported in LTT 8 Aug

There must surely have been a sharp intake of breath, a few shivers, a cold feeling of doom among the readers of LTT’s story last month ‘Lawsuits pile up in Oz for toll road forecasts’ (LTT 08 Aug). No fewer than five lawsuits are in progress in Australia against consultants Arup, AECOM, Parsons Brinckerhoff and Booz Allen. They forecast good revenue from toll roads, and investors lost a lot of money when the actual traffic was a lot less than forecast. In two cases companies have gone bankrupt, and it is Receivers who are taking the action. I was particularly interested because, ten years ago, my old friend Professor David Hensher of Sydney University and I wrote a paper together (published in the journal Transport Policy, 2004).

The paper was called ‘Using values of travel time savings for toll roads: avoiding some common errors’, and in it we said that using orthodox forecasting procedures, toll road revenues were likely to be overestimated. While the errors were likely also to affect traditional non-tolled schemes, in those cases it would be many years before anybody noticed, by which time there would be a wide range of different explanations, and few of the original analysts around to take the blame. “In tolled and charged regimes, however, and especially where there are complex questions of risk sharing”, we wrote, “it is much more difficult to evade or delay consideration of the error…Some convenient practices and simplifications… are re-examined sharply when what is at stake is real cash, received, or not received, by real companies”. The technical reasons for our warning derived from the use of values of time which should measure how much people are prepared to pay to save travel time – the core issue in toll road revenues. For any given average value of time individual travellers will always have a distribution of values more or less than the average: this distribution is skewed, which biases the estimated total, and the result is a sort of poker game (as we didn’t call it) between different institutional interests, which “are almost bound to lead to later disputes, attempts to renegotiate, or collapse of confidence”. This was more prescient that we expected.

Now, I cannot pretend to have any expertise or authority in Australian law, and the actual legal merits of the cases will be solved in court, or by agreement outside it, as is right. It should be said that the consultants do not accept that the cases have legal merit. One of the consultants, AECOM, said "We do sincerely regret that investors in the CLEM7 tunnel have seen a loss on their investment. However, Aecom in Australia is neither legally nor ethically responsible for this loss." On the other side, it is claimed that it had “failed to disclose material facts to potential investors – notably that it had earlier provided the City of Brisbane (where the tunnel is located) with a traffic forecast one half that which was published for investors”. The Australian Government has published some most interesting discussion, consultations and recommendations on this question, on the website http://www.infrastructure.gov.au/infrastructure/public_consultations/patronageforecasting.aspx They very carefully avoid attributing blame or prejudging the legal issue, but they do accept that there are serious problems in the way in which public-private partnerships are designed and contracted. They point to some ‘intrinsic positive biases’ in the four-stage models used for forecasting toll road revenues, ‘opportunities for optimistic selection of parameters’, and pressures built in to the bidding process itself. They say: “Pressures for optimistic selection may occur when:

• Government proponents keen to have a road built do not make use of advice available from government sources, such as treasuries, to consider the commercial viability of the project and so seek full private sector funding for a risky project

• High private sector ‘appetite’ for funding road projects leads to bidding for risky projects. At the time of bidding desire for profits exceeds fear of losses

• High construction costs (as for tunnels) and fixed toll levels mean high traffic forecasts are needed to show costs can be recovered

• Financial models for optimising the bid encourage high forecasts

• High levels of competition increase the desire to produce high traffic and revenue forecasts to win (perhaps exacerbated by the high costs of bidding)

• Acceptance by the government of upfront payments as part of a bid.

This reduces ‘costs to the public purse’ for associated works, but needs to be funded by higher revenue forecasts.” It’s a serious and thoughtful paper, and useful reading in this country also. In its response, the Australian Automobile Association, among others, says: “The structure of PPPs has traditionally been inherently biased towards over-optimistic traffic projections… AAA would encourage the adoption of mechanisms that reduce or constrain the incentive to overestimate”. Quite. But the contribution that keeps going around my head is from Denis Johnson, formerly a director of advisory services at AECOM. “(E)ach of the participants in a PPP has a different appetite for risk. It is therefore to be expected that each will want to adopt different assumptions about the future – which reflect their risk appetite. In other words each participant wants a different forecast (or range of forecasts). Where recent PPPs have come into disrepute is when the public has been invited to invest – essentially adopting the risks and the forecasts that the developer has accepted. The real issue here is that it if a developer wants to take an optimistic view of the future and ask his traffic advisor to prepare forecasts on the basis of these optimistic assumptions, it is not the fault of the advisor that the forecasts are 'high.’” That gives a wholly new insight into the selection of assumptions and parameters in traffic forecasts – not, as is perhaps naively usually discussed, a search for the best, most likely, most realistic and most accurate approximation to truth, but a reflection of the different ‘appetite for risk”. What can this mean, I keep asking myself. What is the Department for Transport’s ‘appetite for risk’? What is a county council’s? What, indeed, is a consultant’s?

How can an appetite for risk ever be met by choosing to assume a future you don’t believe to be true? (Though I can see you might want somebody else to believe it). I don’t know about the legal case, but the professional argument seems rather strong to me. Consultants do have a public responsibility which goes beyond simply providing an analytical framework to satisfy their clients’ appetites. I have great sympathy with the proposition that, in many cases, they are only obeying their clients’ orders. But maybe this is a case where some sort of formal statement of ethics and principles, drafted in discussion and agreed collectively, would help to clarify a murky territory. This should define a responsibility on clients, as well as their contractors, both in the public and private sector. Which prestigious institution, I wonder, would take on such a task?