Planners cast critical eye over role of transport modelling

Major concerns about the role transport modelling plays in decision-making were aired at an event organised by the Transport Planning Society last week. 

The panel event – ‘Transport modelling: fact, forecast or fiction?’ – was arranged in response to the report, Who can save us from the misuse of transport models?, written by Yaron Hollander, a former Transport for London modelling manager who now runs the consultancy CT Think! (LTT 13 Nov 15). 

On the panel with Hollander were: Rachel Aldred, senior lecturer in transport at the University of Westminster; David Metz, honorary professor at University College London’s Transport Institute; and Keith Buchan, a director of consultant MTRU and the TPS. The evening was chaired by Lynda Addison OBE of the University of Hertfordshire. 

Hollander began by quoting US professor David Hartgen: “Transport demand modelling is certainly not an art, definitely not a science, probably not even a craft. It is more like a ritual.”

“To a large extent I agree with this provocative statement,” he said. The use of modelling in project business cases was inappropriate because it was impossible to know the future. Modelling was more appropriate in the early stages of project development, to test initial ideas. 

When modelling results were used in public consultation exercises for projects, Hollander said it became hard for the public to tell the difference between marketing material and ‘pure evidence’. 

He said project promoters should adopt a five-point plan he called DITCH:

• Direct  – use data directly

• Inclusive – avoid specialised transport techniques

• Transparent – “remove the monopoly of three [modelling] veterans on how we build our cities”

• Continuous – build software to learn from data feeds

• Humble – take no insight for granted. Rely on human intelligence to justify your policies

Aldred and Metz both advocated more scenario testing in transport planning. Metz praised the DfT’s new scenario approach to road traffic forecasting. Different scenarios made the appraisal process for projects more complex, he acknowledged. 

Metz thought it would need a scandal to give the transport modelling community the jolt needed to put in place a system of formal professional standards for model use. 

Buchan suggested that an independent body should commission  modelling and forecasting for projects. This would avoid conflicts of interest and bias that afflicted modelling conducted for project promoters, and the new body would provide valuable client-side support, particularly to local authorities.

One audience member wondered if Hollander and Buchan were talking at different levels. Would Buchan’s independent body address the fundamental issues that Hollander had identified? “Can models be saved? Are they fundamentally bankrupt?” 

Hollander said that most of the evening’s discussion seemed to assume that models needed to be improved. “I don’t agree. I think they need to be used less.” 

Responding to a comment that the demand for modelling was being driven by clients needing it for project business cases, Hollander said: “There’s no need to provide modelling for business cases.” People were imposing modelling on themselves.  

An audience member asked if modelling was “the continuation of politics by other means”, used to “validate what we first thought of”. 

Hollander said there was a lot of fear about speaking the truth about the link between politics and modelling. If politicians had made a decision to pursue a course of action then he said it was legitimate for officers to build an evidence base to support that decision. But it was not appropriate to conduct modelling to try and justify the decision. 

Buchan acknowledged that client pressures could influence models and forecasts. He had seen instances where errors had been discovered in models but the benefit:cost ratio for the project remained the same.

Hollander took a swipe at academics, saying they didn’t do enough research to point out that modelling had failed to deliver. 

He said he often heard suggestions that the explosion of real-time travel data provided a valuable new source of input for models. But he said the data should be used as an alternative to modelling.

 A TfL delegate said one fear of using scenario testing was that, by presenting multiple futures, stakeholders might ask ‘Don’t you know what you’re doing?’

Hollander replied: “I think you should proudly say we don’t know what’s going to happen in the future.”  He wondered if there was anything new to the discussion. “We have had this discussion ten, 20 years ago – we didn’t say anything new today.” He thought the approach to modelling would change one day, but the change would not come from within the modelling community. Instead, someone who didn’t know it yet would change practices, either through a court case or by showing that raw data can contribute to decision-making better than models. 

In concluding remarks, Metz said: “I think the TPS needs to get a grip on this. I would like to see the committee say this is a major professional issue.” A working party should be formed to produce a report on the state of the art in modelling, which should be shared with the DfT and the Treasury. 

TPS chair John Dales welcomed Metz’s suggestion. “Thank you for the challenge and we’ll respond to it,” he said. 

Both are speaking at at Modelling World 2016