Practitioners seek to improve DfT modelling & appraisal tools

The last issue covered the LTT seminar on the future of transport investment appraisal, which raised wide-ranging issues about professional practice and the way transport projects are looked at in their wider context by other parties. The discussion included current DfT consultation on potential revisions to WebTAG – the DfT’s guidance on preparing the economic case for transport investment. The DfT has also been holding a series of specialist theme-based workshops. Tom Van Vuren (pictured right), who led the first workshop on priorities for developing and maintaining modelling and appraisal tools, reports here on that discussion.

Does WebTAG place too much emphasis on mechanised modes, rather than simply the movement of people? Are current tools and guidance overly concerned with the business case, rather than also considering the use of models in the development of strategy and ultimate design? And is this mirrored in the types and size of models being used, representing roads policies or public transport supply projects much better than other interventions? 

These were the opening questions raised at a DfT Appraisal and Modelling Strategy consultation workshop on developing and maintaining modelling and appraisal tools held in Birmingham last month. Attended by a cross-section of practitioners, software developers, data providers and public sector client organisations, the workshop addressed the three consultation questions:

• What should our priorities be for improving the development of modelling and appraisal tools and why? 

• How can we best encourage innovation whilst maintaining a consistent and robust approach?

• What new and emerging techniques and methods should we potentially explore and what specific problems might they solve?

Uncertainty was a common thread throughout the discussions while improvements in environmental and freight modelling were highlighted as areas requiring immediate attention.

Proportionality

Despite being guidance, WebTAG is commonly interpreted rigidly, focused on compliance with a small set of quantified indicators. This often hinders proportionality in practical applications. A number of delegates suggested that the indicators and acceptance criteria should be revisited on the basis of practical experience, both in achieving them and their contribution to robustness in practice. To assist with this, the DfT could better communicate how the guidance has been interpreted in the past to produce a successful funding bid, particularly where the approach taken was not ‘classically compliant’. 

An example of this could be the application of a tiered modelling structure with different levels of detail and validation per tier. Also, delegates asked the DfT to add a frequently asked questions section in the guidance promoting awareness of where the guidance or proportionality has been well applied.

An underused tool to support proportionality is the Appraisal Specification Report (ASR). Despite there being plenty of detail in WebTAG’s Guidance for the Technical Project Manager, in practice ASRs lack detail of how the project team intends to deliver the modelling supporting the appraisal outlined. This then causes issues with proportionality, leading to the development of overly complex, computationally slow and financially expensive models. But as one delegate remarked: we want model outputs to be more timely and less costly, to give us confidence and ultimately to be useful for decision-makers. 

Uncertainty

Many perceive scenario planning as an essential aspect of modelling and appraisal, but this is new territory for most. Additional guidance is needed to show practitioners how to develop and select an ‘appropriate’ set of scenarios that are: distinctive but plausible; diverse enough to really reflect alternative futures; and consistent, internally and between projects and schemes.

Delegates felt that practitioners would benefit from receiving better documentation and/or a well-structured repository, highlighting examples of good practice.

Also discussed was whether existing model structures are too rigid to allow sufficiently for the representation of uncertainty. Should model development be more flexible and adapt to best reflect the new datasets available? Generally, new data sources are not being used to derive new behavioural insights or to prompt new model approaches. Instead, data are used chiefly to provide inputs into existing modelling paradigms. 

Many of the practitioners commented on the often experienced disconnect between the intended purpose of a transport model and how its results are ultimately used, particularly by those outside the profession. 

An example is environmental modelling that focuses on air quality and noise impacts. This is not to say that the transport models used today are unsuitable for providing outputs for use in environmental appraisal, but that models, even if fully compliant, may not always be able to produce the required or expected outputs directly, given differences in the level of granularity needed for environmental analysis. 

Thus, it was proposed that the DfT should: strengthen the role of the Appraisal Specification Report, but striking a balance between helpful checklists and being too prescriptive; and facilitate and formalise communications between environmental specialists and model developers to improve the robustness of the ultimate environmental analysis. 

Freight data and modelling

Most delegates agreed that there is a lack of data and guidance for reflecting freight movements and impacts in modelling and appraisal. This was both in terms of input base year travel patterns and in terms of our ability to reflect future changes. Furthermore, unlike personal travel, there is no common approach or access to commercial off-the-shelf software products for modelling freight. 

There was a consensus over a lack of applicable research into the freight and logistics field, with LGVs being singled out, given the recent growth in their numbers. Modelling and appraising freight is under-represented in WebTAG. Goods vehicle movements contribute substantially to economic benefits within highway scheme appraisals while their importance in environmental terms was also mentioned. Stronger guidance from the DfT is desirable, with better access to data, and possibly engagement with software providers.

Innovation

The workshop considered if there is enough R&D funding by the DfT compared with, say, 10-15 years ago. Is academic research of changing travel behaviour sufficiently visible? And is that research actually used within practical modelling and appraisal methodologies? Delegates felt that the DfT should provide better access to national datasets, and make a greater effort in collating and undertaking research into important modelling and appraisal assumptions. Three examples of the latter are:

• The fundamental assumption of stability and transferability of behavioural parameters over time, which recent trends increasingly contest. 

• Rail demand forecasts in multi-modal models vs using the Passenger Demand Forecasting Handbook (PDFH).

• Incorporating the DfT’s own software tools, such as TUBA and WITA, into existing commercial software platforms.

An unexpected finding of the workshop was that WebTAG guidance inhibits innovation in the use of commercial software products. Guidance has been written for products that have existed and been developed over a long time, with little perceived space for new entrants that might enable practitioners to undertake innovation utilising such platforms.

Many attendees were keen to see the DfT embrace new techniques, already applied in other sectors, to help foster a stronger culture of innovation within the transport modelling profession. Suggestions on how the DfT could do better in encouraging innovation included:

• Organising and/or sponsoring forums and practical sessions, including on proportionality and the modelling and appraisal of packages.

• The use of crowdsourcing platforms or hackathons to research ideas.

• Staff exchange between DfT and WebTAG users to stimulate collaboration and mutual insights into opportunities and constraints in development and application of the guidance.

• Revisiting links with practice and academia, for example by sponsoring PhD programmes.

---

Tom van Vuren is a Divisional Director at Mott MacDonald and a Visiting Professor at the University of Leeds. Tom has been Chairman of the annual Modelling World conference since 2006, and between 2008 and 2010 chaired the European Transport Conference. He is a member of DfT’s Joint Analysis Development Panel. Email: tom.vanvuren@mottmac.com

---

Contrary to expectation, changing model requirements from emerging technology and future mobility options were not high on the list of priorities. Agent and activity-based models were mentioned; but in both instances there was a lack of understanding of data requirements and operation, and a concern that the current rigid WebTAG compliance interpretation by consultants and clients would be a hindrance in accommodating these innovations.

 

Workshop attendees:

Nik Bowyer       AECOM

Peter Sykes       Aimsun Ltd

Neil McCormack      Arup

Ashish Chadha Atkins

Oliver Charlesworth    Citilabs

Phillipe Perret   CitiLogik

Robin Cambery Department for Transport (DfT)

Nila Sari          Department for Transport (DfT)

Graham Powell      Highways England

Joe Payne          Highways England

Phillip Clarke     Jacobs

Harvey Tims     Mott MacDonald

Tom van Vuren Mott MacDonald

Gareth Evans     Welsh Government

Chris Davis        PTV

Neil Chadwick   Steer

Pete Kidd           SYSTRA

Ian Palmer        Transport for Greater Manchester (TfGM)

Mark Honey      Transport for London (TfL)

Helen Ursell      Transport for West Midlands (TfWM)

Firuz Sulaimi     WSP