Towards innovation excellence in modelling

Many of the traditional assumptions underlying forecasting and modelling are being challenged by radically different travel behaviours. The way in which younger people, and people in cities generally, choose how to travel is already undergoing dramatic change. The emergence of Mobility as a Service (MaaS) in cities and Connected Autonomous Vehicles (CAVs) on the strategic network makes the point and their development will be fast – years rather than decades. We need to be prepared.

Transport modelling and forecasting has been integral to transport planning in the UK for decades, and has earned international respect for its practise and its innovation. However, there are longstanding issues concerning its application, for example in the development of policies and options (particularly non-car options), and in the appraisal of transport interventions and schemes (revenue or capital). 

This is often an issue of how models are used, for example not recognising levels of uncertainty and the sensitivity to underlying assumptions. Above all, the feeling that models are frequently used to provide post-hoc justification for established schemes, rather than problem-solving, has undermined public credibility – this needs to be addressed up front in the era of fake news and post truth. 

Perhaps the biggest fake news of all is that this is a recent phenomenon: in fact this ‘era’ began well before recent events. Decision-makers throughout history have tended to ask the questions which were likely to produce the answers they wanted, and then gone on to cherry-pick the results.

This has produced a debate on the use of models which has rumbled on and on, but now is put into new focus by the way in which many of the traditional assumptions underlying forecasting and modelling are being challenged by radically different travel behaviours. 

This is not just techy speculation about future technologies. The way in which younger people, and people in cities generally, choose how to travel is already undergoing dramatic change. The emergence of Mobility as a Service (MaaS) in cities and several elements of Connected Autonomous Vehicles (CAVs) on the strategic network makes the point and their development will be fast – years rather than decades. 

The emergence of Mobility as a Service (MaaS) in cities and several elements of Connected Autonomous Vehicles (CAVs) on the strategic network makes the point and their development will be fast – years rather than decades

This view is reflected in the falling number of car trips per person, now recognised in the latest DfT scenario-based traffic forecasts (in itself a very interesting and welcome development). The production of benefits from decongesting strategic roads over the next 60-65 years, without assuming CAVs are being used, is patently wrong but we appear still to be doing it.

Key issues

There are thus a range of issues in modelling and forecasting which have widely publicised at events and in the press in the last year, including:

• Dealing with the issue of how qualified experts, using the same data, can come to very different conclusions 
• A lack of public confidence and understanding 
• A parallel lack of understanding among many non-modelling professionals
• The use of alternative models for economic benefit alongside traditional transport models without clarifying their relationship properly
• Whether the emergence of ‘big data’ is making modelling less important 
• Criticism that models have been over used and sometimes wrongly used 
• A realistic assessment of the uncertainty of modelling and forecasting emphasised by the emergence of disruptive technologies such as CAVs and MaaS.

In light of this, practitioners themselves, including the Transport Planning Society and the Transport Modellers Forum, have begun a debate to explore these issues, but more importantly to see if there are actions which could be taken to make progress.The views in this article are, however, entirely the responsibility of the author.

The overall aim of this discussion is to encourage:

• Use of the right model at the right time for the right purpose
• Public and client understanding of what models can and can’t do
• Independent review of the modelling process in particular:
• Suitability for purpose
• Levels of uncertainty
• Interpretation of results
• An environment where innovation is supported

To do this, I suggest that a range of specific issues need to be addressed, including the need for clients and others to better understand the use of models, as well as promoting the best modelling, forecasting and appraisal practice.  

I set out below some more detailed points for discussion below: I think we need to ensure that:

1. Modelling and forecasting is proportionate and used at a sufficiently early stage to explore option generation and scheme development (both are in WebTAG, but are they used?)

2. Centralised guidance does not itself cause a high level restraint on innovation (there has been some criticism of WebTAG for this), on the other hand how are standards to be maintained?

3. There is diversity in provision and more innovation: reducing the perceived client risk of new approaches and smaller companies (dealing with the “do you have a turnover of at least £10million” question).

4. Assumptions behind the model and their impact on the output are made explicit and comprehensible to the public, hopefully reducing argument and speeding up the technical parts of public scrutiny.

5. There is a realistic testing of the model assumptions by practitioners who are not interested parties in the project for which the modelling and forecasting is being undertaken.

6. Support for local authorities is available, for example advising on:

• The best time to use modelling in the scheme/option development process

• Whether to use modelling or collect better data

• Which models were appropriate and their strengths and weaknesses

• How to deal with the challenges which arise during the modelling and forecasting procedures.

7. There is a centre for innovation and excellence, which will promote knowledge sharing and contribute to skills development in both clients and practitioners. This could build on the work of existing bodies such as the Transport Systems Catapult.

8. Pre-qualification procedures are simplified to avoid duplication across organisations and create a more level playing field among suppliers.

Making progress:options?

An early idea canvassed by TPS was for an independent commissioning body and centre of excellence (either new or using existing organisations).  This would provide support for local authorities and an independence of view from the earliest stage.  It could absorb some of the risk perceived by clients in commissioning innovative procedures in assessing real world schemes.  

Reaching out to a wider public could also be part of its remit. This is not a universally agreed option, in particular practitioners have expressed concern about contractual issues and being over prescriptive (some feel WebTAG is already too prescriptive). This needs to be explored further.  

Probably the other main option would be based on the idea (supported by David Metz at the TPS event last year) for a tough regulatory body with powers to ‘strike off’ practitioners (as in the legal and medical professions). This body would undertake independent auditing but a key question is at what stage this would be carried out and how early the intervention would be.

Either option would also be able to provide a centre for excellence and influence or possibly take on a role in running WebTAG.  We want all ideas to be on the table in the first instance and stress our openness to new thinking at this stage.  

Contributions have already been made by the Transport Modellers Forum, as well as TPS events and the TPS annual membership survey. We want to widen this in the coming months and produce a final report on findings once the process is complete.

We are interested in ideas and comments which consider:

• Are the objectives above the right ones?
• What other approaches might meet the objectives?
• What are the strengths and weaknesses of the different approaches?
• Is the idea of a new body overkill?

Issues with a new body

If the idea of a new body was progressed, what are the structural issues to consider. Should it be one body or a number of bodies? Should it be based on a number of independent approved agencies or a single body? Should it be accountable to Government like Highways England, or run independently? How would it relate to academic centres of excellence? Would an advisory Board be suitable, if so who should be on it? Are there more innovative models for governance?

The issues around data 

Should there be a new body to act as a centre for ‘big data’, much of which is considered commercially valuable, and ensure it is widely available? Does the advent of big data and real time exchange of data (for example between vehicles) mean that at least some modelling will be redundant? How do other practitioners and clients deal with these issues elsewhere in the world?

Modelling World brings together a mix of leading experts to discusss these issues and will provide a great opportunity to canvass ideas and opinions on this crucially important subject.

Keith Buchan is Director of Skills, Transport Planning Society