Open your heart (in transport modelling)

With a fond reference to Madonna’s 1986 hit, Tom van Vuren and Robin Lovelace explore the state of play in the transport modelling journey towards a more open approach

Many of the software products we now use, emerged from individual efforts in public sector research and transport planning institutions at a time that commercial tools did not exist – and computing power was very limited and expensive.

A fascinating history of those early days in first the US, and later the UK, is told in Forecasting Urban Travel by David Boyce and Huw Williams (2015). 

TfL’s London Transport Studies (LTS) model, and DfT’s original National Transport Model are good UK examples of in-house developed tools. SATURN, originally written by Dirck Van Vliet at the Leeds Institute for Transport Studies, is still in use today. EMME, AIMSUN and VISUM all have their roots in academic institutions (in Montreal, Barcelona and Karlsruhe, respectively).

And who remembers CONTRAM, the original dynamic highway assignment model developed by David Leonard at TRL?

---

Modelling World, held in Birmingham on 19 June 2024, will again address the Open Source question, highlighting further successes since the 2023 event. Sign up here

---

These models contained long-unresolved bugs, had poor or no user support, were updated infrequently and in some cases disappeared altogether - woe be the organisation that relied on such tools that became obsolete through a single point of failure.

The most successful of these early transport models were acquired by businesses. The commercial model, of a one-off purchase price plus annual maintenance, funded marketing and sometimes expensive training and technical support. Software developers could be hired, tasked with providing quality-controlled software releases.  The number of users has grown, with 20k+ members in the Transport and Traffic Modelling Group on LinkedIn for example. Both TfL and DfT have procured proprietary software from companies such as Bentley and PTV.

Commercial solutions can be slow to respond to feedback (development costs time and money, and introduces risk to both vendors and users). They can impede innovation, lock-in people and organisations to monolithic systems that can be hard to adapt to new situations, and divert staff resources towards skills in the tool rather than development of cross-transferrable methods and understanding. An associated frustration can be the sluggish transfer of new academic insights to practice, and a lack of attractiveness of transport modelling as a career to mathematical modellers and data scientists, compounding the reliance on what commercial products can do – no more

And the cost of software licences is now such that their use in practice is generally restricted to the larger consultancies and public sector organisations, making it hard for individuals or micro-organisations to operate and compete. The complexity of models, despite their friendlier user interfaces, means that it takes months, if not years, to become as skilled as is necessary to operate them with confidence. As a result access is not evenly distributed, certainly between promoters of projects and policies, and those that are affected and object.

Are open-source models the answer? Tom van Vuren, Chair of Modelling World and Strageic Consulting Partner at Amey, talks to Robin Lovelace,  Associate Professor of Transport Data Science and Lead Data Scientist at Active Travel England. Robin is author of the open access article “Open source tools for geographic analysis in transport planning”: and co-author (jointly with Tom Cohen and John Parkin) of "Open Access Transport Models: A Leverage Point in Sustainable Transport Planning”.

Tom: What would you say, Robin, are the main benefits of moving towards an open source approach to transport modelling?

Robin: For me these can be classified into three categories:

1) flexibility and ‘power’ of the approach, 2) transparency and reproducibility, and 3) community. A great thing about these benefits is that they are mutually reinforcing, and grow as more people adopt open source, so you have a kind of exponential effect. I have seen this in academia where there are more incentives to collaborate and share, but also in industry, particularly in the data science world. 

For example, my frustration with not having access to all ArcMap functionality as a student motivated me to explore alternatives, which led me to install QGIS. With its hundreds of free and open plugins (currently 1000+  including AequilibraE (https://github.com/AequilibraE/aequilibrae) for transport modelling), it was a breath of fresh air.

Anyone wanting to do something a bit unusual can benefit in a similar way: popular open tools have many existing add-ons to choose from. And if none of them meet your needs, you can always develop your own, as with the R package stplanr (https://github.com/ropensci/stplanr) And the Rust crate odjitter, which I have helped develop, for geographic analysis and OD data disaggregation, respectively.

With an open source tool you can trace every step generating results. You can change any assumption and input parameter. An associated benefit is reproducibility: open source software allows you to go from a situation in which only one person (or the people with a license and know-how) can generate a particular result, to many people being able to.

And if you ever want to publish your work to scientific or public scrutiny, you can prove that your inputs and methods led to the published results. This transparency and reproducibility can transform the level of trust in your results, perhaps from something like “you’re a wizard using dark magic” to “wow you’re a wizard whose spell can run on any computer, that’s amazing and I get it now and trust your work”.

Community benefits are a broad category that may take longer to be realised. They include the benefits for you and colleagues of collaboration, and wider benefits in terms of team morale, ability to recruit talent, and the perception of your work and your organisation as a forward-thinking, ethical, and future-proof outfit. 

The most immediate benefit can actually be access to a vast community continually asking and answering questions on your behalf. Rather than having to contact a ‘support’ team, the whole world can be your support team.

Moving towards an open source approach, and sharing some of your best work on GitHub (as organisations including Transport for the North, Department for Transport and Active Travel England are doing) will make it easier for you to find, recruit, and retain talented people who are part of these expert communities of practice.

As highlighted by IBM’s 2021 report “Drive innovation with open source solutions” 87% of IT leaders worldwide see enterprise open source as more secure or as secure as proprietary software, so it’s clear to me that industry overall regard open source as future-proof and a good way to drive positive digital transformation. A blunter way of putting the future-proof and community-building benefits is this: all proprietary software eventually dies.

Tom: Can you give examples of where this has future-proofing happened successfully?

Robin:  The first that comes to mind is the Department for Transport...

I recall delivering data science training courses to DfT staff in early 2019. Their computers lacked open source software, with most barely able to run R or Python, let alone to install packages. Five years later and things have moved forward dramatically: the DfT has since adopted open source solutions in their IT, staff can install and use R and Python, and there are managed open source services available through products such as GitHub Enterprise, Rstudio Server, VS Code, and Google’s Cloud Workstations.

I have seen first-hand the gradual adoption of open source solutions in the Institute for Transport Studies, which has developed several popular solutions including the choice modelling package apollo, and the R package stats19 for downloading road traffic casualty data, now endorsed by the Department for Transport.

Tom: Which organisations are in your view First Movers, and what are they gaining?

Robin: As with the broader areas of data science and computing, the first movers were individuals, academics and ‘hackers’

Professor Kai Nagel at the Berlin Institute of Technology was a prominent early adopter of the open source approach in transport planning, and one of the pioneers of the agent-based  MATSim model which is now also widely used in industry.

Likewise the German Aerospace Centre (DLR) pioneered the open source approach with SUMO, also agent-based, which now boasts a large community including regular conferences and webinars. Leading the development of popular products leads to lots of demand and consultancy opportunities. In business, an early mover is Pedro Carmargo of Outer Loop Consulting, and some parts of established consultancies who deliver and maintain open source projects.

Tom: Will we see the emergence of a new type of transport modellers? What will their training needs be?

Robin: We already have seen the emergence of a new type of transport modeller...

People like Dustin Carlino, Pedro Camargo and Anastassia Vybornova, who are active in open source development and transport planning spaces. In a way, they are similar to the original transport modellers you described earlier, in that they had to develop new tools and tackle technical challenges, rather than just using what existed at the time.

The training needs for the new generation include data science, programming, and use of version control and collaboration tools, as you develop in a team rather than in a silo. And by the way, these are useful skills more generally, anyway.

In response to demand we are looking to re-start our short course (1 or 2 days) on transport data science at the University of Leeds, likely to take place in autumn 2024, watch this space!

Tom: Are resource constrained local authorities not just as disadvantaged with open source modelling approaches as they are with buying commercial services?

Robin: This is a good question, and there are of course dangers associated with any kind of digital transformation

The short answer is no, you won’t be equally disadvantaged. The distribution of pros and cons will vary depending on what investment (financial, social, technical) you make. If you adopt an open source approach in a sensible way the benefits can be huge. 

Sticking with the proprietary approach, benefits can include getting up-and-running quicker, reduced IT costs (if it’s all provided as a cloud service, which is increasingly the case), and reduced staffing requirements for ‘point and click solutions’. Disadvantages for me include cloud and software lock-in, deskilling of your organisation, and lack of flexibility.

Specific advantages of adopting open source software and the broader open source approach outlined above include capacity-building, future-proofing, and more opportunities for recruitment, collaboration and input. What I like most is that an open source approach can help ‘de-silo’ transport planning: many departments will likely use common open source tools like Python, R, Docker, and adopting these common tools will allow much more collaboration across different parts of your organisation.

A real issue in any technical area is staff retention, and I understand this is definitely the case in the public sector. The job satisfaction benefits of using and developing open source tools and not only ‘using’ the community but helping to build the community, should not be underestimated. The Department for Transport organised things like ‘coffee and coding’ clubs to great effect.

I recommend identifying the lowest hanging fruit. For many this may actually be the software that first got me interested in open source: QGIS. I have heard of many local authorities and other organisations progressively adopt QGIS, which is well-maintained and future-proof. It could save money compared with proprietary solutions like ArcMap, and I can’t see a reason why that would not apply to more specific transport modelling, too.

Tom: The $64,000 question: will the future be open source, commercial off the shelf of both?

Robin: It’s hard to predict the future as any honest transport modeller will tell you

However, we can get some insights into likely future pathways by extrapolating trends in the current transport modelling landscape and by looking at what has happened in related sectors.

There has clearly been a rapid increase in open source approaches over the last decade, as outlined in the “open source tools” paper mentioned above, and it is safe to assume that this trend will continue and accelerate.

My sense is that we are at the early phases of an exponential growth curve regarding use of open source software. The current landscape already contains both open source and commercial off-the-shelf solutions and this will continue. The history of open source in data science suggests that early adopting large companies will incrementally buy-into open source approaches, realising that they need to invest to ensure that they are future proof. 

Examples from data science illustrate how commercial funding can support open source projects in mutually beneficial agreements. In April 2024, the developers of the open source Python package for data science Polars announced a high-level collaboration with NVIDIA.

Companies benefit by prominently supporting a cutting edge tool, meaning more people want to use their products, and with benefits for their staff; the open Polars community will benefit from the inward investment. Imagine similar collaborations happening in the transport modelling space! 

And remember: open source and free-to-use are not the same thing: open source means free as in ‘freedom to modify’, while free-to-use means anyone can use them. There are many examples of ‘freemium’ propriety software products and commercial open source projects.

Tom: And to be fair, I see some commercial vendors already moving in that direction. Is there hope for old dye in the wools like me? What do you suggest we do to get up to speed?

Robin: It has been experienced people in this space who have been the some of the biggest supporters of the open approach, to my partial surprise

Professor John Parkin, who spent the early part of his career designing major road infrastructure, co-authored the paper on “open access models” and that means a lot: it shows that there is buy in and that it’s not just a flash in the pan. 

Leadership comes from those that see the longer term benefits, for their people and their organisation. In this context, here’s a fun checklist of things you can do right now to get up-to-speed:

• Sign up to GitHub.com, it takes only ~5 minutes, and you don’t have to do anything else. Being on GitHub allows you to participate in most open projects that are hosted there, whether in a Conversation, an Issue tracker, or by simply reading code and maybe contributing typo fixes or other things to documentation

• Read-up on the open source alternatives to your favourite transport tools and think about how open source code do something similar. If you want to act on it, you could start asking questions either internally, to the open source developers, or out in the open on LinkedIn or wherever (e.g. urbanists.social, an open source alternative to Twitter)

• Install LibreOffice and other mature ‘low barrier to entry’ tools on your home computer, to see how good open source has got recently

• Use your experience and connections to advocate for more open approaches (not in the narrow sense of open source software but particularly in terms of open culture) in your organisations

• Take a read of work on open approaches, including the papers listed above (both open access), or for more philosophical takes Karl Popper’s The Open Society and its Enemies (published in 1945!), or other texts or videos: education is key

• Challenge prejudice from people who say “we cannot do this because we’ve always done it like that” and be open to new ideas

• Ask questions about where you and your organisation stand in relation to digital transformation and where you want it, and its legacy, to be in 5, 10 and 50 years’ time

Tom: Any final words?

Robin: It’s not an all-or-nothing situation: there’s an open-proprietary continuum and we’re all somewhere on it

(I have to use Windows at work while happily using the open source Linux-based Ubuntu operating system at home.) We’re all on a journey and can take action to improve where we’re going regarding the open-proprietary continuum, just like Microsoft and Meta did with their recently open sourced VS Code and Llama products.

Whether we take big leaps or small steps, we can all move in a more open direction, and this is about more than just choice of software: it’s about a culture of sharing, openness, honesty and collaboration. Cultural change takes time and it requires advocates at all levels in an organisation.