The value of electronic data capture

Within the world of transport data collection a recurring theme has been prevalent for several years now – the need to minimise timescale from fieldwork to provision of processed results

With video cameras replacing people on most count surveys over the past decade or so, the lag between fieldwork and final results has increased as data processing does not begin on the survey day and, whilst the improvements in data quality, verifiability and potential scale are recognised as a worthwhile trade-off, it is nonetheless the greatest challenge to overcome in our sector. This delay between survey fieldwork and provision of data increases with the scale and complexity of the data collection task and it is largely in response to this that we have sought new methods to either reduce or cut down on processes that contribute to this delay.

Handheld tablet PCs have been in use for fieldwork data collection (in the transport sphere) for a couple of decades but their widespread adoption has been slow in what has been at times a very conservative industry. The reason for this has been, in my view, largely due to a fear of addressing the barriers to their use. There are several constraints that have traditionally caused problems, relating to battery life, the need to recharge, weatherproofing, etc, and also relating to staff knowledge, both in setting up equipment and surveyors using hardware on site. The increased ease with which this type of technology can be introduced is largely thanks to the almost universal use of smartphones and tablets, such that even inexperienced new recruits can quickly get to grips with menus that are familiar to their everyday tools and toys.

Whilst there is a wide array of readily available tablet PCs on every high street and in every retail park and supermarket, the demands of this type of fieldwork dictates that something more robust is required. There are, however, several suitable units on the market that are fully weatherproof to IP65 standard and capable of resisting the handling that would be expected on site, and these can be used with standard MS Windows operating systems. To be effective on site the units must be capable of lasting three or more hours on a charge and have the ability to easily change batteries without the need to shut down.

Our first introduction to this type of technology was driven by the need to use a mapping base for highway asset data capture surveys. To achieve the required levels of accuracy and quality, the use of paper records on site was immediately rejected as a methodology. Having equipped our teams with these ruggedised tablets, running GIS software, it quickly became apparent that the possible applications of a similar technology extend well beyond asset capture.

As much of our traffic data collection involves a spatial aspect it was clear that using a mapping base to collect information on site could result in improvements in the methodology, measured in terms of timescale, flexibility and quality of data. Parking surveys, queue length surveys, and recording the way that people move around a space – be it within a station or crossing a road – are all far more logically undertaken and explained in terms of a map rather than columns of numbers.  For example, many of our team have prior experience of undertaking parking surveys using a tabular format on pencil and paper, whereby each ‘virtual bay’ on the roadside is allocated a space number and hard copy booklets are generated where each column represents a bay. Even with detailed notes and an overview map this format can be very confusing for survey staff and the likelihood of getting ‘lost’ in the beat means a very high supervisor/surveyor ratio is required to achieve good quality data. The advantages of offering survey staff a mapping background, which can be set to move with them with the addition of a GPS receiver, are obvious to anyone who has experience of both methodologies.

Similarly, any method of electronic recording means there are advantages in quality through avoiding the need to divide the study duration into surveyor noted time periods – the system logs every action against a set time, preventing the accidental missing of time periods.

The major client benefit of recording using a mapping base is the opportunity to output data into flexible, user-friendly formats. Whilst this style of output might not be so important to the numbers-hungry readership of this publication, it is generally received far more favourably than tables of data by the non-technical decision makers who often hold the project purse strings. Often, our partners within consultancies would generate visual outputs for the benefit of their clients in any event, but with the collection of data in a mapping format this process is effectively built into the surveys budget and frees up time for the engineer.

Of course, just as microsimulation models are more than pretty pictures, the beauty of GIS data capture is the ability to output tabular data from the mapping base to replicate data formats that would be provided as output from traditional survey types (parking occupancy by time period, queue lengths by lane and so on). Rather than replacing tabular data, therefore, this approach allows for spatial or temporal queries within the data set to enable a far broader range of outputs to be provided.

From this point of our development, in recognition of the values in timescale and quality of data from a mapping base, it seemed a natural progression to expand the range of survey types that can be undertaken using electronic means. With the physical tools and experienced fieldwork team in place, there was nothing to prevent the roll-out of similar technology to cover market research interviews, counts and vehicle occupancy surveys for example. Even traditional survey types such as video surveys and automatic traffic counts can be made more efficient by the use of electronic transmission of data and cloud storage, enabling remote sites to be downloaded and issued to client within minutes of completion of the survey in the case of ATC data.

Over the past couple of years we’ve introduced systems for these that have become our standard and have enjoyed considerable quality and turnaround improvements through this. We read with some surprise the article in a previous edition of this supplement on roadside interview (RSI) surveys methodologies, which seemed to suggest that a suitable alternative to the traditional pencil and paper method had not been found. We have used tablets on RSIs for several years and have enhanced the software through a number of iterations. Our current bespoke system utilises a quarterly updated full UK postcode dataset backend to validate addresses at the roadside, including using search tools to find, for example, the full address, postcode and OSGR for a shop name within a specified town. This type of exercise would traditionally involve hours of laborious trawling by teams of coders post-survey, with an inevitable proportion of addresses not returning a valid result. In addition, we avoid all of the difficulties associated with interpreting hand-written records. With an electronic system these errors and omissions can be avoided at the roadside and the turnaround time of the data can be cut from several weeks to several days for a reasonable scale of RSI programme. As a further benefit, this approach avoids generating several forests worth of waste paper – there are colleagues who recall entire offices given over to storage of hard copy data that can easily now be accommodated on a memory stick, with suitable data encryption and password protection.

Through some further development of the script, the same electronic questionnaire platform can be manipulated to enable effective streaming of a complex questionnaire that would be very difficult to administer in hard copy. This computer assisted personal interview (CAPI) approach is not particularly new but has failed to take root on a widespread scale within the transportation data collection industry.

With bespoke software development, the ability to expand the possibilities of electronic data capture is limited only by the imagination of the project team. For example, we have responded to a client’s needs to achieve pedestrian counts in six second interval outputs by developing a simple piece of script that logs the time of each screen strike within a defined area of the screen (to differentiate between pedestrian classifications) and from there re-configuring the data to appear in a pre-defined format within MS Excel. This is a relatively straightforward technical exercise but has enabled the collection of passenger counts, for use with modelling software, which accurately represents the surges of flow within a station environment in a way that would be impossible using traditional methods.

With changes in the way that we can source data through an increasing use of existing datasets, the traffic data collection industry, more than ever before, needs to make sure that it stays relevant and focused on the needs of the user. Increased client expectations on timescales, throughout the supply chain, as well as an increase in competition within the industry has put more and more pressure on the requirement to slash the lag between fieldwork and data provision, whilst maintaining quality standards. We believe that this in turn will drive the movement towards making greater use of electronic tools, and we are delighted to be pioneers in this field.