I’m pleased to share the completion of a recent commercial engagement with the Heart Foundation: the Community Walkability Map, a new interactive tool designed to help communities understand and advocate for more walkable, healthier neighbourhoods.

Table of contents

About the Project

The Heart Foundation has a longstanding commitment to environments that support heart health, physical activity, and healthy living. As part of this mission, they commissioned a user-friendly platform that visualises pedestrian access to everyday destinations – empowering residents, advocates, and planners alike to identify opportunities for improvement in the built environment.

The map allows users to drop a pin anywhere in Australia and visualise accessible destinations within 5, 10, 15, and 20-minute walk catchments. These walkability catchments are not based solely on distance – they account for real-world walking conditions including path connectivity, barriers such as highways, and signal timing delays.

This map is now a central feature of the Heart Foundation’s Community Walkability website, which also includes community checklists, fact sheets, planning guides, and a Supporters’ Toolkit for professionals in the built environment sector.

A Data-Driven Tool for Policy and Advocacy

A key innovation of this project is the integration of traffic signal delay-aware isochrone modelling – a methodology that highlights delays to people walking at crossings, often overlooked in conventional walking catchment calculations.

Built using open-source OpenStreetMap data, the tool translates complex urban transport data into an intuitive, map-based interface that supports both strategic communication and policy engagement.

From a technical perspective, I led all aspects of the solution design and delivery, including:

• Co-design and scoping with the Heart Foundation team
• Project management and delivery oversight, using an incremental approach with regular status updates
• Backend isochrone algorithm development, implementing signal delay estimation
• Geospatial data integration and transformation
• Serverless backend development
• Graphic design
• Frontend software engineering

The end result is a tool that blends open data, urban analytics, and visual storytelling to support healthier, more walkable communities.

Data Sources

The map’s core walkability modelling is based on OpenStreetMap (OSM) data. Where possible, users are encouraged to contribute updates to OSM or via the Mapbox contribution portal.

Additional datasets integrated under agreement with the Heart Foundation include:

• Street lighting data from Ausgrid and Citipower
• Tree canopy coverage from the NSW DPHI (CC BY 4.0)
• Traffic signal delay data from Better Intersections (ODbL licensed), supplemented with interpolation or estimated averages in data-sparse areas

Users can explore data attributions directly via the map interface and are welcome to contribute additional signal timing measurements here.

Reflections

It was a pleasure to work closely with Anna Gurnhill and the broader Heart Foundation team throughout this project. Their vision, clarity of purpose, and genuine commitment to creating healthier, more accessible communities made this a rewarding collaboration.

What’s Next?

If you have an interesting challenge in the urban analytics, planning or community consultation space and would like to collaborate – let’s chat.

You can also read more about my background, including my five years as a full-stack engineer at Atlassian, open source work and policy advocacy here. I’ll have more to share on a new commercial entity soon.

I’m looking to further explore the potential of traffic signal-aware isochrone analysis to quantify urban accessibility and support data-driven planning and placemaking initiatives.

I’m particularly interested in applying similar methodologies to cycling network analysis.

I retain ownership of the underlying IP developed in this project and am open to discussing its further application in other domains.

This blog post has been reviewed with the client before publication.

The post Introducing the Heart Foundation’s Community Walkability Map first appeared on Jake Coppinger.

This blog post provides an overview of traffic signal operation in Sydney (focusing on the inner city), based on technical documentation, conversations with government & industry experts and data I’ve collected after building Better Intersections.

Traffic signals are quite literally at the intersection of our urban spaces and transport policy. They have a huge impact on city life, pedestrian safety, and the uptake of active transport like walking and cycling. Through the brilliant Movement & Place framework, Transport for NSW has a stated goal of increasing walking and cycling trips and reducing pedestrian wait times at intersections.

Currently there is no open data on traffic light timing programming or timing in Sydney or NSW. Timing programming in a proprietary format can be purchased for AU$200 per intersection with a restrictive license that does not guarantee advocacy organisations would not be faced with legal action. Meanwhile, WA Main Roads publishes monthly exports of past actual cycle time phases in GeoParquet format, and Victoria publishes all their operational sheets under an open CC BY 4.0 license, including > ~10 PDF pages (or HTML sheets) of documentation per intersection (but still don’t include nominal cycle time).

I hope this article demonstrates why signal data should be as public as a train network timetable – and subject to the same public scrutiny and debate as one (even though complex trade-offs are involved). It’s long and dry, but should be a helpful resource for others looking to write more detailed or opinionated analyses.

Note: I am not a trained traffic engineer; my background is computer science and I work as a software engineer. This analysis is a side project. Please let me know of any mistakes, corrections or improvements in the comments below (or privately to jake@jakecoppinger.com) and I will update if incorrect.

Table of contents

Maps

Automated pedestrian signals

Above: Map of automated pedestrian signals in Sydney’s CBD, ref GS22402_V006_17JAN23 (Source: TfNSW, supplied. Published with permission.)

In the blue area some major roads (state roads) have the automatic pedestrian signal feature disabled. These exact locations are publicly unknown. 

Earlier automatic signal maps are published as:

• GS22402_V004_15DEC22: https://transportnsw.info/document/6231/gs_2402_automated_pedestrian_crossings_map_v004_15dec22.pdf
• SCO_0573 / 002/ 23MAR20: https://transportnsw.info/document/4897/sydney-cbd-pyrmont-pedestrian-access-automation-map_23mar20.pdf

Above: Map of CBD signal phasing times, 2018 (Source: TfNSW, supplied. Published with permission). Filename CCO_0604_SYDNEY CBD BRIDGE ST TO GOULBURN ST PED EVAL MAP_05APR18_V005.

This map has not previously been publicly available. The above map shows the state of the city in April 2018. The “Study area” border is still a broadly accurate border of the 90 seconds cycle area, with minor changes at the edges (also not publicly available). A number of years ago (I don’t know exactly when) the cycle time was 120 seconds, which had been reduced to 110 seconds.

Some of these intersections (10 or 15) sometimes runs 45 seconds in off peaks. These appear to be intersections with more people, but there is no guidance on how these were selected.

• Map of traffic signals with countdown timers in OpenStreetMap (needs some work, let me know if you know of others): https://overpass-turbo.eu/s/1wVx
• Map of scramble crossings in OpenStreetMap (needs some work, let me know if you know of others) https://overpass-turbo.eu/s/1wAk

Traffic light cycle times

Why do traffic light cycle times matter?

Walking and cycling are more attractive when you don’t have to wait for minutes at a traffic light. When pedestrians have to wait more than 30 seconds they are more likely to cross on a red light. (Martin, A., 2006. Factors influencing pedestrian safety: a literature review (No. PPR241). Wokingham, Berks: TRL (Transport for London). Longer light cycle times increase car throughput and increase vehicle speeds at the expense of pedestrian & cyclist amenity.

Sydney is an important study location as it is the birthplace and development location of the Sydney Coordinated Adaptive Traffic System (abbreviated SCATS). This traffic management system includes an algorithm that optimises traffic throughput at signals, but it can only measure vehicle volume on the road. I have been told it has been used in Shanghai to measure bicycle volumes to calculate phase times for cyclists, but am unable to verify this.

SCATS does not count people on the footpath, and the crossing phase is only “called” by either a push button or by being automated. It currently does not support a non-binary pedestrian presence – innovative solutions like LiDAR or computer vision need to count pedestrians and then activate a (boolean) sensor value. It is unclear whether in-progress upgrades to SCATS will provide any functionality to measure or prioritise pedestrians and cyclists. Press releases mention prioritisation but previous work to add more interfaces to SCATS has not been widely released (see Future Signal Development section).

The NSW Government sells the system commercially to 30 countries and 200 cities around the world by the NSW Government. We are quite literally exporting our biases (in technical respects like limited pedestrian counting functionality – maximum cycle time and other choices are made by the local operator).

The Transport for NSW (TfNSW) technical guide, “Traffic Signal Design Guide: Section 7 Phasing and Signal Group Display Sequence” (2016) states one of the design objectives of choice of phasing should be to minimise the cycle time (pg 5).

Sydney lags behind best practice signal timing compared to London, Copenhagen and the USA

The Sydney CBD has a 90 second SCATS maximum cycle time which is generally considered short by Australian standards. The CBD cycle time was reduced from 110 seconds to 90 seconds on the 7th of January 2018 (CoS motion 2018-08-17). This achievement was noted in the 2020 report by Gehl Architects 2020:

The reduction in waiting times has improved pedestrian amenity, however further improvements can be made.
…
Transport NSW have reduced pedestrian signal cycle time from 110 seconds to 90 seconds enabling more frequent crossing opportunities for pedestrians.

Further changes will be explored after light rail becomes operational to ascertain further efficiencies.

“Public Space & Public Life Study 2020” Gehl Architects, page 142

TfNSW & the RMS undertook monitoring for a three month period to assess impacts across the network (“Look up: Keeping pedestrians safe, June 2019, NRMA, PDF page 8, in turn referencing “Transport for NSW (2019). Personal correspondence.”) It was reduced from 120 seconds to 110 seconds a few years prior (I do not know when).

Copenhagen has a 70 second cycle time as the maximum, and high pedestrian usage areas have cycles of 48-60 seconds. Gehl Architect’s state “Jaywalking is not a common phenomena in Copenhagen as traffic lights accommodate for reasonable waiting times for pedestrians” (Gehl Architects, Public spaces & public life: Sydney 2007 (part 1), page 142).

The London Cycling Design Standards states “junctions with pedestrian crossing facilities, signal cycle times should only exceptionally* be longer than 90 seconds.” *(emphasis mine).

The 2018 transport for London walking action plan states:

TfL has responsibility for all London’s traffic signals, undertaking annual timing reviews at 1,200 signal junctions and crossings. In 2017, TfL reduced the wait times for people walking at 200 crossing locations close to schools, hospitals and transport hubs (see Figure 8). From 2018 onwards, this will be an objective for all signal timing reviews and, for the first time, ‘pedestrian time saved’ will be measured alongside other performance metrics.

Of the 200 reviewed crossings to date, 94 per cent now run a cycle time of less than one minute, meaning that if a person walking was to arrive just as the red signal for pedestrians came on and traffic was given a green light, they would only wait around 40 seconds or less to cross.

Chapter 6 – Building and managing streets, “Walking action plan – Making London the world’s most walkable city”, Transport for London, July 2018 (PDF page 29)

The Urban Street Design Guide by the (USA) National Association of City Transportation Officials recommends cycle times of 60-90 seconds in urban areas.

See traffic signal cycle times around the world on Better Intersections, and see graphs and charts at betterintersections.jakecoppinger.com/analysis.

Open source data collected with Better Intersections

I previously wrote a blog post on a website I built to crowd-source the collection of pedestrian traffic light timings: https://jakecoppinger.com/2023/06/mapping-pedestrian-traffic-light-timing-in-sydney-australia/

The data is licensed ODbL (for you, OpenStreetMap contributors!) and the website is open source too.

It has collected over 800 data points, each with measurements on green duration, flashing red duration, solid red duration, and notes such as whether the crossing is protected or has a countdown timer. Community groups including WalkSydney have expressed interest in involving their members in sourcing measurements.

Max Bo, a software engineer at Canva, has also begun some preliminary analysis of the data using the informational visualisation platform Observable: https://observablehq.com/@mjbo/better-intersections-sydney-traffic-light-timings

Who sets signal cycle times in Sydney?

In NSW the state transport department (TfNSW) builds, maintains, programs and operates all traffic signals. This is the case even on local roads (90% of all roads), where the local council is the authority.

TfNSW Operations has the very challenging task of designing and operating the timing of the network, with many stakeholders and a steady flow of complaints.

At present I understand that there is not a perceived need for the data to be published, however it is not considered to be confidential (see the below section TfNSW reasoning against release of signal timing data in NSW). On the contrary I would like to note the extremely restrictive license conditions upon purchase of traffic data detailed below. From what I can understand, one of the main reasons why the data has not been freely published is that the data export is a manual process. However, no data on minimum or maximum green times is published in NSW, though Victoria (who also use SCATS for their traffic lights) publishes all their operational sheets under an open CC BY 4.0 license.

Many transport advocacy groups, such as WalkSydney and BicycleNSW have attempted to source this data for a number of years for advocacy purposes to further understand how to better support vulnerable road users, but to date have not gained access. The restrictive license on data sourced from the TfNSW Traffic Signal Portal does not make it clear if any use of the data for advocacy purposes would result in legal action by the state.

In the City of Sydney (the CBD) the TfNSW operations team will know in little time if gridlock occurs, either from stakeholders or an extensive network of remotely controlled pan/tilt traffic video cameras (though they’re likely too busy to be watching such feeds unless an issue presents). They have to manage the traffic volume entering the CBD grid from motorway off ramps to prevent the queuing of cars on motorways.

Whilst many cities are demolishing their urban motorways, the recent WestConnex urban motorway project was recently constructed under a public-private partnership between the NSW State Government and Transurban. Transurban is a publicly listed company that after considerable privatisation, has majority ownership of 95 of the 99 kilometres of Sydney’s toll roads.

A given intersection will have a number of phases programmed into SCATS. A phase is a particular movement at an intersection (such as cars green in one direction and red for the other direction).

Note the phases on the left hand side of the SCATS software.

For a deep dive on the SCATS software, see “The New Generation SCATS 6”: https://www.aldridgetrafficcontrollers.com.au/ArticleDocuments/230/Introduction_To_New_Generation_Scats_6_5.pdf.aspx

TfNSW does not currently have a policy on cycle times

Transport for NSW’s Traffic Signal Operation document does not provide specific guidelines on cycle times and only focuses on safety critical timings. This means they do not currently have public guidelines for traffic signal cycle times.

However, “method 3”, page 7 of the Technical Direction on Traffic Modelling (https://roads-waterways.transport.nsw.gov.au/trafficinformation/downloads/ttd_2018-002.pdf) states

Method 3 (for new signals)
Assume a typical phase configuration and a cycle time of 140 sec (longer or shorter than 140 sec times may be used when accepted by RMS) and calculate phase times using traffic signals software such as SIDRA, LinSig, TRANSYT, etc. Then set initial maximum green times of all phases in the same way as in Method 1.

https://roads-waterways.transport.nsw.gov.au/trafficinformation/downloads/ttd_2018-002.pdf

This appears to be a statement of an internal policy, but note the words “unless accepted by RMS” (the predecessor to TfNSW).

The TfNSW Road User Space Allocation Policy provides strategic advice but does not provide any guidelines for cycle times. It specifies the equitable allocation of of temporal and spatial space must consider walking first:

This policy sets out the mandatory principles and requirements Transport staff must follow when allocating physical and temporal road user space safely and equitably to support the movement of people and goods and place objectives. (emphasis mine) 

Transport must allocate road user space based on the following principles:
…
– allocate road user space based on the network vision and road functions, considering all road users in order of: 

1. walking (including equitable access for people of all abilities)
2. cycling (including legal micro-mobility devices) 
3. public transport 
4. freight and servicing 
5. point to point transport 
6. general traffic and on-street parking for private motorised vehicles. 

TfNSW RUSAP, page 2

Although there is no policy or commitment to lower cycle times and pedestrian waiting times, it is apparent the lower cycle times are situated in areas of high pedestrian activity.

Note that for coordination, a common cycle time must be used, so generally, intersections in a network or along an arterial road need to have the same minimum and maximum.

I have been told SCATS has a feature of multiple minimum cycle times which are chosen based on the degree of saturation (DS). Switching between such minimums can result in a large drop in cycle time relatively quickly.

How long do pedestrians get to cross the road?

Length of green light

The length of the green pedestrian signal at traffic lights is calculated by summing a minimum value, plus a constant amount predetermined by how many people will cross at the intersection (in theory).

The NSW standard is for a minimum 6 second green light, and then an extra 2 seconds for each additional rank (row of people) above 3. This presumes that 6 seconds is enough to clear 3 rows of people from the adjacent footpath.

This NSW operational theory falls down in the CBD when there are large volumes of people.

Although not formalised the operations team are constantly reviewing intersections in the CBD especially for pedestrians. Furthermore, for major projects such as Sydney Metro or Light Rail, modelling with Pedestrian Comfort Levels as per TfL standards or TfNSW Walking Space Guide is undertaken on nearby footpaths and intersections to ensure reasonable levels of service are maintained.   

In general the length of time is set by manual observation (not real time CCTV or computer vision or anything), and is private. (see below section on SCATS data cost).

Information I’ve not been able to ascertain includes:

• how often measurements of the pedestrian volume are taken;
• what time of day they are taken;
• whether the min/max/avg/85th percentile is used;
• how often they are updated.

In Victoria, Volume 5: Intersection Modelling of the Transport Modelling Guidelines (2020) specifies:

At existing locations, where a revision to the intersection control is proposed, pedestrian crossing counts should be undertaken. These would usually be done as part of video surveys of peak periods (Section 6.6.1.1). This is particularly important where pedestrians will be provided with priority over vehicular traffic as it can have a significant impact on capacity e.g. left hand turn movements. The presence of high numbers of pedestrians may also influence the form of intersection control on safety and amenity grounds. The data collection may involve more than just an aggregate number of people at a site but might identify how pedestrians impact the movement of traffic.

(emphasis mine)

The Victoria Department of Transport even release their manual count data from detector and survey locations, though the link appears to currently be broken.

The Department of Transport have a repository of manual count data from detector and survey locations. For further information and project specific request visit the following link: https://www.vicroads.vic.gov.au/traffic-and-road-use/road-network-and-performance/road-use-and-performance

When there is no need to make the pedestrian light red (that is, there are no cars to implicitly prioritise and move across the intersection), the pedestrian green light will stay on for as long as possible until the end of the non-conflicting phase (subtracting the time for the flashing red of course).

The green walk period can be extended at schools during school hours, either by time-of-day routine (Personal correspondance, 2025-03). I am unsure of how such locations are specified or how much extension is provided.

SCATS has three methods of controlling the length of the green walk signal:

1. terminate “x” seconds from the start of the phase
2. terminate “x” seconds from the end of the phase
3. walk for green – stay green for as long as possible regardless of phase length

In all cases, the green signal for pedestrians is never be displayed for less than the minimum time as defined in the times setting saved inside the traffic signal controller.

Length of flashing red light

The length of the flashing red light is determined by the road width. People are assumed to walk at 1.2 metres per second (an 85% percentile figure, so about 15% of people walk slower than this), so the road width divided by this figure is used to determine the minimum flashing red time.

When and why are cars allowed to cross when people are still walking?

The flashing red light is legally defined so that pedestrians may not step off the kerb, but may finish their crossing.

Generally cars are allowed to have a green light when a flashing red light exists for pedestrians, and they are meant to yield to pedestrians.

Vehicles turning left or right who enter an intersection on a green roundel must give way to pedestrians on parallel crossings when either the Green man or flashing red man is displayed.   

TfNSW is undertaking a safety program to add red arrows to prevent “green-on-green” occurrences – when pedestrians and cars have a green traffic light which is in conflict.

It is being rolled out at two-phase signalised intersections across NSW, and is being expanded to cover all traffic signals. The modification is mandated whenever major works are undertaken outside the program.

While this program undeniably improves pedestrian safety and is a positive change, it is unclear whether reducing vehicle speeds would be a more cost effective use of taxpayer money than retrofitting thousands of traffic signals in NSW (or could be pursued in addition). I consider this outside the scope of this blog post.

Automated pedestrian traffic signals / automatic introduction

Automated pedestrian traffic signals don’t require a pedestrian to press a button to request to cross the road.

When the pedestrian phase is automated:

• The button does nothing when pressed (placebo)
• The pedestrian phase is never skipped – every SCATS traffic light cycle includes the pedestrian phase

Non-automated pedestrian signals may skip the pedestrian phase if the button is not pressed.

There is no such functionality where holding in a PB/5 pedestrian button extends the green light for pedestrians.

Pressing the request button before the car phase goes red should guarantee the green man for pedestrians (prevent skipping the green pedestrian phase), however this varies depending on the age and model of the controller. I’ve not come across any public information on differences between controllers on this behaviour in my readings thus far.

The actual pedestrian-experienced duration between when you push the button and you can actually cross is a mystery – not public knowledge, and I have not yet implemented the ability to measure this with Better Intersections (PRs welcome!).

Automated pedestrian introduction at traffic signals has existed in some areas in the Sydney CBD since the late 1970s or early 80s.

For certain time periods during the day they are also operating in Parramatta, Redfern, Chatswood, Liverpool, North Sydney, Newtown, St Peters, and numerous isolated intersections (outside of these times pedestrians are still required to press the push button).

To my knowledge there are no publicly available maps of the automated pedestrian signal area until COVID, even though automation has existed since the mid 90s.

Pedestrian crossings during COVID outbreaks

I wrote an article about the situation during the initial COVID outbreaks:
Sydney CBD is bringing back pedestrian “beg buttons” (December 2022).

In summary, more pedestrian signals were automated and covers were placed over the buttons so there was less physical contact.

The locations of all newly automated pedestrian signals programmed during the initial COVID outbreaks was not public, however a 2020 media release mentioned 11 health precincts (TfNSW Media Release, 28/3/2020).

Removal of automated pedestrian crossings after COVID outbreaks

After initial COVID outbreaks:

• the plastic button covers were removed
• a publicly unknown number of crossings at not publicly locations were reverted to non-automatic operation permanently
• the automated crossings that remain are only automated for daylight hours (hours for the CBD is known)

Buttons in the CBD are affixed with an adhesive plastic label explaining the hours of automation which is already coming off in many places.

On Friday the 16th of December, Transport for NSW announced:

From mid-December 2022 Transport for NSW is removing the push button hard covers but maintaining the automation of pedestrian crossings at traffic signals during daytime hours. This is to reduce overnight noise for local residents and allow for more efficient intersection operation for all users when activity is low.

The overnight noise mentioned – the stated reason for removal – was due to many complaints from residents living near signals, where the frequent green pedestrian walk sound disturbed sleep.

I do not know, and it is publicly unknown how many complaints were received. I also do not know why the changes were applied to area where there are few ground floor residents (eg. many streets in the city centre).

It was not uncommon for signals to allow a green phase every cycle (often a 24-36 second cycle due to low traffic volumes). The areas impacted by overnight noise are not publicly known – likely in dense residential areas where homes are very close to traffic signals.

However, the “green walk” sound of the pedestrian signals can be muted. SCATS supports this functionality and the National Cooperative Highway Research Program study “Accessible Pedestrian Signals: A Guide to Best Practices” (https://www.trb.org/Publications/Blurbs/164696.aspx, DOI: 10.17226/22902), summarised at http://www.apsguide.org/chapter10_australia.cfm states:

APS [Accessible Pedestrian Signals] are sometimes turned off at night due to neighbors’ complaints about noise.

NSW has no crossings where the audio feature is muted. To my understanding the NCHRP guidelines were overridden disallowing selective overnight muting of individual problematic traffic signals in NSW on safety grounds. I do not have context for why this was the case.

Muting of audible traffic signals in Brisbane

There is a 2014 article about Brisbane turning off audible traffic signals for the entire CBD between 9:30pm and 6:30am, with very real impacts on people with a visual impairment.

The current content of the Brisbane City Council website suggests a more case-by-case basis approach to muting traffic signals, however I have not found any recent updates on the current state.

Automatic introduction of pedestrians signals in Melbourne

In Melbourne, automatic introduction currently applies from 5:30 am to 12:30 am every day. This is two hours more each day than in Sydney and there are more people living in the centre of Melbourne.

The automated phases matches the times that tram services are operating.

Automated phases also operate 24/7 crossings of the narrow ‘Little’ streets (where pedestrian lanterns exist) and at the intersections along King St (Flinders Lane to Latrobe St) and along Collins St (Queen St to Exhibition St).”

(Review of Signal Operations for Pedestrians in the City of Melbourne, David Nash, 2014)

Pedestrian priority signals / cars requesting to cross the road

In Sydney there are very few places where cars have to request to cross the intersection, rather than pedestrians and cyclists.

Some of this rare locations are:

• Most of Martin Place
• Carrington St (the eastern bus stops outside Wynyard Station)
• Harbour St near Hay St, Chinatown (multi lane road) – just north of the zebra crossing near the QMS advertising panel
• Somewhere in Chatswood, likely near the station

These locations are where there is very low car volume and high pedestrian volume.

Where they exist a zebra crossing would likely be more optimal, however in NSW zebra crossings are not permitted on roads with more than one lane in each direction for safety reasons.

There are none outside the CBD – although there are many locations where the volume of people could justify this type of intervention (eg: near railway stations, shopping streets or along beach fronts)

Research

Transport for London (TfL) trialled pedestrian priority signals at 18 locations in May 2021, and has published new data that giving priority to people walking, showing them a continuous green signal until a vehicle is detected approaching, resulted in:

Compliance with traffic signals by people walking increased by 13 per cent, reducing the risk of a collision with a vehicle, while compliance by people driving stayed the same. The data also shows that the signals had virtually no impact on traffic, with only minimal changes to journey times for buses and general traffic – the largest increase for buses was only 9 seconds and for general traffic only 11 seconds. Several sites showed improvements in journey times.

New TfL data shows success of innovative ‘pedestrian priority’ traffic signals, 24 February 2022, TfL

Isolated signals

Signals may also operate in Isolated mode, with local vehicle actuation (by detector loops) being the sole operating strategy. In Isolated mode the sequence and the maximum duration of each phase is as specified in the local controller time settings. The local controller may terminate any phase under the control of the local vehicle actuation timers or skip an undemanded phase, unless prohibited by the local controller settings. Isolated mode may be specified as the fallback mode of operation.

The New Generation SCATS 6

This results in faster start to the green pedestrian phase when a button is pressed

SCATS can set any intersection to isolated mode at any time for any reason. It could be based on

• time of day
• day of week
• cycle length
• a low degree of saturation (measure of how much demand there is relative to the capacity)

The decision is up to the signal programming.

A type of isolated signal is known as “Master-Isolated”. These behave like an isolated intersection (where a pedestrian pressing the button may almost immediately get a green), however are under the control of Masterlink (under remote control of SCATS). This gives a pseudo-Isolated operation that’s managed by the SCATS cycle length and plan data. Where designed, it’s usually enabled when the cycle length drops below a pre-defined threshold.

Locations of isolated signals

• The mid-block crossing on Joynton Avenue outside the Gunyama Park Aquatic and Recreation Centre in Green Square
• The mid-block crossing on Chalmers Street (near Central station) outside the swimming pool

Let me know if you know of more!

What sort of “green waves” are used?

A green wave occurs when a series of traffic lights are coordinated to allow continuous traffic flow over several intersections in one main direction.

They are commonly implemented for peak hour vehicle traffic into and out of the city. Where possible, the operations team also attempts to provide a “reverse” green phase in the opposite direction. My understanding is that this is tricky and not always possible.

I’m not aware of a any examples in Sydney of a green wave for cyclists or pedestrian traffic, except in cases where electric bicycles can achieve a similar speed as road traffic for a car-focused green wave (eg. Bridge Rd in morning peak). Cycling southbound along the new Castlereagh Street cycleway can sometimes feel like a green wave, however this is because cyclists may catch the motor vehicle green wave.

When switching between directions of green wave, SCATS rotation can adjust cycle time 20-30% to switch link plans.

Green waves for cyclists in Melbourne

There is a green wave operational on Albert St bike lanes in east Melbourne. It runs 8:00-9:30am inbound and 4-6pm outbound. Few people know that it exists – there are limited public comms. There are no inbuilt progression LED lights built into the pavement due to budget constraints.

There is also a current trial on Rathdowne Street, Carlton, using fisheye cameras. I’m unsure if this is yet integrated with SCATS (and if so, whether it presents a virtual loop or some other integration method)

University of Melbourne is partnering with Cubic Transportation System to roll out a number of smart cameras that watch road users and could control traffic flow accordingly – in real time.

Seven fish-eye cameras will be installed over the next few months above intersections along Rathdowne Street in Carlton as part of the Australian Integrated Multimodal Ecosystem (AIMES), led by Melbourne University professor Majid Sarvi.

https://bicyclenetwork.com.au/newsroom/2020/07/16/melbourne-tests-traffic-controlling-smart-cameras/

On Canning St, the Princes and Johnston St signals are timed to give a good bicycle run in AM & PM peaks. However, it is set to ~23km/h which in practice might be too fast.

Green waves for cyclists in Copenhagen

Meanwhile, the City of Copenhagen established the first green wave for cyclists on Nørrebrogade in 2007, so bicycles travelling at 20kph did not encounter red lights. This system included LEDs embedded in the pavement to help cyclists keep the same pace.

As of 2014, Copenhagen was experimenting with “Green wave 2.0”, a detection system that keeps traffic lights green if there are 5 or more cycling citizens (roughly) cycling together.

The evaluation report on green waves on Nørrebrogade showed that “travel time decreased by 17 percent for a cyclist who cycled out of town at 20 kilometres per hour”. [6] (Centre For Public Impact, 2016).

The complaint process for pedestrian signal timing discourages feedback

Complaining about a long pedestrian wait time at a traffic signal is not an easy process. It takes 7 steps on https://www.nsw.gov.au/transport-for-nsw/contact-roads-waterways/online and it is not clear that this is the correct form for walking related feedback at all:

• Click report an issue -> roads -> management -> set location -> then explain issue.

Operators may sometimes state non-existent policy:

…
the whole principle of pedestrian movements is they are a slave to vehicular traffic.
…
The reason we don’t hold the walk is that we don’t want people running to get the walk because then, quite often, they’re running looking at the signal rather than the traffic, and in some instances, they step off the footpath in front of an oncoming car. So it’s a design standard that’s adhered
…

~5pm on the 28th of August 2024

After further correspondence this was corrected and a “walk for green” was implemented (York St and King Street). Walk for Green” means the green pedestrian light is displayed for only slightly less than the green roundel for vehicles.

Another successful “walk for green” implemented after a phone line complaint (which I have been informed of) is Regent Street at Lawson Street.

Why do traffic signals take so long to implement / redesign?

A consistent thread I have seen mentioned in this space is that the issue of the implementation of new signals or signal redesign is often longest part of any road redesign or cycleway implementation project, often requiring upwards of two years.

I have limited detail on why this is the case, however some contributing reasons may be:

• Design of traffic signals being safety critical, requiring 4 engineers signing off on documents and detailed infrastructure diagrams
• Team busy working on adding red turning filter lights (and associated design) throughout the state along with countdown timers

I received a suggestion that another contributing factor could be loss of experience and expertise as traffic engineers reach retirement age. The sender noted he had worked in DMR/RTA/RMS/TfNSW for over 50 years, and that they knew many others in Australia that have been in traffic nearly as long.

The extremely high cost of purchasing SCATS timing data

Victoria publishes all their operational sheets under a a free and open CC BY 4.0 license, including > ~10 PDF pages (or HTML sheets) of documentation per intersection. This data contains minimum and maximum green times, but doesn’t appear to specify nominal cycle times. In Sydney or NSW, you must pay at least $200 to purchase the programming of a traffic signal for a single intersection under a very restrictive license.

The walk timings (along with target cycle and phase timings) appear to be stored in a plaintext format known as LX, which I’m not currently aware of any open source way to process except https://github.com/johntrieu91/scatsutilities (Python, MIT license).

An incomplete specification of the LX format is detailed at Attachment A (pg 15) of “Traffic Signals in Microsimulation Modelling”, a Technical Direction document by TfNSW. Further documentation is on page 48 of the Signal Data Information for Modelling report (Main Roads WA).

It appears the LX file for a given region can be purchased under restrictive terms and conditions at https://www.transport.nsw.gov.au/trafficsignal. The fee schedule is outlined at: https://www.transport.nsw.gov.au/system/files/media/documents/2022/Network_Operations_Schedule_of_Fees_01.08.22.pdf

My understanding is that the export is a manual process an operator must run, followed by manual clean up of the data and sending it.

A few items:

Explanation of Phasing – Detailed lay explanation of traffic signal phasing for a specified date. – $600
SCATS Traffic Counts (TCS graphics plots included): $300
Interpreted SCATS history files (TCS graphics plots included): $400
TCS graphic plots: $200
SCATS Region LX file (all TCS sites on region): $200

An LX file for a single SCATS region costs $220. I’m unsure how many intersections (ie. TCS sites) are in each region, but I know SCATS supports up to 250 intersections per region. I am unsure how many regions cover the City of Sydney for example.

There are also strong terms and conditions, however I see a permitted purpose is “(d) you undertaking traffic modelling or transport related projects,but excludes the right to sell, monetise or commercially exploit the Traffic Signal Information and/or Developed IP (Licence).”: https://www.transport.nsw.gov.au/trafficsignal/traffic-signal-portal-terms

The SCATS regional traffic control software has a maximum capacity of 250 intersections per region. With a maximum of 64 regions, the total capacity is 16,000 intersections.

I note that on page 13 of “The New Generation SCATS 6” document there is an undated screenshot of a region display:

TfNSW reasoning against release of signal timing data in NSW

Response to my public Q&A question at the “SCATS – What is it and where is it heading” talk on 2025-02-06:

Question: Jake Coppinger, day job at Atlassian, volunteer with Better Streets, but representing myself here.
A question on open data and cycle times: My understanding is from mid-March, WA Main Roads will be able to release monthly dumps of their cycle times that were the real-time cycle time data. Do these upgrades enable that feature in New South Wales, or would we be able to release open data here? Thanks.

Answer: Essentially, the New South Wales government hasn’t determined whether or not we will release information completely on an AI platform on cycle time data.

We will do it at a user pay cost just because that open platform, I think, because that open platform hasn’t been developed yet. So, you know, we will provide that data. The way in which we present it at the moment, again, is very labour intensive, so, we charge for that data. Do we have any real objection to providing that data as operators, to providing that data openly? No, I don’t think so. But exactly what the government and transport at a senior executive level wants to do, obviously that hasn’t been determined yet.

David Ballm (Director, Network Operations, Transport for NSW), transcript of Q&A question asked 2025-02-06 in public Q&A setting (SCATS – What is it and where is it heading?)

SCATS upgrade

SCATS has been installed at more than 60,000 intersections across 200 cities and 30 countries worldwide.

A major upgrade is underway. The upgrade will likely involve a visual facelift, judging by the dated visual language of the diagrams generated, but little additional knowledge of the improvements is public.

There was briefly a desire for a new commercial model however this looks to have been abandoned.

“TfNSW is seeking to implement a new commercial model, with a commercial partner to accelerate innovation and importantly future-proof the SCATS product and business,” tender documents said at the time.
But the department has now ditched that approach, with a new team to be established within TfNSW to accelerate the development of SCATS and expand its commercial activities. (ITNews)

Do buses get signal priority? Which routes and when?

Information from the public “SCATS – What is it and where is it heading?” talk. Corrections are welcome as always.

Bus priority only exists on 5 or 6 bus routes in Sydney:

• Liverpool to Parramatta transitway
• North West Transitway
• A couple on routes on Windsor Road

The tech is relatively scalable – if decision made to give route priority, it can be done without major capital investments

Traffic signals only give priority if the bus is 50% full and are running 2-3 minutes behind the timetable.

I received correspondence that bus priority is also used in Newcastle and Wollongong.

There is, of course, priority for light rail/trams in various cities. In Sydney this proper prioritisation for the L2/L3 came after some time, but that’s another topic in itself.

Future signal development

Media reports on SCATS upgrade

In July 2021, It reported:

A spokesperson told iTnews that “after further work, TfNSW has chosen to proceed with a new structure, which extends on aspects of TfNSW’s original plan in 2019”, part of which will include “working with partners”.
…
Features include better integration with third-party sensors, machine learning-based prediction, sophisticated analytics and performance reporting and a UI/UX refresh.

https://www.itnews.com.au/news/transport-for-nsw-ditches-scats-traffic-system-commercialisation-566722

ARN reported in 2021:

Called the Sydney Coordinated Adaptive Traffic System (SCATS), it aims to trim travel time by a third, reduce fuel consumption by 12 per cent and lower emissions by 15 per cent, with more gains yet to be realised as the partnership pans out.

“Thanks to a new partnership with software development company Industrie&Co, our TfNSW development teams will collaborate to deliver the next generation of SCATS,” TfNSW deputy secretary for customer strategy and technology Joost de Kock said.

“SCATS uses algorithms and real-time traffic signal data to help customers get to their destinations faster and advanced sensors will allow cyclist and pedestrian movements to be included…

https://www.arnnet.com.au/article/693549/industrie-co-wins-transport-nsw-deal-new-scats-drive/

My understanding is that SCATs does not currently prioritise cyclist and pedestrian movements using any sensors – merely not skipping this phase when they are present.

https://www.arnnet.com.au/article/693549/industrie-co-wins-transport-nsw-deal-new-scats-drive/

More advanced pedestrian and cyclist detectors

Lidar detectors

In Victoria there is an ongoing trial using Lidar sensors to record road user positions in real time, which does currently feed data into the traffic light system. It can generate bounding boxes for vehicles, pedestrians, cyclists in real time and identify queues. I am unclear what the data from this trial will be used for.

My understanding is currently SCATS cannot natively use this Lidar input data as it does not have the capability to – the workaround is creating “virtual loops” that function like a magnetic resonance loop, giving a binary input to SCATS (with the added benefit of being able to create virtual loops for pedestrians). There has been development work on adding new SCATS interfaces but it has not been widely released. It is unclear if this will be included in the new SCATS upgrade.

There are multiple international vendors implementing traffic detection using Lidar, notably Velodyne’s Bluecity product.

Infrared and camera detectors

For pedestrians there is a trial in progress at Manly Wharf. This will likely also be applicable to areas outside schools; where there are usually few pedestrians, then many at once.

Infrared detection will be trialled at Manly Wharf where there is high pedestrian activity. 

This trial seeks to address the requirement to enhance efficiency and safety at intersections with large variable volumes of pedestrian traffic. 

This initiative aims to dynamically adjust intersection crossing times for pedestrians based on real-time detection. By using advanced detectors, we can:

• Identify if a pedestrian is waiting in a designated zone
• Determine the pedestrian volume in the detection zone
• Analyse the demand for pedestrians
• Measure the time it takes for pedestrians to clear the intersection.

“Pedestrian Detectors”, Intelligent Traffic Light Program, TfNSW

There is also an infrared camera trial for cyclists:

This trial aims to assess the performance of infrared cameras as a consistent and reliable method for detecting cyclists. Improved detection is anticipated to improve intersection efficiency and safety by reducing unused green phases and/or stranded riders. 

Transport for NSW is planning to demonstrate the cameras in use at the Future Mobility Testing and Research Centre on the Community Open Day – 31 October 2024.

The proposed trial site is the intersection of Gardeners Road and Bourke Street in Alexandria. 

This is a busy intersection due to the M8 off-ramp feeding into Gardeners Road. The traffic light phases are permanently configured to accommodate bicycles, regardless of their presence, causing unnecessary delays on Garden Road and impacting vehicle traffic flow. These factors make it an ideal testing site for this trial. 

“Bicycle Detectors”, Intelligent Traffic Light Program, TfNSW

Touchless pedestrian activation trial

There are prototypes of touchless buttons with audio feedback when it registers an intention to cross.

In Melbourne there are examples of buttons with an acknowledgement light but no automatic sensor. These lights, with or without the automatic sensor, mean pedestrians know whether they have to press the button or not. There are some disadvantages however:

When automatic introduction is activated, the red ‘wait’ light on the push button assembly is illuminated at all times that the pedestrian signals are not showing a green man (see Figure 1). This is a helpful indicator that the pedestrian does not need to push the button as a ‘call’ is already recorded. In bright sunlight, it can be difficult to see whether the wait light is illuminated or not. In a quick survey of sites in the CBD, it looks like not all push buttons have the ‘wait’ light. At William St/Bourke St, for example, only 1 of the 8 push button assemblies includes a ‘wait’ light.

Review of Signal Operations for Pedestrians in the City of Melbourne, David Nash, 2014

An example of this button is at Henderson Rd & Garden St. Acknowledgement of the intention to cross is displayed with a red light on the button and an audible beep.

Additional notes

The Sydney CBD effectively operates as a fixed time system during peak times

…but without the benefits of a fixed time system, such as pedestrian timers counting down until the end of the red light, rather than until the end of the green light.

SCATS will attempt to increase the length of a green phase for cars to satisfy the demand when the “degree of saturation” is high (as measured by cars travelling over magnetic resonance loops in the ground) – that’s what the “Adaptive” in Sydney Coordinated Adaptive Traffic System means. However, SCATS will only increase the green time such that it won’t make the cycle time higher than a given value (the maximum cycle time).

In the Sydney CBD, the 90 second figure is the maximum cycle time. If the SCATS algorithms determine there is no need (for cars), the cycle time will drop lower. Measurements of actual cycle time are recorded and can be accessed in the SCATS software, and to request this data you must pay $400 per intersection – see the section below on SCATS data cost.

With a 90 second cycle in the Sydney city, there usually isn’t any spare time in a cycle for SCATS to adjust the phase length (especially for bidirectional roads with more cycles), and will usually be running constantly at 90 second cycles (unless in off peak periods).

As a result of these decisions, this results in the Sydney CBD effectively operating as a fixed time system during peak – that is, the phase lengths are very similar to the expected, pre-programmed length.

The traffic signals tend to operate a peak of 1.5/2 hours in the morning and 2 – 3 hours in the afternoon.

In off peak periods it can sometimes run down to ~50 seconds.

That’s all folks!

Once again, I hope this article demonstrates why signal data should be as public as a train network timetable – and subject to the same public scrutiny and debate as one (even though complex trade-offs are involved) and I hope this document provides plenty of source material and an overview of the landscape for other advocates to dive into.

Note: I am not a trained traffic engineer; my background is computer science and I work as a software engineer. This analysis is a side project. Please let me know of any mistakes, corrections or improvements in the comments below (or privately to jake@jakecoppinger.com) and I will update if incorrect.

Further reading

Apologies for the haphazard referencing!

• ARRB Research Report 123 Traffic Signals: Capacity and Timing Analysis (1981). https://www.sidrasolutions.com/media/189/download
• Pedestrians should get the green light on traffic signal prioritisation, Cecilia Duong https://www.unsw.edu.au/news/2021/07/pedestrians-should-get-the-green-light-on-traffic-signal-priorit
• Python implementation of SCATS LX data reading (I haven’t read through), https://github.com/johntrieu91/scatsutilities
• Sydney CBD Cycleways Traffic Signals Optimisation, A Finlay, https://roads-waterways.transport.nsw.gov.au/documents/roads/bicycles/bicycles-sydney-cbd-traffic-signal-optimisation-2009.pdf
• “Traffic Signal Design Guide: Section 7 Phasing and Signal Group Display Sequence” (2016). URL appears to be currently broken (this is an archive link). Unsure if it’s been superseded. Great info on various types of pedestrian phase protection.
• Signal Data Information for Modelling report (Main Roads WA). Page 48 has some info on the LX timing format
• Review of Signal Operations for Pedestrians in the City of Melbourne, David Nash, 2014
• “The New Generation SCATS 6”, date unknown
• New TfL data shows success of innovative ‘pedestrian priority’ traffic signals, 24 February 2022, TfL
• Post metadata image: http://www.jctconsultancy.co.uk/Symposium/Symposium2014/PapersForDownload/A%20Review%20of%20SCATS%20Operation%20and%20Deployment%20in%20Dublin.pdf
• Traffic lights in ’s-Hertogenbosch; an interview, Biycle Dutch: https://bicycledutch.wordpress.com/2016/06/21/traffic-lights-in-s-hertogenbosch-an-interview/
• “The Evolution of Traffic Signal Operations”, TfNSW, March 2024: https://www.transport.nsw.gov.au/system/files/media/documents/2025/Active-Transport-Community-of-Practice-Evolution-of-Traffic-Signal-Operations-March-2024.pdf
• “Pedestrian Safety” brochure, September 2020, TfNSW, https://www.transport.nsw.gov.au/system/files/media/documents/2023/Pedestrian%20Safety.pdf (file removed, archived 2025-05-15 from the original https://web.archive.org/web/20250515002811/https://www.transport.nsw.gov.au/system/files/media/documents/2023/Pedestrian%20Safety.pdf), https://www.transport.nsw.gov.au/system/files/media/documents/2025/crs_pedestrian_safety.pdf
• “Traffic Signal Operation” (Superseded standard), RTA-TC-106, April 2010 (“First published in this form October 1991, Reformatted October 2000, Revised April 2010”), https://standards.transport.nsw.gov.au/_entity/annotation/6b38b3c7-b835-ed11-9db1-000d3ae011f9
• “Traffic Signal Operation”, TS 05493:1.0, RTA-TC-106, 18 November 2024, Revision 1.0 (replacing First issue as RTA-TC-106 effective 1 April 2010), https://standards.transport.nsw.gov.au/_entity/annotation/5c3b61e9-4ea5-ef11-b8e9-000d3a6b1b0b
• “Staying safe as a pedestrian”, TfNSW (NSW Centre for Road Safety), https://www.transport.nsw.gov.au/roadsafety/pedestrians/staying-safe
• “Pedestrian Priority at Traffic Signals”, Network Operations, TfNSW, February 2023, (supplied) http://files.jakecoppinger.com/traffic-signals/Pedestrian+Priority+-+Public+Presentation+V2.pdf

The post Shining a Light on the Traffic Signals of Sydney first appeared on Jake Coppinger.

See comments below, on Mastodon, Hacker News (51 comments), or LinkedIn (22 comments, 278 likes, 32k views)

Transport for NSW, the government agency which controls traffic signal timing in Sydney and elsewhere in NSW, has an excellent stated goal of increasing walking and cycling trips – and reducing pedestrian wait times at intersections.

However, there is no public data on traffic light timing in Sydney or NSW.

In the absence of traffic light timing data, and as we hold hope for it to become publicly available; the aim of Better Intersections is to crowdsource measurements and inform where positive changes could be made. You can add data yourself via a simple Google Form, and instructions are on the website about page.

Table of Contents

Better Intersections is a tool to record and visualise timing details for pedestrian and bicycle signals. It’s focused on Sydney & NSW, Australia, but is adaptable for anywhere in the world. This website is open source on Github (contributions welcome!), and the data is under an open license (ODbL license).

If you have ideas for improvements, please create a Github issue, comment below, email me at jake@jakecoppinger.com or message me on Mastodon (@jakecoppinger@aus.social).

It’s a work in progress! I’ve tinkered on it for a few afternoons and started working on it about two weeks ago.

This website bridges the excellent TfNSW Active Transport policy guidelines and pedestrians on the street themselves, allowing people on foot (and bicycle) to see their experience represented.

Why does the timing of pedestrian signals matter?

Transport for NSW, the government agency which controls traffic signal timing in Sydney and elsewhere in NSW, has an excellent stated goal of increasing walking and cycling trips – and reducing pedestrian wait times at intersections.

However, there is no public data on traffic light timing in Sydney or NSW.

In the absence of traffic light timing data, and as we hold hope for it to become publicly available; the aim of this project is to crowdsource measurements and inform where positive changes could be made.

This website bridges the excellent TfNSW Active Transport policy guidelines and pedestrians on the street themselves, allowing people on foot (and bicycle) to see their experience represented.

Increasing pedestrian priority and providing crossing opportunities at the right locations and along desire lines, reduces the risk of pedestrian injury at intersections by encouraging safer behaviours. Transport is currently rolling out measures at intersections to improve pedestrian priority in areas of high pedestrian activity. These measures may include automation of pedestrian crossings, reduced pedestrian wait times, provision of pedestrian crossings on missing legs and kerb ramps, where applicable.

— TfNSW Active Transport Strategy, page 30. Emphasis added.

Research has shown that 30 seconds is the longest a pedestrian will wait at a signalised crossings before attempting to cross against the ‘red man’. (Martin, A., 2006. Factors influencing pedestrian safety: a literature review (No. PPR241). Wokingham, Berks: TRL (Transport for London.)

From the above report:

Hunt, Lyons and Parker (2000) state that ‘Although no clear relationship has been established between pedestrian delay and casualties, a more balanced and responsive approach to the allocation of time at Pelican/Puffin crossings has the potential to make a substantial contribution to a decrease in pedestrian casualties as well as improving pedestrian amenity’. They point out that because pedestrians are more likely to become impatient when a red man continues to be shown during periods of low vehicle flow, the reduction of unnecessary delay for pedestrians should encourage pedestrians to use crossings correctly and reduce risk taking.

In 2020, people driving vehicles killed 138 pedestrians on Australian roads (Department of Infrastructure, Transport, Regional Development and Communications (2021) Fact sheet: Vulnerable road users, National Road Safety Strategy.)

But isn’t traffic light timing variable?

Sydney uses a system called Sydney Coordinated Adaptive Traffic System (SCATS) to control traffic signals, which makes use of many data feeds to control timing data.

Neither the inputs used, or the algorithm used to weigh the input data is public (as far as I know). This crowdsourced method of discrete measurements provides shows the output of the black box. In the case of outliers, multiple measurements (at different times of day/week) can be used to determine if the timing is variable.

Gehl Architects have a great methodology for measuring the overall impact of traffic light delays on pedestrians. At its most basic you walk along a street with two stopwatches;

• one you pause only when you’re walking;
• one you pause only when you’re waiting at a traffic light.

Divide one by the other at the end, and you have a single number that quantifies pedestrian delay walking along a street.

– Public spaces & public life: Sydney 2020, Gehl Architects, pg 142

The limitations of this method are that

• it cannot inform the exact problematic intersections;
• paths must be long enough to gather a large enough sample size to be statistically significant.

I’ve experimented with automating this method by recording a GPX (GPS) trace with a phone, uploading that file and getting a number instantly. Unfortunately the urban canyon effect (GPS signal loss caused by tall buildings) makes this method unreliable in cities, even with the remarkable sensor fusion on modern phones.

As with any of my projects, I am always open to collaboration. If you have any ideas, iterations or improvements, please drop me a line!

Technical details

This is a fairly simple Typescript app created using Create React App that I built in a few afternoons – please don’t consider it my finest code!

It is a static React app hosted on Cloudflare pages, and uses Mapbox GL JS to display the map (but could be easily updated to use Maplibre GL JS).

Google Sheets is treated as a backend (for simplicity using a Google Form for submissions), and the app makes use of OpenStreetMap node IDs as primary keys for intersections. This is definitely suboptimal but it’s quick to build – ideally I’d have a more custom form that is easier to use and doesn’t require a Google account – however using Google accounts for the form is a quick and easy method of minimising spam (and making it easy to identify) remove spam from a single person).

The OpenStreetMap API is used for looking up coordinates of OSM nodes and finding adjacent ways. There is currently one request per intersection made – this will not scale and I’ll likely need to cache the JSON (or hit an Overpass Turbo server instead).

The code is fully covered by Typescript types but doesn’t (yet?) have unit tests – it’s very easy to work with if you’re interested in tinkering with it!

Possible further work

• Adding a simplified version of the form
• Support OSM ways as the primary key rather than just nodes (useful for crossings across divided carriageways with multiple traffic light nodes per crossing leg)
• Tagging state roads and looking if these have longer wait times on average (probably)
• Scatter plot of crossing times vs number of lanes (as rough proxy for traffic volume)
• “sparkline” or other graphs of measurements for a given intersection
• Thinking about how to record relationships between intersections (ie. green wave/lack of green wave for pedestrians)
  • Using YOLOv3 or another off the shelf commodity computer vision model for recognising green/flashing red/red traffic lights for algorithm measuring cycle times

Related organisations

If you support better conditions for pedestrians and cyclists in Sydney/NSW, consider joining BetterStreets or 30 Please.

The post Mapping pedestrian traffic light timing in Sydney, Australia first appeared on Jake Coppinger.

The above poster is an orthographic (ie. no perspective) render of a 3D model of the underground train station.

I created a 3D model by walking around the station holding an iPhone 14 Pro running the 3D Scanner App, combining the segments in the 3d modelling software Blender and rendering it out as an image.

Aren’t the escalator and staircase crossovers a beautiful symmetrical pattern? I had no idea that’s how they were arranged. I think it’s a wonderful way to visualise architecture.

If you would like to download and print your own A3 poster, here are some high resolution copies. All are licensed CC BY-NC 4.0.

• 9921 × 7016, 600dpi JPEG, ~32MB: files.jakecoppinger.com/town-hall-station-poster/town-hall-station-poster-jake-coppinger.jpg
• PDF optimised for computers (RGB, interactive, ~4MB): files.jakecoppinger.com/town-hall-station-poster/town-hall-station-poster-jake-coppinger.pdf
• PDF optimised for printing (CMYK, marks & bleeds, ~50MB): files.jakecoppinger.com/town-hall-station-poster/town-hall-station-poster-jake-coppinger-print.pdf

Enjoy!

Read on if you’d like to learn more about how to create a model of your own local train station or a multi-story building.

Table of contents

Behind the scenes

Overview of method

• install the 3D Scanner App on an iPhone that supports LiDAR (iPhone Pro 12 or later, 2020+ iPad Pro)
• slowly walk around with the app to generate a 3D LiDAR model
• importing the 3D models into Blender
• align the models
• cleaning up any “noise” and chopping off half of some objects
• render the model to an image
• create a nice poster in Adobe InDesign!

Detailed method

Setting up your iPhone

These instructions are similar to those for my blog post on Generating aerial imagery with your iPhone’s LiDAR sensor.

I recommend using the app titled 3d Scanner App as it allows considerable customisation of the scan settings, though other scanning apps like Polycam would also work. It allows finishing a scan and extending later, though this can be buggy. I haven’t had a crash during capture – I’ve had Polycam crash halfway through a large scan losing all data.

Download 3d Scanner App and use the LiDAR Advanced mode. I recommend the using the following options for scanning streets:

• Confidence to low if you need the maximum 5 metre range for the sensor, otherwise set it to medium for a less noisy model
• Range to 5.0 metres
• Masking to None
• Resolution to 50mm (the lowest – for large models like buildings or streets)

In the app settings, make sure to set:

• Units to metric
• GPS can be turned on but likely won’t help you underground!

Capturing the model

Note that any commercial photography in a train station will require paperwork and special permission. Amateur/hobbyist/student use of a hand-held mobile device appears to be permitted in Sydney train stations, just treat it like taking a photo/video for Instagram or Tiktok – don’t photograph people and keep an eye on your surroundings.

The safety precautions are common sense, but have a read. Be careful not to cross over the yellow line when scanning. People will introduce artefacts into the model, so go in off peak or on a weekend to minimise crowds. The resolution and accuracy of the LiDAR scanning is not high enough to present any security concerns (any more than an Instagram photo).

When scanning, walk slowly with a sweeping motion to increase the maximum scanned width. If the area is wide enough to require a grid pattern, follow the same shape as a drone survey (an S-shape with considerable overlap). Not enough overlap or higher speeds mean the linear passes don’t connect correctly due to (I assume) inertial measurement unit drift.

Aligning and cleaning the models

You will likely need to do multiple takes of your desired building, and then combine them together.

Share your models from 3D scanner app as OBJ via Airdrop (or your preferred method), decompress the ZIP file and then import the OBJ into Blender (File -> Import -> Wavefront (.obj)).

Some basic Blender skills come in handy here – it’s not an easy program to use, and I am no expert. If you haven’t used it before I recommend following a beginner blog post or video tutorial. A few specific tips:

• Make liberal use of the numpad preset views: 1 for front on, 3 for side view, 7 for top, and 5 for toggling perspective
  • If you don’t have a numpad you will need to enable numpad emulation, see instructions at https://www.hack-computer.com/post/how-to-emulate-a-third-mouse-button-and-keypad-for-blender
• To remove noise or parts you don’t want, select the object, change into edit mode (tab key), and then change the Viewport Shading to Wireframe (with the globe button in the top right). Dragging with the left mouse button will select areas, or click c to drag a circle to select vertices (with +/- keys enlarging the circle).
  • Changing the shading to Wireframe means that you can select vertices “hiding” behind others that may not be directly visible
• I couldn’t figure out how to disable perspective and get the framing I wanted in a proper render, so I did a Viewport render with the background set to white, Material Preview as the Viewport Shading mode and Toggle X-Ray to remove the grid
  • The background still wasn’t perfectly white even though I set it to be, I had to use levels in Photoshop to get the background #FFFFFF. This may just be an error on my part!

Possible future improvements

• Capturing pointclouds instead of meshes, aligning them in Cloudcompare (I believe there are tools for aligning pointclouds), then either rendering straight to an image or generating a mesh and rendering only faces/normals facing the camera
• Reducing drift in capturing models. This would be algorithm heavy; maybe utilising survey markers or just known positions in a station, using Record3D, exporting sequences of .obj models (1 per frame), then constructing offline.
• A cross section from the north or south would be interesting! (this view is from the west looking east)
• Is it possible to create an accurate model with less takes? This would require less stitching in Blender
• Are there ways to create a full 3D model, then just not render faces/normals that aren’t facing the camera?
  • Better yet, not rendering faces/normals that are less than x degrees incident to the camera to reduce noise for flat surfaces
• How to handle glass surfaces better (LiDAR isn’t great with glass out of the box)
• 3D printing models

Please let me know if you use this as inspiration for modelling stations elsewhere!

Related/further reading

• Catharia O, Richard F, Vignoles H, Véron P, Aoussat A, Segonds F. Smartphone LiDAR Data: A Case Study for Numerisation of Indoor Buildings in Railway Stations. Sensors (Basel). 2023;23(4):1967. Published 2023 Feb 9. doi:10.3390/s23041967: https://www.researchgate.net/publication/368448078_Smartphone_LiDAR_Data_A_Case_Study_for_Numerisation_of_Indoor_Buildings_in_Railway_Stations

• Great tips for better 3D scans: https://docs.3dscannerapp.com/

• 3D maps (orthographic projection) of every London Underground station

• https://www.whatdotheyknow.com/request/maps_of_public_corridors_on_larg
• https://www.ianvisits.co.uk/articles/3d-maps-of-every-underground-station-ab-14630/

The post Subterranean Sydney: A cross-section of Town Hall Station made with iPhone LiDAR first appeared on Jake Coppinger.

Why is this useful?

Usually for such a task you would use a drone and process with WebODM (or Pix4D), but there are areas that are unsafe or illegal to fly in. I’ve previously detailed how to generate imagery using a bicycle helmet mounted GoPro camera, however this can include artifacts where there are lots of people. The helmet camera method requires a decent GPS lock (unsuitable indoors, urban areas or under a bridge) and has relatively low detail.

Again, why might you want to do this? With your own high detail and up-to-date models and street imagery you could:

• Map new street interventions, like bollards, modal filters or raised crossings
• Record pothole locations (and their depth!)
• Take measurements such as road and cycleway widths around crowds of people in urban centres
• Measure footpath obstructions in 3D and rate pedestrian amenity
• Survey features underneath large highways
• Survey street parking using the new OSM spec: wiki.openstreetmap.org/wiki/Street_parking
• Map indoor pedestrian areas in OpenStreetMap for better pedestrian routing
  • The Transport for NSW Connected Journeys Data team is currently doing a fair bit of this work: https://www.openstreetmap.org/changeset/133107592
• Attach your iPhone to your bike and generate LiDAR point clouds of the kerb and cycleway infrastructure (it works, just go slow!)

This method results in very high detail (5mm resolution if desired) 3D models and accurate orthoimagery. Manual georeferencing is required (which I also explain how to do) which limits the confidence in alignment. This is a proof of concept – if you have corrections/suggestions/ideas to improve the method, please comment below or on Mastodon!

Note: This method also provides a solution to creating 2.5D oblique orthophotos from drone imagery.

Process overview

This guide covers how to:

• Capture a 3D model using 3d Scanner App (recommended) or Polycam
  • The iPhone LiDAR sensor has 5 metres max range, so you’ll need to walk around
• Export the model to an .obj file with textures
• Rotating the model in Blender to the required orientation
• Use the odm_orthophoto program inside the OpenDroneMap Docker container to generate a raster .tiff
• Georeference the tiff using QGIS
• Uploading the Geotiff to OpenAerialMap to generate a tileset, viewable in the OpenStreetMap iD editor or a Felt map with a custom layer

Capturing the model

Capturing a 3D model on an supported iPhone is easy. I recommend using the app titled 3d Scanner App as it allows considerable customisation of the scan settings. It allows finishing a scan and extending later, though this can be buggy. I haven’t had a crash during capture – I’ve had Polycam crash halfway through a large scan losing all data.

Download 3d Scanner App and use the LiDAR Advanced mode. I recommend the following options for scanning streets:

• Confidence to low. This extends the range of the LiDAR sensor readings used at the expense of more noise. You can clean up this noise in the processing settings or Blender.
• Range to 5.0 metres
• Masking to None
• Resolution to 50mm (the lowest – for large models like streets)

In the app settings, make sure to set:

• GPS tag scans to ON
• Units to metric

When scanning a street, walk (or cycle) slowly with a sweeping motion to increase the width. If the area is wide enough to require a grid pattern, follow the same shape as a drone survey (an S-shape with considerable overlap). Not enough overlap or higher speeds mean the linear passes don’t connect correctly due to (I assume) inertial measurement unit drift. I’m unsure if the GPS information is used in the sensor fusion (via ARKit), please comment if you know!

Exporting and preparing the model

In the 3d Scanner App use the Share button, then select the .obj file type. Send this to your computer (Airdrop works great if using macOS). If using Polycam, set “Z axis up” in the mesh export settings.

Rotating the model into the correct orientation (required for 3d Scanner App)

Unfortunately the 3d Scanner App exports objects with the Z axis as “up”, while the odm_orthophoto program expects the Y axis to be “up”. Confusingly, you can skip this step if using Polycam if exporting with “Z axis up” in the mesh export settings, though Blender shows the Y axis as up in this export. If you know why this is, please leave a comment!

To rotate the model, import it to Blender and rotate it 90 degrees.

• First, install Blender via your preferred method (https://www.blender.org/download/).
• Open Blender, delete the initial default cube (right click -> delete, or x hotkey)
• Import the .obj file: File -> Import -> Wavefront (.obj)
• (optional: you can view the pretty texture by selecting “viewport shading” in the top right (the horizontal list of sphere icons))

• To rotate
  • Click the object and make sure it is selected (orange border)
  • Press hotkey r (from any view)
  • Press x to only allow rotation on X axis
  • Type 90 (or desired degrees to rotate)
• Optional: You can check if the rotation is correct by pressing numpad key 1. If you don’t have a numpad you will need to enable numpad emulation (see instructions at https://www.hack-computer.com/post/how-to-emulate-a-third-mouse-button-and-keypad-for-blender).
  • The rotation is correct if you have a “birds eye view” in the numpad key 1 view, where the blue Z axis is towards the top of screen and the red X axis is towards the right of screen

• File -> Export as an .obj to the same folder with a new name (eg. blender_export.obj)
  • Note: Blender doesn’t create a new texture .jpg. If you export to a different folder the path to the .jpg in the .mtl file will need updating.

Generating the raster orthophoto

Use the odm_orthophoto command line tool to generate a raster orthophoto from a .obj file. This tool is available at https://github.com/OpenDroneMap/odm_orthophoto but has a considerable number of dependencies.

I believe the easiest method currently is to install WebODM locally, copy the .obj and texture files (.mtl and .jpg) into the Docker container and then run the program from inside the Docker container.

Installing WebODM locally

Running the software using Docker is a breeze. Install Docker from https://www.docker.com/ (or your preferred method) and then:

git clone https://github.com/OpenDroneMap/WebODM --config core.autocrlf=input --depth 1
cd WebODM
./webodm.sh start 

See https://github.com/OpenDroneMap/WebODM#getting-started for more details. WebODM itself is excellent and great fun if you have a drone!

Copying the object into the ODM Docker container

You can start a shell in the container with the following command:

docker exec -it webodm_node-odm_1 /bin/bash

Make a new directory to keep your files in

mkdir /iphone_model
cd /iphone_model

In another shell, copy the object and texture files from your local machine into the new Docker container folder. docker cp can only copy one file at a time.

cd path/to/your/model/
docker cp blender_export.obj webodm_node-odm_1:/iphone_model/
docker cp blender_export.mtl webodm_node-odm_1:/iphone_model/
# Note: The blender .obj export doesn't create a new texture .jpg
#   If your Blender export wasn't in the same directory, check
#   update the path in blender_export.mtl
docker cp textured_output.jpg webodm_node-odm_1:/iphone_model/

Running odm_orthophoto

In the shell you started in the docker container above, run the following command:

cd /iphone_model/
/code/SuperBuild/install/bin/odm_orthophoto -inputFiles blender_export.obj -logFile log.txt -outputFile orthophoto.tif -resolution 100.0 -outputCornerFile corners.txt

The resolution argument is how many pixels per metre – this may require changing.

Exporting the orthophoto out of the Docker container

To copy the generated orthophoto out, from a shell on your local machine run:

docker cp webodm_node-odm_1:/iphone_model/orthophoto.tif .

Use a similar command to extract the log file if required.

Georeferencing the orthophoto

Georeferencing is the process of specifying the location and orientation of the image so it perfectly aligns with maps or GIS software. While a rough location (with a moderately incorrect rotation) is stored in the model, it appears to be removed by the Blender rotation step. If you know how to fix this please comment below!

To do this:

• Install QGIS by your preferred method: https://www.qgis.org/en/site/forusers/download.html
• Install the plugins (via the Plugins -> Manage & Install plugins… menu)
  • QuickMapServices (to pull in Bing satellite imagery easily)
  • Freehand raster georeferencer (a beginner friendly georeferencing tool)
• Add a Bing satellite base layer: Web -> QuickMapServices -> Bing -> Bing Satellite
  • Feel free to choose another satellite background of your chosing
  • If you’re in NSW: the NSW LPI Imagery is likely the most detailed, follow: https://www.spatial.nsw.gov.au/products_and_services/web_services/qgis
• Zoom & pan to the rough location of the 3d scan (the initial .tif location will be wherever you’re viewing)
• Drag the .tif output by the previous step into the sidebar (it won’t be visible yet as it is not aligned)
• Go to Raster-> Freehand raster georeferencer -> Add raster for freehand georeferencing and select the same .tif
• Use the Move, Rotate and scale buttons in the toolbar to align your orthophoto with the imagery background (tip. Hold Cmd or Ctrl before scaling to keep the aspect ratio)

• Click the “Export raster with world file” button (Green on the right with exclamation marks).
• Check the “Only export world file for chosen raster” button. Make sure to do this before chosing the image path.
• Select the existing .tif image and press OK
• Remove the orthophoto from the QGIS sidebar (right click -> remove layer)
• Drag the existing .tif image back into the sidebar. QGIS will now find the worldfiles next to it (orthophoto.tif.aux.xml and orthophoto.tfw) so it will be positioned in the right place

Export geo-referenced GeoTIFF (without worldfile)

If you would like to upload the GeoTIFF to OpenAerialMap or somewhere else, you will need to “bake in” the location into the GeoTIFF itself, rather than in the worldfile – OpenAerialMap can’t read the worldfile.

To do this:

• right click your orthophoto layer (after the above steps) and click Export -> Save As…
• Set CRS to your desired coordinate system (if not yet in a coordinate system, I assume you should use EPSG 3857 if you want it to be aligned with OpenStreetMap tiles, but this is the limit of my current understanding – I haven’t studied surveying yet!).
• To avoid confusion, create a new subfolder and save it with the default settings (eg. make folder qgis_export and save as orthophoto.tif).
• You now have a nice georeferenced GeoTIFF!

Uploading to OpenAerialMap

If you want the imagery to be publicly viewable and accessible from the OpenStreetMap iD editor, OpenAerialMap is a free place to host your imagery.

This is the imagery from the above example: https://map.openaerialmap.org/#/-18.6328125,18.562947442888312,3/latest/

I’ve heard of plans for a relaunch of the website, but currently the upload form can be finicky.

• Open the explore page: https://map.openaerialmap.org/
• Sign in (only Google & FB Oauth supported)
• Press upload
  • Currently uploading from local file doesn’t appear to work, see https://github.com/hotosm/OpenAerialMap/issues/158 for updates
  • Uploading via Google Drive with my account (2fa enabled, Gsuite) fails with This app is blocked: This app tried to access sensitive info in your Google Account. To keep your account safe, Google blocked this access.
    • Enabling less secure apps is not possible for 2fa accounts. Otherwise, if you’re comfortable turning it off you can do that here: https://myaccount.google.com/lesssecureapps
  • Using a URL is likely the only way. Creating an S3 bucket is one way. If you have a fast connection it would be faster to run a local webserver with Python and running ngrok to make it publicly available. I recommend not keeping this server running for longer than necessary. Eg:

cd qgis_export
python3 -m http.server 8080
ngrok http 8080
# Your file is now available at https://SOME_PATH.ngrok.io/orthophoto.tif

Specify this url in the form and add other details, then press upload.

Limitations

• Manual alignment limits the real world accuracy of imagery
• Drift during long model captures occurs. My understanding is drift occurs more when there are sudden or fast movements. The 3d Scanner App unfortunately doesn’t warn you when you’re moving to fast, but Polycam does. As far as I know, the iOS ARKit doesn’t attempt to reconcile drift when completing a loop/circuit.

Future work

• Automation! This process is slow but it works.
  • Adding a Makefile or other compile tooling to https://github.com/OpenDroneMap/odm_orthophoto would skip the requirement to install WebODM and transfer files to/from the Docker container
  • Rotating the model could be added (behind a flag to be backwards compatible) to the odm_orthophoto script
• Generating pointclouds (supported by 3d Scanner App) and then exporting as a raster from CloudCompare. This might make larger captures possible.
  • If there is a way of addressing drift of pointclouds for multiple captures – let me know how!
• Georeferencing using ground control points rather than a freehand referencer
• Creating street facade montages and evaluating doors & soft edges (Jan Gehl (1986) “Soft edges” in residential streets, Scandinavian Housing and Planning Research,3:2,89-102, DOI: 10.1080/02815738608730092)

Let me know if you have any corrections/suggestions/feedback!

The post Generating aerial imagery with your iPhone’s LiDAR sensor first appeared on Jake Coppinger.

I’ve built safecyclingmap.com, an open-source proof of concept map that renders cycleways and streets down to the individual lanes, to assist cyclists picking safe routes and support advocacy for reallocating road space to more efficient modes (as supported by the TfNSW Future Transport Strategy).

I’ve open sourced the backend infrastructure so that others can incorporate the details into their web and phone maps (see osm2streets-vector-tileserver or read below). This builds upon the incredible work of Dustin Carlino who leads the A/B Street traffic simulation project.

I wrote the initial vector tileserver from scratch in one afternoon in Typescript, calling the JS bindings for the Rust osm2streets code compiled to wasm. It generates GeoJSON which is then returned by Koa as a protobuf. I’ve spent some more hours improving performance, but there is much that could be improved.

The TfNSW Future Transport Strategy states “We will focus on getting more out of our existing investments, by reallocating road space to more efficient modes of transport like buses, walking, cycling and micromobility devices.”

This blog post explains how and why I built it.

Existing detailed street maps

Most street maps just use a line for each street:

Usually the colour of this line denotes how large or important the road is. If your city doesn’t have much dedicated bicycle infrastructure, picking a safe road can be a challenge.

In 2020 Google announced that it would be implementing more detailed street maps in selected cities:

Soon, you’ll be able to see highly detailed street information that shows the accurate shape and width of a road to scale. You can also see exactly where sidewalks, crosswalks, and pedestrian islands are located… We’ll start rolling out detailed street maps in London, New York, and San Francisco in the coming months, with plans to expand to more cities over time.

A more detailed, colorful map, Aug 18th 2020, Google.

Justin O’Beirne, a prolific writer on the consumer maps competitive landscape, wrote in April 2022 that Google has released detailed street maps in 40 cities so far. No cities in Australia make up this list.

Open source competition

OpenStreetMap (OSM) is a free, editable map of the whole world that is being built by volunteers largely from scratch and released with an open-content license. Because of the license it has grown quickly and now rivals (and often exceeds) the detail of commercial map providers. OSM data is used in Instagram, Wikipedia, Strava, Snapchat, Uber and many others.

An project using this data has been quietly generating highly detailed street maps. Dustin Carlino started building the A/B Street traffic simulation app since around 2019, and as part of this released the osm2streets project. osm2streets takes OSM data and draws detailed shapes of streets – though didn’t yet support rendering into generic web or phone “slippy” maps as detailed on this Github issue.

I wrote an open source vector tileserver osm2streets-vector-tileserver to enable “slippy” web and phone maps to load the detailed car and bicycle lane information that osm2streets generates.

As a proof of concept, I’ve built safecyclingmap.com to consume the tiles produced by osm2streets-vector-tileserver. It is a visualisation of this detailed street data to assist cyclists with picking safe routes in cities with little dedicated cycling infrastructure.

Technical details

Bear in mind: I initially wrote the tileserver in one afternoon, and I haven’t put much effort into reliably hosting the infrastructure yet, so this could be improved.

The backend works by taking vector tile requests from a frontend (with a url like https://api.safecyclingmap.com/tile/18/241180/157318).

Once a request is received, the tile at zoom level 15 that contains the requested tile as a subset is calculated.

The OSM XML is then fetched from a local Overpass Turbo instance for that superset tile, and the osm2streets street network output for this tile is generated (using the JS to wasm (Rust) bindings at https://www.npmjs.com/package/osm2streets-js).

Requested subset tiles are then generated using the street network object that covers the superset tile.

The geojson layers are then generated and combined, and returned to the client as a Protobuf.

If you have any questions or are interested in contributing, come and join the Matrix room at #osm2streets-vector-tileserver:matrix.org or raise a PR at https://github.com/jakecoppinger/osm2streets-vector-tileserver.

Caching

I’ve used two least-recently-used caches to cache the osm2streets network output, as well as the final individual vector tiles. Vector tiles requested that fit within the bounds of an in-progress zoom level 15 Overpass Turbo download (and associated network generation) busy-wait for the cache to become available (with a timeout if there is an error).

Currently it works in all of Australia, as downloading all map data for the entire world needs a larger hard drive! I needed 8GB of swap space to set up the docker image without a failure.

The frontend is currently hosted on Github pages (behind Cloudflare) because haven’t yet configured SSL config on AWS S3 for this domain.

The backend currently runs on a Vultr VPS, using Cloudflare Tunnel to send requests to localhost:3000.

Further work

Further features that I think would be useful:

• Making it work worldwide by passing requests outside Australia to a public Overpass instance from https://wiki.openstreetmap.org/wiki/Overpass_API#Public_Overpass_API_instances – with the new caching there shouldn’t too much traffic
• Contributing to osm2streets to pass extra attributes in the vector tiles returned, such as:
  • Speed of the street, to be able to colour code streets
  • Level of the street, to hide underground roads

The post Lane-accurate street maps with OpenStreetMap – writing a vector tileserver for osm2streets first appeared on Jake Coppinger.

This technical guide details how you can create your own orthorectified (aka satellite view/bird mode) imagery, point clouds and 3D models of streets with nothing but a 360 degree camera mounted on bicycle helmet, and the open source photogrammetry software OpenDroneMap.

Why might you want to do this? With your own up-to-date and highly detailed orthorectified imagery you could:

• quantify and communicate inefficient road space allocation
• record necessary infrastructure repairs
• take measurements such as lane and cycleway widths
• measure footpath obstructions in 3D and rate pedestrian amenity
• map kerb features on OpenStreetMap
• survey street parking using the new OSM spec: wiki.openstreetmap.org/wiki/Street_parking
• 3D print a model of your home street!

Drones are great for surveying and mapping things on the kerb (parking/public space/road widths/building shadows) but there are a lot of places you can’t fly a drone.

OpenDroneMap is usually used to combine a set of geotagged drone images into one coherent 3d model, but it can also be used with any geotagged images.

Roughly the process is as follows:

• Take spherical images on a 360 degree camera
• Import the files from the camera
• Optional: Convert the photos into rectangular, spherical images (for GoPro: convert from the proprietary GoPro format to standard spherical images using GoPro Fusion v1.2)
• Start/login to WebODM, upload the images with fisheye lens setting for 180 degree images or spherical for rectangular 360 degree images and generate imagery & model

Please comment any projects you make after reading this guide, or if you have any questions! There are certainly issues with this process with I’ve added under Limitations.

Table of Contents

Buying a 360 degree camera

360 degree cameras can be very expensive (the ones mounted on Google Street View cars are possibly $45,000!), but GoPro makes relatively affordable models. See https://help.mapillary.com/hc/en-us/articles/115001465989-About-360-cameras for more suitable cameras. One common second hand (and now unsupported) model is the GoPro Fusion, which I’ll base this guide on.

Note: Seth Deegan has written a PowerShell script for using Insta360 cameras with ODM. I haven’t tried it but it seems worth a look: https://github.com/lectrician1/Insta360-2-ODM

To generate a detailed model you will need to capture enough images close together. You could mount it to the top of your car, but unless you bought an expensive elevated mount much of the image would just be your car roof!

Some advantages of taking imagery from a bicycle:

• The unobstructed field of view is larger so ODM gets more data for a better model
• The average speed is lower permitting more photos (the maximum self timer frequency of this camera is 2 shots/second)
• You can capture images from different places in the street for more data, such as the road, footpath and bike lanes
• If you’re putting in this much effort to study public space and urban planning you probably like bikes

You’ll need to buy a helmet mount for the GoPro or your chosen camera. GoPro makes a bicycle helmet mount: https://gopro.com/en/us/shop/mounts-accessories /vented-helmet-strap-mount/GVHS30.html (though many non-branded mounts exist on eBay/Amazon too). Be aware a camera mounted on a bicycle helmet is possibly a safety risk to yourself if you crash – be careful.

If you have the ability to mount the camera on a long pole above your head, this will increase the unobstructed field of view and improve the perspective of the street which will improve the results. I imagine this would attract even more attention! If you have tips for building rigs like this please leave a comment. Andrew Harvey wrote an OpenStreetMap diary entry on his setup here: https://www.openstreetmap.org/user/aharvey/diary/42139

GoPro Fusion specific tips

Set the camera to:

• Timelapse photo mode (icon of camera & timer circle)
• Image frequency to 0.5 seconds
• Enable GPS geotagging

Mount the camera to the helmet, wait until the GPS location icon turns solid, and start capturing! Sometimes the first few shots don’t have any location in the EXIF data but this doesn’t seem to confuse WebODM, you can check this with identify -verbose image or your metadata viewer of choice.

Preprocessing the images

The GoPro Fusion will output two 180 degree “fisheye” images to two separate SD cards. Copy all these images to a folder onto your computer.

For an alternate method, see instructions under the generating equirectangular image heading.

Unfortunately, only the “front” images have geotagged information, so you’ll need to copy the exif data from each “back” image to each similarly named “front” image.

A quick and dirty way of doing this is generating a list of terminal commands, that copies all exif tags from each “front” to each “back” photo:

ls -l | grep "GF" | sed -E "s/^.* GF(.*).JPG*/exiftool −overwrite_original_in_place -gps:all -tagsFromFile GF\1.JPG GB\1.JPG/g" > script.sh

I’m sure there are better and easier ways of doing this, please let me know if you come up with one.

Generating the model and orthorectified imagery with Web OpenDroneMap

WebODM (https://opendronemap.org) is an absolute marvel of open source engineering. Unfortunately, generating 3D models takes some serious computing horsepower. You can either use the paid cloud version (WebODM Lightening) at https://webodm.net/ (fast) or run the software on your own computer (a few hours/overnight/days depending on the number of images).

You can directly upload images to WebODM Lightening to process, however you don’t get some friendly/useful features like a browsable map and 3d model viewer. I recommend setting up WebODM locally and adding WebODM Lightening as a “processing node”, so you get the power of the cloud and the extra features of WebODM.

Setting up WebODM locally

Running the software using Docker is a breeze. Install Docker from https://www.docker.com/ (or your preferred method), allocate as much memory & CPUs as you can, and then:

git clone https://github.com/OpenDroneMap/WebODM --config core.autocrlf=input --depth 1
cd WebODM
./webodm.sh start 

See https://github.com/OpenDroneMap/WebODM#getting-started more more details including GPU acceleration.

You’ll now be able to open WebODM on http://localhost:8000

Optional: Adding WebODM lightening as a processing node

Click lightening network in your WebODM sidebar and login.

Generating the model & selecting the correct options

Add a new project, click “Select images & GCP”, select your images, then you will see options for processing your imagery. If you’re using WebODM Lightening, make sure to set the Processing Node appropriately.

Some options I’ve found are required:

If you are using the raw 180 degree “fisheye” images straight out of the GoPro Fusion, set camera-lens to fisheye (this is critical). If you are using equirectangular images set to spherical. WebODM seems to be unable to auto-identify the lens type in both cases.

• enable sky removal – this uses AI to mask out the sky in each frame so that there are less artifacts in the final model
• enable auto-boundary
• enable bg-removal

This is just from my trial and error, there may be better option configurations.

You will likely need to enable “resize images” to 2000px or it is very easy to run out of memory.

Though it is out of scope for this article, you can also set up a VPS instance to speed up the process if you don’t want to use the hosted cloud processing tool. Ive tried this, but it’s probably more effort than it’s worth unless you’re making a business out of this.

Even with an EC2 instance with 124GB of RAM and 324 images at 5760 × 2180 pixels I ran out of memory – remember to enable some swap space if you want to run at full size: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04

Output accuracy: taking measurements

OpenDroneMap can make measurements of the 3D model. I’ve found these are usually quite accurate; they may have small “measurement” errors due to artifacts or distortion but will tend towards the correct value – GPS coordinates ensure the scale is correct.

For example, measuring the width of the green paint of the cycle path:

Large path:

• OpenDroneMap: 4.54 metres
• Tape measure: 4.56 metres

Small path:

• OpenDroneMap: 1.89 metres (though this varies slightly due to distortions)
• Tape measure: 1.89 metres

Alternative: Generating and processing equirectangular spherical images

I’m currently unclear whether this method is faster or produces better results than the raw 180 degree images. I’m very interested to hear if you have experience (and I’ll update this if I get more evidence). I believe the GroPro Max generates these images in camera, so this won’t be required.

If you use the GoPro Fusion, unfortunately because this camera is no longer supported GoPro discontinued the software: https://gopro.com/en/au/news/fusion-end-of-life

You’ll (unfortunately) need to use the proprietary GoPro Fusion software to generate spherical images. I’ve found on an M1 Mac (Monterey) the only version that still runs is 1.2, 1.4 just crashes. The least dodgy download I can find is https://macdownload.informer.com/fusion-studio/1.2/.

If you make an open source solution (maybe building from https://stackoverflow.com/questions/37796911/is-there-a-fisheye-or-dual-fisheye-to-equirectangular-filter-for-ffmpeg) please let me know! Edit: Looks like this repo will do it. https://github.com/trek-view/fusion2sphere

This camera has two SD card slots – one for the front facing camera, and one for the back facing camera. Sometimes only one camera works for part of the shoot and the proprietary software refuses to stitch any of the photos at all! The only way I’ve found to combat this is to format the card in camera before each shoot. Spending more on a GoPro Max may solve a lot of headaches!

Limitations

Little helmets appear! I’ve tried some techniques but haven’t found a proper solution that doesn’t degrade the model/orthophoto structure.

Appendix: Things that didn’t work

Removing helmet artifacts by cropping

I tried using the convert and mogrify tools (part of the amazing open source ImageMagick suite) can crop the spherical photos and retain the geographical information! The cropping works great, but OpenDroneMap doesn’t seem to be able to understand a cropped equirectangular image. If you’d like to try to get this working, here are the steps I took.

First install ImageMagick (sudo apt-get install imagemagick on Linux/brew install imagemagick on macOS).

The general syntax is:
convert -crop {x_size}x{y_size}+{x_offset}+{y_offset} inputfile outputfile

See https://deparkes.co.uk/2015/04/30/batch-crop-images-with-imagemagick/ for more detail and helpful diagrams.

Helpfully, the helmet artifact is always at the bottom of frame, so the x offset and y offset will be zero.

To crop 700 pixels off the bottom of a spherical image, you can use

convert -crop 5760x"$((2880-700))"+0+0 inputfile outputfile

To batch convert a number of files, where the input files are in ./input/, and you have made an output directory ./output/, you can use:

mogrify -monitor -crop 5760x"$((2880-700))"+0+0 -path ./output ./input/*

Removing helmet artifacts by adding a mask

Adding custom drawn masks, either covering just the helmet or also the surrounding black “void” either:

• removed the helments but caused extra “holes”/missing segments in the ground
• didn’t remove the helmets

I think this may be due to clashing behaviour of the bg-removal and sky-removal flags with manual masks.

ODM requires a mask image for every image with _mask.{EXT} as a suffix (where EXT replaces the original extension).

The two mask types I tried:

If you know what may be happening please let me know!

Further research/experimentation

• How many “capture lines” per street are necessary to make a decent model? I didn’t have any luck with one but it may have been my settings. I used ~4 for the above model.
• How much better are models when the camera is on a stick?
• Improved WebODM settings for generation

Prior art

• “A Satellite in Your Pocket: Ground Based Action Cameras to Create Aerial Perspective for OSM Editing”. This was recorded at OpenStreetMap US: Connect 2020 by Sean Gorman. The company Pixel8 doesn’t appear to exist any more.
  https://www.youtube.com/watch?v=tfab-iuWlsQ
• Twitter thread by @klaskarlsson (@klaskarlsson@fosstodon.org on Mastodon) on creating a model of his house using a “GoPro on a stick”: https://twitter.com/klaskarlsson/status/1583401741386936320
• “Create aerial imagery base on 360° pictures” – discussion in the OpenDroneMap community: https://community.opendronemap.org/t/create-aerial-imagery-base-on-360-pictures/12339

The post Creating aerial imagery with a bike helmet camera (GoPro) and OpenDroneMap first appeared on Jake Coppinger.

It’s open source on Github, however I am no longer hosting it due to server costs.

The web app displays a map side by side with a Marey diagram. When you move your mouse across the horizontal (time) axis of the diagram, the locations of the buses are displayed at that point in time. The colour of the line represents the occupancy (warmer colours -> higher occupancy) of the service.

Sydney Transit Graph uses a Marey diagram to view bus congestion in Sydney – showing the current (and historial) location of every bus.

Clicking on a line on Sydney Transit Graph highlights the position of that particular bus on the map.

The slope of the line indicates how fast the bus is going (distance verses time). A shallow line can be read as bus congestion.

Long headways

A large vertical gap between lines indicate a large distance gap between buses, and a large horizontal gap means a large time gap. Usually these are equally big or small!

In this example, there is a bus heading north-west that doesn’t have a service behind it. More people accumulate at the bus stops in that time, leading to the occupancy of the next bus being higher (orange means higher occupancy). The next bus after that also appears delayed, leading to…

Bus bunching

When the lines are bunched together, there is bus bunching – the phenomenon of multiple buses on the same route arriving at the same time (see the bus bunching Wikipedia article for an explanation on why this occurs)

Why does bus congestion matter?

Less bus congestion results in faster trip times, which in turn reduces labour costs, decreases bus headways (or reduces need for new buses) and improves passenger throughput.

Possible future work

Further analysis that would be interesting would be:

• Correlating congestion hotspots with OpenStreetMap lane data to better inform bus lane implementation policy
• Analysis of pre/post COVID-19 transit patterns
• Including train data in the pipeline (it’s the same API)
• Improve interpolation of routes on the frontend so it’s not just a straight line average between dots (interpolating along the route instead)

I’m not sure if I’ll have the time to get to this, but please let me know if you’re interested in analysis with this tool! I’ve been busy with life and other side projects so I haven’t made updates in a little while.

How it works

The web app is built with vanilla JS (TypeScript), p5.js for the graph, and Mapbox GL JS for the map (which uses OpenStreetMap data).

Data is sourced from Transport for NSW via OpenData. Realtime data is ingested every few seconds via an service running on AWS EC2 to store Protocol Buffer files on AWS S3.

When requested via an API, a service on AWS Lambda retrives these Protocol Buffer files and computes the trip progress and occupancy of every bus in a specified time window.

I adjustably quantise the number of Protocol Buffers requested (and thus the number of results) via looking up available timestamps.

Trip progress is calculated by comparing positions against bus route shapes. These route shapes have been precomputed from GTFS database dumps (via Sqlite) and stored on AWS S3 as JSON.

It’s definitely not the cleanest or most efficient architecture, however I haven’t had the time to make a big refactor recently.

Known issues

• There are some line rendering issues in Safari

Prior art

• Visualizing MBTA Data: An interactive exploration of Boston’s subway systemMike Barry and Brian Card: https://mbtaviz.github.io/

Conclusion

Contributions are very welcome! If you have ideas on a project or analysis I’d love to hear. Feel free to contact me at jake@jakecoppinger.com or leave a comment.

The post Visualising Sydney bus congestion with Marey charts first appeared on Jake Coppinger.

In mid 2021 Australia had an acute vaccine shortage for various reasons, which meant vaccine appointments were extremely hard to come by – NSW had just declared a national emergency.

To help Australia increase it’s vaccination rate ASAP I built GPVaccineSearch.com over a few weekends in July/August, an open-source (on Github) vaccine appointment search engine, as a side project.

As of Jan 2022, 23k users visited the site, and 5,809 unique users started a HotDoc vaccine booking after using my site!

After the shortage eased in October 2021 and the government booking website improved traffic dropped off, but here is the story of how I built it!

Wanting to help Aussies get vaccinated as soon as possible, I searched for every possible data source (and considered crowd-sourcing phone call responses). I found that HotDoc (a Melbourne healthcare booking company) had lots of data on available bookings but had not yet built a way to search across clinics for the soonest appointments.

Always keen for a side project I managed to deploy a headless Chromium instance to AWS Lambda to walk through GPs near the user and find appointment times, and soon had a working demo. As you can imagine, I was concerned about the load this would add to the HotDoc site and resources to run it so I added strong rate limiting, but it’s hard to emphasize how in demand vaccines were at the time – I had friends asking me via social media to run queries for their location!

After posting on Reddit and sharing with a few friends, almost 2800 people visited on the first day. The most requested feature was searching by postcode rather than geo-location: once I implemented that I found most users preferred using that option. The Pfizer vaccine also became available via GPs which further added to the surge of visitors.

I got in contact with a friend of a friend working at HotDoc so I could make sure they were comfortable with myself using their data and API.

They were happy with the project and agreed my rate limiting threshold was well chosen, and submitted a PR so they could track referrals. Altogether the experience of communicating with HotDoc was excellent – they also emphasised how committed they are to helping Australians access vaccines.

After vaccines became more widely available visitors dropped off, though a smaller second wave visited when I added booster search support.

All up

Overall this was one of my favourite side projects I’ve built – it was a lot of fun to try and throw my hat in the ring to make this problem a bit easier to solve. It really meant a lot to me that I managed to help a few people in the process. Thanks for reading!

I think it’s important to acknowledge the number of people who used gpvaccinesearch.com pales in comparison to how many people found their vaccines using HotDoc itself – they’ve built a truly brilliant product.

Ken Tsang (the author of COVID-19 Near Me) also wrote up an article about his vaccine booking search engine titled “Why I decided to build my own vaccine booking search engine instead of using the Government’s one”.

Currently, a side project I’m working on is Gatho (gatho.party), a platform for hosting small events. It supports one click RSVP links for guests, and can integrate with a Matrix group chat. It’s open source and works great, let me know what you think!

Some more graphs of gpvaccinesearch.com, because everybody loves graphs!

The post How I helped ~6,000 Australians find a vaccine appointment first appeared on Jake Coppinger.