It is a “living document” and will change (and hopefully improve!) over time.

See embed of the page at https://betterintersections.jakecoppinger.com/analysis below. Please let me know if you spot any errors, bugs, or have suggestions on further charts!

The post Preliminary analysis of Better Intersections data first appeared on Jake Coppinger.

I built it to quickly compare and contrast Australian cities with their international counterparts. Clicking on any statistic opens an Overpass Turbo query displaying the relevant data.

I previously wrote a blog post on turn-by-turn bicycle navigation apps that use this data at The Best Apps for Bicycle Directions (2020).

Architecture

The website is a simple frontend React Typescript app, however the data is statically compiled into a large JSON blob.

Generating the JSON blob requires thousands of Overpass Turbo requests. These requests are cached at build time on the filesystem using a hash of the query string as a key. This currently requires clearing the cache to completely regenerate data to fetch new updates from OpenStreetMap. When generating data for Australian councils I use a self-hosted Overpass server (also improving speed dramatically), while using overpass-api.de for the few international examples.

I could achieve a faster first paint by async loading this JSON blob at runtime but I haven’t yet implemented this.

Population counts are sourced from Wikidata (and pregenerated in the JSON blob) based on the wikidata tag on OpenStreetMap relations.

When area names have a non-english name (identified by a name:en tag present), both English and local names are displayed.

Overpass queries

Overpass queries are written in https://github.com/jakecoppinger/australian-cycleway-stats/blob/main/static-backend/src/utils/overpass-queries.ts and are somewhat complex. They contain some “opinionated” tradeoffs (informed by policy) in what roads are considered safe (<= 30kmh) and what is considered a dedicated and shared cycle path. Improvements or questions are welcome!

The post Which Australian councils are building the most cycleways? first appeared on Jake Coppinger.

A street is defined as ‘contraflow’ or two-way for cyclists (and in some cases pedestrians) where people on bikes (or on foot) are legally able to travel in both directions on a street designated a one-way for motor vehicles. Suitable streets typically have low traffic volumes and low speeds, with one (or no lanes) marked, sufficient width and may already be designated as a shared zone (the Australian term for a living street).

In 2014, the state agency Transport for NSW issued a technical direction permitting contraflow bicycle travel on suitable streets if approved signs and markings are allowed. When provided on suitable roads, these simple street treatments create safe new routes for cyclists with an incredible value for taxpayer funds.

The TfNSW Technical Direction (TTD 2014/002) describes contraflow cycling facilities as “a cost-effective treatment that enhances the cycling experience by improving the permeability of neighbourhoods and by reducing bicycle trip lengths”.

This is by no means an argument against more substantial investment in separated cycleways. The Queensland Government found cycling infrastructure has a 5 to 1 return on investment, and the UK Government found with beneft-to-cost ratios in the in the range of 5:1 to 19:1 – some as high as 35.5 to 1 (that is, a return on investment of up to 3550%). These benefits include improved public health, reduced emissions, reduced traffic, and reduction in expensive taxpayer funded road maintenance (road wear is proportional to the 4th power of axle weight). We can’t afford not to invest in dedicated cycling infrastructure.

New contraflow cycling in the City of Sydney

The City of Sydney has recently approved 159 suitable streets across 24 suburbs, which will greatly improve the network of legal cycling routes in inner city Sydney. The implementation will proceed as soon as budget and works capacity allows.

This blog post is a proposal of additional streets which may be suitable for basic contraflow cycling infrastructure that the council could install in future, with a focus on the utility of OpenStreetMap for researching potential streets.

What makes a safe contraflow street?

Most streets can become contraflow streets with the right infrastructure (a separated cycleway), but this blog post focuses the extremely cost efficient cases where they are feasible with only a sign and/or painted markings.

The TfNSW technical direction specifies:

Contra-flow bicycle facilities should be assessed as a potential treatment on all local low speed, low volume one-way streets, including shared zones.

Ideally, all contra-flow bicycle movements will be delineated by a bicycle lane. A bicycle lane must be installed in locations where:

• Sight distances are restricted due to bends in the road or other features.
• Motor traffic volumes or speeds present a safety risk.
• Bicycle traffic volumes or speeds present a safety risk.
• The gradient and/or other road geometry increase the risk of collisions or unsafe driving or riding behaviours.
• The number or location of driveways present a safety risk.

Note: The NSW Road Rules prohibit parking in signposted bicycle lanes.

If the road space is too narrow to permit a marked bicycle lane and there is good sight distance, motor traffic volumes and speeds are low and the road geometry does not present an unacceptable safety risk, the contra- flow movement can be provided by signage alone.

TTD 2014/002 Signposting for contra-flow bicycle facilities

With this in mind, OpenStreetMap does not include traffic volumes, however by filtering out multi lane roads, roads with high speed limits and roads that are designated as important to the motorway network we can remove from our query most high volume roads and identify further potential candidates for contraflow cycling facilities.

Streets that currently allow contraflow cycling

Below is a map of current streets that permit contraflow cycling. All maps are generated just for you as you load this page – so it’s always up to date.

I make use of open source OpenStreetMap (OSM) data which is a rich data source of cycling and road infrastructure. OSM is collaborative – if you notice any errors you can edit the map, and they will show up here. The dataset is heavily used (and improved) by commercial entities including Facebook, Amazon, Microsoft, TomTom, Uber, Strava, Citymapper and government entities such as the Transport for NSW Trip Planner and the Victorian Department of Transport and Planning (who recently made the strategic decision to use OSM as their foundational mapping data source).

A technical note: these maps are using Overpass Ultra – a brilliant, open source, vector map powered OpenStreetMap query engine by Daniel Schep (on Mastodon), inspired by Overpass Turbo. Overpass Turbo shows dots for small details by default which are not possible to disable in a shared map.

You can click on highlighted streets to view more data, such as:

• name
• lane count
• road classification
• whether it’s a shared zone / living street

View/edit query

(note: some of the map embeds are currently broken, please follow the “View/edit query” links)

Previously approved but not yet built contraflow streets

Below is a map of streets where the contraflow infrastructure (signs and/or road markings) is approved but under construction, or not yet built (previously approved). I have mapped them on OSM with oneway:bicycle=construction.

View/edit query

Potential future contraflow streets

Below is an automated query of streets that are potential contraflow street candidates. As the data is automated there may be streets not suitable, such as circular one-way service roads. The intention of this queried data is to provide a helpful starting point, from which to filter for potential candidates, through use of the easily accessible, open source data. You can run the query yourself here and modify it if desired. See the wiki for the available OSM tags relevant to bicycles to filter by.

You can read the full query on which roads are shown in the appendix below.

View/edit query

A proposal of streets that may be suitable contraflow streets

These are a non-exhaustive manual selection of streets from the above query (in no particular order). Some I have taken width measurements using Lidar on an iPhone.

I’ve excluded lanes that appear to be very narrow and lanes where an alternative is very close (eg. William Ln & Corfu St).

There may be errors or other reasons they aren’t suitable – any feedback or corrections is welcome in the comments below!

Other notes

• Devonshire St would be very useful, but a paint & sign treatment likely not sufficient
• There are plenty of one way segments on the pedestrianised George St which would be perfect for cycling – the cycling situation on George St likely needs it’s own blog post

Appendix

More maps

Contraflow permitting streets globally

Potential future contraflow streets globally

All the Overpass queries!

Query: CSV of candidates of contraflow lanes

https://overpass-turbo.eu/s/1ytu

Note:

• See the OSM Wiki for what tags mean
• This is an automated query of open source data – there may be errors

Query: Current streets allowing bicycle contraflow

[out:json];
(
  // Relation 1251066 is COS boundary:
  // https://www.openstreetmap.org/relation/1251066
  rel(1251066);map_to_area->.region;
  way(area.region)["highway"]
  ["oneway:bicycle"="no"]->.ways;
);
.ways out geom;

Query: Streets that CoS will turn into contraflow

[out:json];
(
  // Relation 1251066 is COS boundary:
  // https://www.openstreetmap.org/relation/1251066
  rel(1251066);map_to_area->.region;
  way(area.region)["highway"]
  ["oneway:bicycle"="construction"]->.ways;
);
.ways out geom;

Query: Potential future contraflow streets

[out:json][timeout:25];
(
  // Relation 1251066 is COS boundary:
  // https://www.openstreetmap.org/relation/1251066
  rel(1251066);map_to_area->.region;
  
  // Select roads
  way(area.region)["highway"]
  
  // Only roads which are marked one way,
  // and don't allow bicycle contraflow
  ["oneway"="yes"]
  ["oneway:bicycle"!="no"]
  
  // Exclude already approved contraflow lanes
  ["oneway:bicycle"!="construction"]
  
  // Excluded roads under construction
  ["highway"!="construction"]
  
  // Excluded proposed roads
  ["highway"!="proposed"]

  // Exclude driveways
  ["service"!="driveway"]
  
  // Don't include roads that are bidirectional,
  // but are separated (and appear to be one way)
  ["dual_carriageway"!="yes"]
  
  // Don't include if a cycleway is already mapped as separate
  ["cycleway"!="separate"]
  ["cycleway:left"!="separate"]
  ["cycleway:right"!="separate"]
  
  // Don't include if a cycleway already present
  ["highway"!="cycleway"]
  
  // If a road is customers only it's likely
  // in a parking lot
  ["access"!="customers"]
  
  // Don't include roads where public access not allowed
  ["access"!="no"]
  
  // Don't include link roads (on ramps/slip roads)
  ["highway"!="motorway_link"]
  ["highway"!="primary_link"]
  ["highway"!="secondary_link"]
  ["highway"!="tertiary_link"]
  
  // Don't include major roads
  ["highway"!="primary"]
  ["highway"!="secondary"]
  
  
  ["lanes"!=2]
  ["lanes"!=3]
  ["lanes"!=4]
  ["lanes"!=5]
  ["lanes"!=6]
  
  ["access"!="private"]
  
  // Don't consider parking isles
  ["service"!="parking_aisle"]
  
  
  

  ->.ways;
);

.ways out geom;

Query: All streets with bicycle contraflow allowed, including outside City of Sydney council

[out:json][timeout:25];
(
  way["highway"]
 
  ["oneway:bicycle"="no"]
  
  ({{bbox}});
);
out body;
>;
out skel qt;

Other useful data

Ethan’s Sydney Bike Map

One of the best OpenStreetMap powered maps of cycling infrastrure, including proposed and under construction paths: https://sydneybikemap.ethan.link/

Possible further work

• Categorising sightlines of road segments computationally to narrow down the list of possible candidates
• Creating an Overpass Turbo query of manual candidates

The post Contraflow streets in the City of Sydney first appeared on Jake Coppinger.

See comments below, on Mastodon, Hacker News (51 comments), or LinkedIn (22 comments, 278 likes, 32k views)

Transport for NSW, the government agency which controls traffic signal timing in Sydney and elsewhere in NSW, has an excellent stated goal of increasing walking and cycling trips – and reducing pedestrian wait times at intersections.

However, there is no public data on traffic light timing in Sydney or NSW.

In the absence of traffic light timing data, and as we hold hope for it to become publicly available; the aim of Better Intersections is to crowdsource measurements and inform where positive changes could be made. You can add data yourself via a simple Google Form, and instructions are on the website about page.

Table of Contents

Better Intersections is a tool to record and visualise timing details for pedestrian and bicycle signals. It’s focused on Sydney & NSW, Australia, but is adaptable for anywhere in the world. This website is open source on Github (contributions welcome!), and the data is under an open license (ODbL license).

If you have ideas for improvements, please create a Github issue, comment below, email me at jake@jakecoppinger.com or message me on Mastodon (@jakecoppinger@aus.social).

It’s a work in progress! I’ve tinkered on it for a few afternoons and started working on it about two weeks ago.

This website bridges the excellent TfNSW Active Transport policy guidelines and pedestrians on the street themselves, allowing people on foot (and bicycle) to see their experience represented.

Why does the timing of pedestrian signals matter?

Transport for NSW, the government agency which controls traffic signal timing in Sydney and elsewhere in NSW, has an excellent stated goal of increasing walking and cycling trips – and reducing pedestrian wait times at intersections.

However, there is no public data on traffic light timing in Sydney or NSW.

In the absence of traffic light timing data, and as we hold hope for it to become publicly available; the aim of this project is to crowdsource measurements and inform where positive changes could be made.

This website bridges the excellent TfNSW Active Transport policy guidelines and pedestrians on the street themselves, allowing people on foot (and bicycle) to see their experience represented.

Increasing pedestrian priority and providing crossing opportunities at the right locations and along desire lines, reduces the risk of pedestrian injury at intersections by encouraging safer behaviours. Transport is currently rolling out measures at intersections to improve pedestrian priority in areas of high pedestrian activity. These measures may include automation of pedestrian crossings, reduced pedestrian wait times, provision of pedestrian crossings on missing legs and kerb ramps, where applicable.

— TfNSW Active Transport Strategy, page 30. Emphasis added.

Research has shown that 30 seconds is the longest a pedestrian will wait at a signalised crossings before attempting to cross against the ‘red man’. (Martin, A., 2006. Factors influencing pedestrian safety: a literature review (No. PPR241). Wokingham, Berks: TRL (Transport for London.)

From the above report:

Hunt, Lyons and Parker (2000) state that ‘Although no clear relationship has been established between pedestrian delay and casualties, a more balanced and responsive approach to the allocation of time at Pelican/Puffin crossings has the potential to make a substantial contribution to a decrease in pedestrian casualties as well as improving pedestrian amenity’. They point out that because pedestrians are more likely to become impatient when a red man continues to be shown during periods of low vehicle flow, the reduction of unnecessary delay for pedestrians should encourage pedestrians to use crossings correctly and reduce risk taking.

In 2020, people driving vehicles killed 138 pedestrians on Australian roads (Department of Infrastructure, Transport, Regional Development and Communications (2021) Fact sheet: Vulnerable road users, National Road Safety Strategy.)

But isn’t traffic light timing variable?

Sydney uses a system called Sydney Coordinated Adaptive Traffic System (SCATS) to control traffic signals, which makes use of many data feeds to control timing data.

Neither the inputs used, or the algorithm used to weigh the input data is public (as far as I know). This crowdsourced method of discrete measurements provides shows the output of the black box. In the case of outliers, multiple measurements (at different times of day/week) can be used to determine if the timing is variable.

Gehl Architects have a great methodology for measuring the overall impact of traffic light delays on pedestrians. At its most basic you walk along a street with two stopwatches;

• one you pause only when you’re walking;
• one you pause only when you’re waiting at a traffic light.

Divide one by the other at the end, and you have a single number that quantifies pedestrian delay walking along a street.

– Public spaces & public life: Sydney 2020, Gehl Architects, pg 142

The limitations of this method are that

• it cannot inform the exact problematic intersections;
• paths must be long enough to gather a large enough sample size to be statistically significant.

I’ve experimented with automating this method by recording a GPX (GPS) trace with a phone, uploading that file and getting a number instantly. Unfortunately the urban canyon effect (GPS signal loss caused by tall buildings) makes this method unreliable in cities, even with the remarkable sensor fusion on modern phones.

As with any of my projects, I am always open to collaboration. If you have any ideas, iterations or improvements, please drop me a line!

Technical details

This is a fairly simple Typescript app created using Create React App that I built in a few afternoons – please don’t consider it my finest code!

It is a static React app hosted on Cloudflare pages, and uses Mapbox GL JS to display the map (but could be easily updated to use Maplibre GL JS).

Google Sheets is treated as a backend (for simplicity using a Google Form for submissions), and the app makes use of OpenStreetMap node IDs as primary keys for intersections. This is definitely suboptimal but it’s quick to build – ideally I’d have a more custom form that is easier to use and doesn’t require a Google account – however using Google accounts for the form is a quick and easy method of minimising spam (and making it easy to identify) remove spam from a single person).

The OpenStreetMap API is used for looking up coordinates of OSM nodes and finding adjacent ways. There is currently one request per intersection made – this will not scale and I’ll likely need to cache the JSON (or hit an Overpass Turbo server instead).

The code is fully covered by Typescript types but doesn’t (yet?) have unit tests – it’s very easy to work with if you’re interested in tinkering with it!

Possible further work

• Adding a simplified version of the form
• Support OSM ways as the primary key rather than just nodes (useful for crossings across divided carriageways with multiple traffic light nodes per crossing leg)
• Tagging state roads and looking if these have longer wait times on average (probably)
• Scatter plot of crossing times vs number of lanes (as rough proxy for traffic volume)
• “sparkline” or other graphs of measurements for a given intersection
• Thinking about how to record relationships between intersections (ie. green wave/lack of green wave for pedestrians)
  • Using YOLOv3 or another off the shelf commodity computer vision model for recognising green/flashing red/red traffic lights for algorithm measuring cycle times

Related organisations

If you support better conditions for pedestrians and cyclists in Sydney/NSW, consider joining BetterStreets or 30 Please.

The post Mapping pedestrian traffic light timing in Sydney, Australia first appeared on Jake Coppinger.

Why is this useful?

Usually for such a task you would use a drone and process with WebODM (or Pix4D), but there are areas that are unsafe or illegal to fly in. I’ve previously detailed how to generate imagery using a bicycle helmet mounted GoPro camera, however this can include artifacts where there are lots of people. The helmet camera method requires a decent GPS lock (unsuitable indoors, urban areas or under a bridge) and has relatively low detail.

Again, why might you want to do this? With your own high detail and up-to-date models and street imagery you could:

• Map new street interventions, like bollards, modal filters or raised crossings
• Record pothole locations (and their depth!)
• Take measurements such as road and cycleway widths around crowds of people in urban centres
• Measure footpath obstructions in 3D and rate pedestrian amenity
• Survey features underneath large highways
• Survey street parking using the new OSM spec: wiki.openstreetmap.org/wiki/Street_parking
• Map indoor pedestrian areas in OpenStreetMap for better pedestrian routing
  • The Transport for NSW Connected Journeys Data team is currently doing a fair bit of this work: https://www.openstreetmap.org/changeset/133107592
• Attach your iPhone to your bike and generate LiDAR point clouds of the kerb and cycleway infrastructure (it works, just go slow!)

This method results in very high detail (5mm resolution if desired) 3D models and accurate orthoimagery. Manual georeferencing is required (which I also explain how to do) which limits the confidence in alignment. This is a proof of concept – if you have corrections/suggestions/ideas to improve the method, please comment below or on Mastodon!

Note: This method also provides a solution to creating 2.5D oblique orthophotos from drone imagery.

Process overview

This guide covers how to:

• Capture a 3D model using 3d Scanner App (recommended) or Polycam
  • The iPhone LiDAR sensor has 5 metres max range, so you’ll need to walk around
• Export the model to an .obj file with textures
• Rotating the model in Blender to the required orientation
• Use the odm_orthophoto program inside the OpenDroneMap Docker container to generate a raster .tiff
• Georeference the tiff using QGIS
• Uploading the Geotiff to OpenAerialMap to generate a tileset, viewable in the OpenStreetMap iD editor or a Felt map with a custom layer

Capturing the model

Capturing a 3D model on an supported iPhone is easy. I recommend using the app titled 3d Scanner App as it allows considerable customisation of the scan settings. It allows finishing a scan and extending later, though this can be buggy. I haven’t had a crash during capture – I’ve had Polycam crash halfway through a large scan losing all data.

Download 3d Scanner App and use the LiDAR Advanced mode. I recommend the following options for scanning streets:

• Confidence to low. This extends the range of the LiDAR sensor readings used at the expense of more noise. You can clean up this noise in the processing settings or Blender.
• Range to 5.0 metres
• Masking to None
• Resolution to 50mm (the lowest – for large models like streets)

In the app settings, make sure to set:

• GPS tag scans to ON
• Units to metric

When scanning a street, walk (or cycle) slowly with a sweeping motion to increase the width. If the area is wide enough to require a grid pattern, follow the same shape as a drone survey (an S-shape with considerable overlap). Not enough overlap or higher speeds mean the linear passes don’t connect correctly due to (I assume) inertial measurement unit drift. I’m unsure if the GPS information is used in the sensor fusion (via ARKit), please comment if you know!

Exporting and preparing the model

In the 3d Scanner App use the Share button, then select the .obj file type. Send this to your computer (Airdrop works great if using macOS). If using Polycam, set “Z axis up” in the mesh export settings.

Rotating the model into the correct orientation (required for 3d Scanner App)

Unfortunately the 3d Scanner App exports objects with the Z axis as “up”, while the odm_orthophoto program expects the Y axis to be “up”. Confusingly, you can skip this step if using Polycam if exporting with “Z axis up” in the mesh export settings, though Blender shows the Y axis as up in this export. If you know why this is, please leave a comment!

To rotate the model, import it to Blender and rotate it 90 degrees.

• First, install Blender via your preferred method (https://www.blender.org/download/).
• Open Blender, delete the initial default cube (right click -> delete, or x hotkey)
• Import the .obj file: File -> Import -> Wavefront (.obj)
• (optional: you can view the pretty texture by selecting “viewport shading” in the top right (the horizontal list of sphere icons))

• To rotate
  • Click the object and make sure it is selected (orange border)
  • Press hotkey r (from any view)
  • Press x to only allow rotation on X axis
  • Type 90 (or desired degrees to rotate)
• Optional: You can check if the rotation is correct by pressing numpad key 1. If you don’t have a numpad you will need to enable numpad emulation (see instructions at https://www.hack-computer.com/post/how-to-emulate-a-third-mouse-button-and-keypad-for-blender).
  • The rotation is correct if you have a “birds eye view” in the numpad key 1 view, where the blue Z axis is towards the top of screen and the red X axis is towards the right of screen

• File -> Export as an .obj to the same folder with a new name (eg. blender_export.obj)
  • Note: Blender doesn’t create a new texture .jpg. If you export to a different folder the path to the .jpg in the .mtl file will need updating.

Generating the raster orthophoto

Use the odm_orthophoto command line tool to generate a raster orthophoto from a .obj file. This tool is available at https://github.com/OpenDroneMap/odm_orthophoto but has a considerable number of dependencies.

I believe the easiest method currently is to install WebODM locally, copy the .obj and texture files (.mtl and .jpg) into the Docker container and then run the program from inside the Docker container.

Installing WebODM locally

Running the software using Docker is a breeze. Install Docker from https://www.docker.com/ (or your preferred method) and then:

git clone https://github.com/OpenDroneMap/WebODM --config core.autocrlf=input --depth 1
cd WebODM
./webodm.sh start 

See https://github.com/OpenDroneMap/WebODM#getting-started for more details. WebODM itself is excellent and great fun if you have a drone!

Copying the object into the ODM Docker container

You can start a shell in the container with the following command:

docker exec -it webodm_node-odm_1 /bin/bash

Make a new directory to keep your files in

mkdir /iphone_model
cd /iphone_model

In another shell, copy the object and texture files from your local machine into the new Docker container folder. docker cp can only copy one file at a time.

cd path/to/your/model/
docker cp blender_export.obj webodm_node-odm_1:/iphone_model/
docker cp blender_export.mtl webodm_node-odm_1:/iphone_model/
# Note: The blender .obj export doesn't create a new texture .jpg
#   If your Blender export wasn't in the same directory, check
#   update the path in blender_export.mtl
docker cp textured_output.jpg webodm_node-odm_1:/iphone_model/

Running odm_orthophoto

In the shell you started in the docker container above, run the following command:

cd /iphone_model/
/code/SuperBuild/install/bin/odm_orthophoto -inputFiles blender_export.obj -logFile log.txt -outputFile orthophoto.tif -resolution 100.0 -outputCornerFile corners.txt

The resolution argument is how many pixels per metre – this may require changing.

Exporting the orthophoto out of the Docker container

To copy the generated orthophoto out, from a shell on your local machine run:

docker cp webodm_node-odm_1:/iphone_model/orthophoto.tif .

Use a similar command to extract the log file if required.

Georeferencing the orthophoto

Georeferencing is the process of specifying the location and orientation of the image so it perfectly aligns with maps or GIS software. While a rough location (with a moderately incorrect rotation) is stored in the model, it appears to be removed by the Blender rotation step. If you know how to fix this please comment below!

To do this:

• Install QGIS by your preferred method: https://www.qgis.org/en/site/forusers/download.html
• Install the plugins (via the Plugins -> Manage & Install plugins… menu)
  • QuickMapServices (to pull in Bing satellite imagery easily)
  • Freehand raster georeferencer (a beginner friendly georeferencing tool)
• Add a Bing satellite base layer: Web -> QuickMapServices -> Bing -> Bing Satellite
  • Feel free to choose another satellite background of your chosing
  • If you’re in NSW: the NSW LPI Imagery is likely the most detailed, follow: https://www.spatial.nsw.gov.au/products_and_services/web_services/qgis
• Zoom & pan to the rough location of the 3d scan (the initial .tif location will be wherever you’re viewing)
• Drag the .tif output by the previous step into the sidebar (it won’t be visible yet as it is not aligned)
• Go to Raster-> Freehand raster georeferencer -> Add raster for freehand georeferencing and select the same .tif
• Use the Move, Rotate and scale buttons in the toolbar to align your orthophoto with the imagery background (tip. Hold Cmd or Ctrl before scaling to keep the aspect ratio)

• Click the “Export raster with world file” button (Green on the right with exclamation marks).
• Check the “Only export world file for chosen raster” button. Make sure to do this before chosing the image path.
• Select the existing .tif image and press OK
• Remove the orthophoto from the QGIS sidebar (right click -> remove layer)
• Drag the existing .tif image back into the sidebar. QGIS will now find the worldfiles next to it (orthophoto.tif.aux.xml and orthophoto.tfw) so it will be positioned in the right place

Export geo-referenced GeoTIFF (without worldfile)

If you would like to upload the GeoTIFF to OpenAerialMap or somewhere else, you will need to “bake in” the location into the GeoTIFF itself, rather than in the worldfile – OpenAerialMap can’t read the worldfile.

To do this:

• right click your orthophoto layer (after the above steps) and click Export -> Save As…
• Set CRS to your desired coordinate system (if not yet in a coordinate system, I assume you should use EPSG 3857 if you want it to be aligned with OpenStreetMap tiles, but this is the limit of my current understanding – I haven’t studied surveying yet!).
• To avoid confusion, create a new subfolder and save it with the default settings (eg. make folder qgis_export and save as orthophoto.tif).
• You now have a nice georeferenced GeoTIFF!

Uploading to OpenAerialMap

If you want the imagery to be publicly viewable and accessible from the OpenStreetMap iD editor, OpenAerialMap is a free place to host your imagery.

This is the imagery from the above example: https://map.openaerialmap.org/#/-18.6328125,18.562947442888312,3/latest/

I’ve heard of plans for a relaunch of the website, but currently the upload form can be finicky.

• Open the explore page: https://map.openaerialmap.org/
• Sign in (only Google & FB Oauth supported)
• Press upload
  • Currently uploading from local file doesn’t appear to work, see https://github.com/hotosm/OpenAerialMap/issues/158 for updates
  • Uploading via Google Drive with my account (2fa enabled, Gsuite) fails with This app is blocked: This app tried to access sensitive info in your Google Account. To keep your account safe, Google blocked this access.
    • Enabling less secure apps is not possible for 2fa accounts. Otherwise, if you’re comfortable turning it off you can do that here: https://myaccount.google.com/lesssecureapps
  • Using a URL is likely the only way. Creating an S3 bucket is one way. If you have a fast connection it would be faster to run a local webserver with Python and running ngrok to make it publicly available. I recommend not keeping this server running for longer than necessary. Eg:

cd qgis_export
python3 -m http.server 8080
ngrok http 8080
# Your file is now available at https://SOME_PATH.ngrok.io/orthophoto.tif

Specify this url in the form and add other details, then press upload.

Limitations

• Manual alignment limits the real world accuracy of imagery
• Drift during long model captures occurs. My understanding is drift occurs more when there are sudden or fast movements. The 3d Scanner App unfortunately doesn’t warn you when you’re moving to fast, but Polycam does. As far as I know, the iOS ARKit doesn’t attempt to reconcile drift when completing a loop/circuit.

Future work

• Automation! This process is slow but it works.
  • Adding a Makefile or other compile tooling to https://github.com/OpenDroneMap/odm_orthophoto would skip the requirement to install WebODM and transfer files to/from the Docker container
  • Rotating the model could be added (behind a flag to be backwards compatible) to the odm_orthophoto script
• Generating pointclouds (supported by 3d Scanner App) and then exporting as a raster from CloudCompare. This might make larger captures possible.
  • If there is a way of addressing drift of pointclouds for multiple captures – let me know how!
• Georeferencing using ground control points rather than a freehand referencer
• Creating street facade montages and evaluating doors & soft edges (Jan Gehl (1986) “Soft edges” in residential streets, Scandinavian Housing and Planning Research,3:2,89-102, DOI: 10.1080/02815738608730092)

Let me know if you have any corrections/suggestions/feedback!

The post Generating aerial imagery with your iPhone’s LiDAR sensor first appeared on Jake Coppinger.

This technical guide details how you can create your own orthorectified (aka satellite view/bird mode) imagery, point clouds and 3D models of streets with nothing but a 360 degree camera mounted on bicycle helmet, and the open source photogrammetry software OpenDroneMap.

Why might you want to do this? With your own up-to-date and highly detailed orthorectified imagery you could:

• quantify and communicate inefficient road space allocation
• record necessary infrastructure repairs
• take measurements such as lane and cycleway widths
• measure footpath obstructions in 3D and rate pedestrian amenity
• map kerb features on OpenStreetMap
• survey street parking using the new OSM spec: wiki.openstreetmap.org/wiki/Street_parking
• 3D print a model of your home street!

Drones are great for surveying and mapping things on the kerb (parking/public space/road widths/building shadows) but there are a lot of places you can’t fly a drone.

OpenDroneMap is usually used to combine a set of geotagged drone images into one coherent 3d model, but it can also be used with any geotagged images.

Roughly the process is as follows:

• Take spherical images on a 360 degree camera
• Import the files from the camera
• Optional: Convert the photos into rectangular, spherical images (for GoPro: convert from the proprietary GoPro format to standard spherical images using GoPro Fusion v1.2)
• Start/login to WebODM, upload the images with fisheye lens setting for 180 degree images or spherical for rectangular 360 degree images and generate imagery & model

Please comment any projects you make after reading this guide, or if you have any questions! There are certainly issues with this process with I’ve added under Limitations.

Table of Contents

Buying a 360 degree camera

360 degree cameras can be very expensive (the ones mounted on Google Street View cars are possibly $45,000!), but GoPro makes relatively affordable models. See https://help.mapillary.com/hc/en-us/articles/115001465989-About-360-cameras for more suitable cameras. One common second hand (and now unsupported) model is the GoPro Fusion, which I’ll base this guide on.

Note: Seth Deegan has written a PowerShell script for using Insta360 cameras with ODM. I haven’t tried it but it seems worth a look: https://github.com/lectrician1/Insta360-2-ODM

To generate a detailed model you will need to capture enough images close together. You could mount it to the top of your car, but unless you bought an expensive elevated mount much of the image would just be your car roof!

Some advantages of taking imagery from a bicycle:

• The unobstructed field of view is larger so ODM gets more data for a better model
• The average speed is lower permitting more photos (the maximum self timer frequency of this camera is 2 shots/second)
• You can capture images from different places in the street for more data, such as the road, footpath and bike lanes
• If you’re putting in this much effort to study public space and urban planning you probably like bikes

You’ll need to buy a helmet mount for the GoPro or your chosen camera. GoPro makes a bicycle helmet mount: https://gopro.com/en/us/shop/mounts-accessories /vented-helmet-strap-mount/GVHS30.html (though many non-branded mounts exist on eBay/Amazon too). Be aware a camera mounted on a bicycle helmet is possibly a safety risk to yourself if you crash – be careful.

If you have the ability to mount the camera on a long pole above your head, this will increase the unobstructed field of view and improve the perspective of the street which will improve the results. I imagine this would attract even more attention! If you have tips for building rigs like this please leave a comment. Andrew Harvey wrote an OpenStreetMap diary entry on his setup here: https://www.openstreetmap.org/user/aharvey/diary/42139

GoPro Fusion specific tips

Set the camera to:

• Timelapse photo mode (icon of camera & timer circle)
• Image frequency to 0.5 seconds
• Enable GPS geotagging

Mount the camera to the helmet, wait until the GPS location icon turns solid, and start capturing! Sometimes the first few shots don’t have any location in the EXIF data but this doesn’t seem to confuse WebODM, you can check this with identify -verbose image or your metadata viewer of choice.

Preprocessing the images

The GoPro Fusion will output two 180 degree “fisheye” images to two separate SD cards. Copy all these images to a folder onto your computer.

For an alternate method, see instructions under the generating equirectangular image heading.

Unfortunately, only the “front” images have geotagged information, so you’ll need to copy the exif data from each “back” image to each similarly named “front” image.

A quick and dirty way of doing this is generating a list of terminal commands, that copies all exif tags from each “front” to each “back” photo:

ls -l | grep "GF" | sed -E "s/^.* GF(.*).JPG*/exiftool −overwrite_original_in_place -gps:all -tagsFromFile GF\1.JPG GB\1.JPG/g" > script.sh

I’m sure there are better and easier ways of doing this, please let me know if you come up with one.

Generating the model and orthorectified imagery with Web OpenDroneMap

WebODM (https://opendronemap.org) is an absolute marvel of open source engineering. Unfortunately, generating 3D models takes some serious computing horsepower. You can either use the paid cloud version (WebODM Lightening) at https://webodm.net/ (fast) or run the software on your own computer (a few hours/overnight/days depending on the number of images).

You can directly upload images to WebODM Lightening to process, however you don’t get some friendly/useful features like a browsable map and 3d model viewer. I recommend setting up WebODM locally and adding WebODM Lightening as a “processing node”, so you get the power of the cloud and the extra features of WebODM.

Setting up WebODM locally

Running the software using Docker is a breeze. Install Docker from https://www.docker.com/ (or your preferred method), allocate as much memory & CPUs as you can, and then:

git clone https://github.com/OpenDroneMap/WebODM --config core.autocrlf=input --depth 1
cd WebODM
./webodm.sh start 

See https://github.com/OpenDroneMap/WebODM#getting-started more more details including GPU acceleration.

You’ll now be able to open WebODM on http://localhost:8000

Optional: Adding WebODM lightening as a processing node

Click lightening network in your WebODM sidebar and login.

Generating the model & selecting the correct options

Add a new project, click “Select images & GCP”, select your images, then you will see options for processing your imagery. If you’re using WebODM Lightening, make sure to set the Processing Node appropriately.

Some options I’ve found are required:

If you are using the raw 180 degree “fisheye” images straight out of the GoPro Fusion, set camera-lens to fisheye (this is critical). If you are using equirectangular images set to spherical. WebODM seems to be unable to auto-identify the lens type in both cases.

• enable sky removal – this uses AI to mask out the sky in each frame so that there are less artifacts in the final model
• enable auto-boundary
• enable bg-removal

This is just from my trial and error, there may be better option configurations.

You will likely need to enable “resize images” to 2000px or it is very easy to run out of memory.

Though it is out of scope for this article, you can also set up a VPS instance to speed up the process if you don’t want to use the hosted cloud processing tool. Ive tried this, but it’s probably more effort than it’s worth unless you’re making a business out of this.

Even with an EC2 instance with 124GB of RAM and 324 images at 5760 × 2180 pixels I ran out of memory – remember to enable some swap space if you want to run at full size: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04

Output accuracy: taking measurements

OpenDroneMap can make measurements of the 3D model. I’ve found these are usually quite accurate; they may have small “measurement” errors due to artifacts or distortion but will tend towards the correct value – GPS coordinates ensure the scale is correct.

For example, measuring the width of the green paint of the cycle path:

Large path:

• OpenDroneMap: 4.54 metres
• Tape measure: 4.56 metres

Small path:

• OpenDroneMap: 1.89 metres (though this varies slightly due to distortions)
• Tape measure: 1.89 metres

Alternative: Generating and processing equirectangular spherical images

I’m currently unclear whether this method is faster or produces better results than the raw 180 degree images. I’m very interested to hear if you have experience (and I’ll update this if I get more evidence). I believe the GroPro Max generates these images in camera, so this won’t be required.

If you use the GoPro Fusion, unfortunately because this camera is no longer supported GoPro discontinued the software: https://gopro.com/en/au/news/fusion-end-of-life

You’ll (unfortunately) need to use the proprietary GoPro Fusion software to generate spherical images. I’ve found on an M1 Mac (Monterey) the only version that still runs is 1.2, 1.4 just crashes. The least dodgy download I can find is https://macdownload.informer.com/fusion-studio/1.2/.

If you make an open source solution (maybe building from https://stackoverflow.com/questions/37796911/is-there-a-fisheye-or-dual-fisheye-to-equirectangular-filter-for-ffmpeg) please let me know! Edit: Looks like this repo will do it. https://github.com/trek-view/fusion2sphere

This camera has two SD card slots – one for the front facing camera, and one for the back facing camera. Sometimes only one camera works for part of the shoot and the proprietary software refuses to stitch any of the photos at all! The only way I’ve found to combat this is to format the card in camera before each shoot. Spending more on a GoPro Max may solve a lot of headaches!

Limitations

Little helmets appear! I’ve tried some techniques but haven’t found a proper solution that doesn’t degrade the model/orthophoto structure.

Appendix: Things that didn’t work

Removing helmet artifacts by cropping

I tried using the convert and mogrify tools (part of the amazing open source ImageMagick suite) can crop the spherical photos and retain the geographical information! The cropping works great, but OpenDroneMap doesn’t seem to be able to understand a cropped equirectangular image. If you’d like to try to get this working, here are the steps I took.

First install ImageMagick (sudo apt-get install imagemagick on Linux/brew install imagemagick on macOS).

The general syntax is:
convert -crop {x_size}x{y_size}+{x_offset}+{y_offset} inputfile outputfile

See https://deparkes.co.uk/2015/04/30/batch-crop-images-with-imagemagick/ for more detail and helpful diagrams.

Helpfully, the helmet artifact is always at the bottom of frame, so the x offset and y offset will be zero.

To crop 700 pixels off the bottom of a spherical image, you can use

convert -crop 5760x"$((2880-700))"+0+0 inputfile outputfile

To batch convert a number of files, where the input files are in ./input/, and you have made an output directory ./output/, you can use:

mogrify -monitor -crop 5760x"$((2880-700))"+0+0 -path ./output ./input/*

Removing helmet artifacts by adding a mask

Adding custom drawn masks, either covering just the helmet or also the surrounding black “void” either:

• removed the helments but caused extra “holes”/missing segments in the ground
• didn’t remove the helmets

I think this may be due to clashing behaviour of the bg-removal and sky-removal flags with manual masks.

ODM requires a mask image for every image with _mask.{EXT} as a suffix (where EXT replaces the original extension).

The two mask types I tried:

If you know what may be happening please let me know!

Further research/experimentation

• How many “capture lines” per street are necessary to make a decent model? I didn’t have any luck with one but it may have been my settings. I used ~4 for the above model.
• How much better are models when the camera is on a stick?
• Improved WebODM settings for generation

Prior art

• “A Satellite in Your Pocket: Ground Based Action Cameras to Create Aerial Perspective for OSM Editing”. This was recorded at OpenStreetMap US: Connect 2020 by Sean Gorman. The company Pixel8 doesn’t appear to exist any more.
  https://www.youtube.com/watch?v=tfab-iuWlsQ
• Twitter thread by @klaskarlsson (@klaskarlsson@fosstodon.org on Mastodon) on creating a model of his house using a “GoPro on a stick”: https://twitter.com/klaskarlsson/status/1583401741386936320
• “Create aerial imagery base on 360° pictures” – discussion in the OpenDroneMap community: https://community.opendronemap.org/t/create-aerial-imagery-base-on-360-pictures/12339

The post Creating aerial imagery with a bike helmet camera (GoPro) and OpenDroneMap first appeared on Jake Coppinger.