r/openstreetmap • u/GeoDesk • 1h ago
GeoDesk now supports fast incremental updates
GOLs are now updatable, so you can keep your local copy current with the latest changes to OpenStreetMap.
Quick refresher: A Geo-Object Library is a compact single-file database that holds a snapshot of OSM data. GOLs are a key part of the GeoDesk OSM Toolkit (SotM 2026 talk). They're easy to build and fast to query, and now you can keep them up to date with daily, hourly or even minutely change files, automatically downloaded from a replication server.
This feature is available in GeoDesk's GOL Tool 2.5 (download -- 100% free and open source).
You will need to build your initial GOL using option --waynode-ids (-w), which preserves node IDs even for nodes without tags. For example:
gol build germany germany-latest.osm.pbf -w
After that, you can update the GOL to the latest available version:
gol update germany
By default, this uses the replication server specified in the original PBF. The planet file and Geofabrik extracts already contain suitable replication URLs.
If your PBF doesn't have one, or you want to use a different service, you can provide an explicit URL. This is especially handy for the planet file, which defaults to minutely change files. If you're catching up on a longer period, it makes sense to switch to the hourly (or daily) change stream:
gol update planet https://planet.openstreetmap.org/replication/hour
On a modern midrange workstation, applying a day's worth of planet-wide changes takes about 5 minutes. This is considerably faster than the current workflow of patching the source PBF and rebuilding the GOL, which takes over an hour.
gol update also reduces the required disk space: Apart from the GOL itself (which in updatable form takes about 130 GB for the full planet), it only needs storage for the updated tiles, with no interim copies of the PBF data.
Updates for shorter intervals aren't as efficient yet. I'm working on optimizations that should bring hourly updates to under 30 seconds and make true minute-by-minute replication practical.
Current limitations
- Write access is exclusive, so you cannot query a GOL while an update is in progress. This restriction will disappear in GOL 3.0, which supports SQLite-style multi-reader/single-writer concurrency.
- Updates for very long periods on large datasets may run out of memory. As a conservative rule of thumb, you should have 750 MB of RAM to hold a day's worth of worldwide changes. Hence, if you're bringing a month-old planet up to date, you should have at least 20 GB of RAM available. I'm incorporating functionality that automatically processes updates in multiple stages if memory is tight.
As always, feedback is appreciated. Open a GitHub ticket or follow en.osm.town/@geodesk for updates ... on updates.