Skip to content
This repository has been archived by the owner on Nov 20, 2024. It is now read-only.

Commit

Permalink
docs: add extra info on importer/updater plans
Browse files Browse the repository at this point in the history
  • Loading branch information
spwoodcock committed Sep 3, 2024
1 parent fe1170e commit 1d28344
Show file tree
Hide file tree
Showing 2 changed files with 54 additions and 4 deletions.
46 changes: 44 additions & 2 deletions importer/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,47 @@
# OSM Importer Service

Import data into a fresh OSM Sanbox instance.
Import data into a fresh OSM Sandbox instance.

Uses Geofabrik country data, then filters down to the users required BBOX.
Method:
1. User provides BBOX to download data for.
- First we do a simple calculation to get centroid from BBOX.
- https://nominatim.org/release-docs/latest/api/Reverse
2. (Optional) reverse geocode the country name from BBOX area.
3. Download latest country data using GeoFabrik.
4. Filter data using `osmium` BBOX functionality.
5. Import the BBOX data into the sandbox db using `osmosis`.

> [!NOTE]
> While `osmium` is the most performant and best maintained tool
> for dealing with OSM data, it does not support importing into
> an OSM-type database (dbapi).
>
> It's primary purpose is for importing into an alternative
> PostGIS database for data analysis, using PostGIS representations
> of each geometry (the OSM db does not use PostGIS).
>
> As a result, the only available tool for importing into dbapi
> format is `osmosis`, a now deprecated Java tool.
## Work Modes

### Option 1: Startup

- Each osm-sandbox instance is throwaway.
- The user starts sandbox with a bbox, the data is populated.
- The mapping concludes, data is extracted, and the sandbox deleted.

### Option 2: Triggered

- We run one osm-sandbox instance.
- The user triggers import for an AOI.
- The data is imported using the workflow above.

## Updating Data

See the `updater` section of this repo.

## Future

- This is a test service to demo different approaches.
- The end goal is to contribute to developmentseed/osm-seed.
12 changes: 10 additions & 2 deletions updater/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,13 @@

Update the data in an existing OSM Sandbox instance with latest OSM data.

Uses the `.osc` daily diff files provided by OSM, and filters down to the
users required BBOX.
- After initial load, the user may want to update the data in sandbox.
- To sync with the current OSM database, we need to use replication data.
- This is made available on OSM at intervals: minute, hour, day.
- https://wiki.openstreetmap.org/wiki/Planet.osm/diffs
- E.g. https://planet.osm.org/replication/day/000/004/
- We download the daily `.osc` diff files provided by OSM.
- We must download each file since the date of first data import.
- The diffs can be filtered by BBOX using `osmium`.
- Then the actual data import / applying of data in the db
will likely be done by `osmosis`.

0 comments on commit 1d28344

Please sign in to comment.