Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle (or not) non-minute diffs #108

Open
oesteban-vx opened this issue Oct 31, 2023 · 1 comment
Open

Handle (or not) non-minute diffs #108

oesteban-vx opened this issue Oct 31, 2023 · 1 comment

Comments

@oesteban-vx
Copy link

Summary: hour, ... (anything but minute) diffs won't work because base clone and diff repositories don't supoort it.

I have created a container using OVERPASS_DIFF_URL=https://planet.openstreetmap.org/replication/hour/.

Debugging the issue, I've reached the file /app/bin/download_clone.sh (inside the docker image). The entry-point file calls http://dev.overpass-api.de/api_drolbr/trigger_clone and puts the result into this file

$ cat /db/db/base-url
https://dev.overpass-api.de/clone//2023-10-28

Let's ignore the duplicated /. This is the initial data set we'll download. It includes a replicate_id, which is the latest changeset the data contains; we'll get diffs from that number to update the dataset.
The problem is that the replicate_id corresponds to a minute diff, so we won't be able to find it in OVERPASS_DIFF_URL.

I guess those repositories (cloned dataset, diffs) are created by the source server, so there's nothing we can really do. Well, I wonder if we can get the timestamp of the minute diff, then look its timestamp in https://planet.openstreetmap.org/replication/minute/<replicate_id> and, once we have it, look for the corresponding hour replicate id. We'll probably get a diff not exactly for that timestamp, though.

At least, once the container has been created, write some message explaining that updates won't work (and don't even try).

@wiktorn
Copy link
Owner

wiktorn commented Nov 3, 2023

I think this is possibile, maybe using similar logic that pyosmium-get-changes uses, to match the dump to the replication source.

Feel free to create a PR with that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants