How can I fix the incomplete geometries of ways and relations on a local OSM server?

This problem has plagued me for a while now but hasn't caused any major issues, until a few days ago when the USA relation (148838) and the Canada relation (1428125) broke for me. My server does daily updates, and I was hoping that would fix it, but it hasn't yet. In fact, I assume that the daily updates are the cause of the problem in the first place. There's nothing else changing the database so unless some extraneous other event happened, that would be the only place for things to go wrong.

Looking at the relations in overpass turbo it says "Attention: incomplete geometry (e.g. some nodes missing)" for both of them. This also causes any area filtering using that relation's area to be completely wrong as well as not giving nearly enough results.

I've had this same problem for a while now but have been able to find workarounds to get the query results that I want, but the USA relation and Canada relation breaking is something that I cannot work around.

Any help at all would be greatly appreciated!

Edit: Found more of the specifics. The two nodes that don't exist in my local sever that are causing the problem are 8856238004 and 8856295638, both of which are part of the relation for USA and Canada. They are also part of way 957074345, which does not exist at all on my server. The way itself and all of the nodes that are a part of it just don't exist for me.

Edit 2: In the bin/nohup.out file of my server it has a lot of lines that say something like

compute_geometry: Node 8856295638 used in way 571733868 not found.
Where the actual node and way numbers vary, but all of the node values are in a range between 8853109156 ~ 8861281943 (not exact range). There's similar statements for missing ways for relations.

It has these statements once a day, I assume, when it updates because the time stamps right before them are the same range as the update times.

asked 12 Aug, 15:14

nbreden's gravatar image

nbreden
2649
accept rate: 0%

edited 12 Aug, 16:20


You write "a local OSM server" which I take to mean "a local Overpass server". I can't debug what the problem is exactly; I wonder if maybe you missed an update at some point and therefore are missing ways or so. If you can afford to, switching to regular full imports will of course get rid of the problem. To fix the issue at hand, you could also cheat by:

  1. identifying the IDs of all missing objects
  2. using osmium getid to extract the objects with these IDs from a planet file (or North America file or so), be sure to save them in the "osc" format
  3. feeding the resulting .osc file(s) into the update process of Overpass
permanent link

answered 12 Aug, 16:34

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
77.6k886901200
accept rate: 24%

Thanks for the response! Yes, I do mean "a local Overpass server", sorry for any confusion.

Could you expand more on what you mean by "switching to regular full imports"? Do you mean to do this regularly instead of what I'm doing currently which is from these steps?

(12 Aug, 16:42) nbreden

Yes, that's what I mean, of course you would not load a full planet but just the area of interest. If you have enough disk space, it is also possible to run the import into a secondary database and keep the "production" database running, and then switch over once the import is complete. This would give you only a minimal service interruption.

(12 Aug, 17:03) Frederik Ramm ♦

Hmm, I'm a bit confused. So with the database that I've setup following these instructions, I can take an extract from here and apply it to the database? I can't find how to apply a .pbf to the database, or if I convert it to a different format using something like osmconvert, I still can't find anything to apply a different format to the database... I feel like .osc is what I want but I'm not sure. Sorry if I'm missing something obvious.

(13 Aug, 12:43) nbreden
1

We might be confusing the concepts of "database engine" and "database content" here. Once you've set up the Overpass database engine, you have to load a data set - either the planet-wide data, or a regional excerpt. If you choose the "cloning from the dev server" option you always get a world-wide database. If you choose the "populate the overpass database from a planet file" path you have a choice of loading the full planet, or an extract. This needs to be supplied to the update_database script in plain XML form on stdin, so if you download a .osm.pbf file you will want to do something like

 osmium cat -f osm somefile.osm.pbf | update_database --db-dir=somedir --meta

(you will note that the init_osm3s.sh script mentioned in the install instructions does something similar - if you have a .osm.bz2 file and not an .osm.pbf file then you do not need osmium and can use "bzcat" instead).

Loading a file this way will replace your current database with the new data. Any process that works with .osc files will augment your already existing data.

(13 Aug, 13:30) Frederik Ramm ♦

That cleared up a lot for me, thanks. So I tried that out with this command:

osmium cat -f osm ../canada.osm.pbf | ./bin/update_database --db-dir=./db/

And it didn't do anything? It went through saying

elapsed node 721769976. Flushing to database ...... done.     ]   4%              ]   0%

...

compute_geometry: Node 9002479052 used in way 19693304 not found.

From my understanding (which may be wrong) this shouldn't be happening? I thought it would be taking information from canada.osm.pbf (which presumably doesn't have missing nodes like this) and replacing my current database with that information.

(13 Aug, 17:01) nbreden

I won't delete my previous comment, but it's wrong. Your command did work! Thank you so much! I think this will be enough of a solution for my problems.

(13 Aug, 20:59) nbreden
showing 5 of 6 show 1 more comments
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×445
×191
×111
×4

question asked: 12 Aug, 15:14

question was seen: 275 times

last updated: 13 Aug, 21:01

powered by OSQA