Overpass API area query does not recognise all areas!

Question

Hello,

After installing a fresh version of the overpass api on my private server, I have imported a planet file (http://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/planet/planet-latest.osm.bz2) and most of the features are running smoothly. However, running is_in area queries only returns some areas like countries, rather than more specific areas like buildings. Has anyone encountered a problem like this before? There doesn't seem to be any sort of documentation on this matter. I have not altered the areas.osm3s file and am running version osm3s_v0.7.50.

Example problem:

If I run this request from the public server:

http://overpass-api.de/api/interpreter?data=%0A%5Bout%3Ajson%5D%3B%0Ais_in(40.70523754289769%2C-74.05129551887512)%3Bout%3B%0A

(should return Liberty State Park as an area),

my custom server does not return Liberty State Park, but does return everything else. Does anyone know what the problem here could be?

Thanks to everyone.

Accepted Answer

I could import a 26GB pbf planet file into my local Overpass API instance on my laptop (4GB RAM, SSD, 16GB swap space) without much issue. Database import took around 38 hours, area creation around 6 hours. Total disk usage now is about 182GB for the database. Note: No attic data at this time (I don't need it right now).

For the area creation I used the original script, but had to increase the element-limit to "2073741824" - otherwise the script would terminate with a runtime error "Query run out of memory in "recurse" at line 255 using about 1157 MB" as you mentioned. This is not an OS level error but enforced by Overpass API itself.

I could not reproduce the missing tags as you describe it. However, I removed all "area*" files in the database directory after a failed area creation before trying again. Not sure, if this is really necessary. Just wanted to give it a clean start.

Answer 2

1

Please check the following things:

does "way(27459219);out meta;" find the Liberty Park outline and has it a name tag and a tag "leisure=park", and is it version 25?
does the areas.osm3s in $DB_DIR/rules contain the rule for "leisure" (I know it's unchanged but just in case)
Are there two dispatcher processes running (one with --osm-base and one with --areas)?
What is the content of $DB_DIR/rules_loop.out (last 10 lines should be sufficient)?

permanent link

answered 01 Sep '14, 07:37

Roland Olbricht
6.7k●3●64●89
accept rate: 36%

edited 01 Sep '14, 07:38

Thanks for the response,

"way(27459219);out;" finds Liberty Park, it has a name and leisure tag (but I do not have metadata so cannot see version number)
rules contains the leisure node
Both dispatcher processes are running
Last 9 lines of rules_loop.log are:

2014-08-31 22:49:36: update started 2014-08-31 23:54:41: update finished 2014-08-31 23:54:44: update started 2014-09-01 01:00:35: update finished 2014-09-01 01:00:38: update started 2014-09-01 02:05:22: update finished 2014-09-01 02:05:25: update started 2014-09-01 03:09:48: update finished 2014-09-01 03:09:51: update started

(01 Sep '14, 08:32) gmeister4

Any suggestions as to what could be going wrong? All other node/ way queries work that I have tried so far

(01 Sep '14, 15:19) gmeister4

At least it is nothing obvious. The area generation is suspiciously fast (should be closer to 7 hours than to 1.5 hours). So I would ask you to check

Does "area[leisure=park];out ids;" have any results?
When you look at $EXEC_DIR/bin/nohup.out, do you have a line containing the term "part 0, on line 257" (or line 255)? This would tell us whether the area generation terminates for some reason in between.

(01 Sep '14, 16:55) Roland Olbricht

Area[leisure = park] does return around 4500 id's, however, $EXEC_DIR/bin/nohup.out does not exist but you were right! in /root/nohup.out I have found After 1h4m32s: in "recurse", part 0, on line 255. Stack: 1 of 3884956 0 of 1 and also runtime error: Query run out of memory in "recurse" at line 255 using about 1157 MB of RAM.. My server has 4gb. Note: My server is running 99.9% RAM load, with 'top' telling me that osm3s_query is running heavy:

6898 root 20 0 725m 215m 872 R 94.2 5.3 19:51.32 osm3s_query

What can I do to rectify this? Thanks Roland.

(01 Sep '14, 18:34) gmeister4

Please replace the areas.osm3s in $DB_DIR/rules by the file http://overpass-api.de/misc/areas_low_ram.osm3s

It contains the same queries, but it generates areas immediately after each block. It is probably slower, but it should use less RAM.

A second approach would be to raise the number 1073741824 in the first line of areas.osm3s or its replacement. That is the soft upper limit of RAM the query should use. But it likely won't help much if the script is killed by the operating system instead of the scheduler, because it hits the hard limit of 4GB RAM minus other processes.

(01 Sep '14, 19:16) Roland Olbricht

I have managed to complete the first block in 8h 30m with the areas_low_ram. All of the areas are now showing up, but many of the areas do not have all their correct tags, such as the 'name' tag. Is this due to the modified script? Or should these tags appear after successive blocks?

(03 Sep '14, 09:56) gmeister4

Still not the correct tags

(04 Sep '14, 11:27) gmeister4

showing 5 of 7 show 2 more comments

Overpass API area query does not recognise all areas!

Follow this question

Related questions