Requests limitation and timeout

Question

I'm working on my master thesis for which I need the number of points of interest that can be considered cultural in 43 Italian cities. Basically, what I need is the result of the following query:

query = """
area[name="""+city_name+"""]->.a;
(
  node["amenity"="college"];
  way["amenity"="college"];
  relation["amenity"="college"];

  node["amenity"="library"];
  way["amenity"="library"];
  relation["amenity"="library"];

  node["amenity"="archive"];
  way["amenity"="archive"];
  relation["amenity"="archive"];

  node["amenity"="school"];
  way["amenity"="school"];
  relation["amenity"="school"];

  node["amenity"="music_school"];
  way["amenity"="music_school"];
  relation["amenity"="music_school"];

  node["amenity"="university"];
  way["amenity"="university"];
  relation["amenity"="university"];

  node["amenity"="research_institute"];
  way["amenity"="research_institute"];
  relation["amenity"="research_institute"];

  node["amenity"="arts_centre"];
  way["amenity"="arts_centre"];
  relation["amenity"="arts_centre"];

  node["amenity"="cinema"];
  way["amenity"="cinema"];
  relation["amenity"="cinema"];

  node["amenity"="community_centre"];
  way["amenity"="community_centre"];
  relation["amenity"="community_centre"];

  node["amenity"="arts_centre"];
  way["amenity"="arts_centre"];
  relation["amenity"="arts_centre"];

  node["amenity"="planetarium"];
  way["amenity"="planetarium"];
  relation["amenity"="planetarium"];

  node["amenity"="social_centre"];
  way["amenity"="social_centre"];
  relation["amenity"="social_centre"];

  node["amenity"="studio"];
  way["amenity"="studio"];
  relation["amenity"="studio"];

  node["amenity"="theatre"];
  way["amenity"="theatre"];
  relation["amenity"="theatre"];
)
"""

As you can see, I'm interested in 15 amenities, not a negligible number for a metropolitan area. Despite the chance to set an arbitrary timeout, I faced the limitation of 3 minutes set on the server. Therefore I used the python overpass API to ask one amenity at a time (15 queries for 43 cities), but before finishing the first 43 queries, I received the following response: 'HTTP/1.1 429 Too Many Requests\r\n' I beg your pardon if my actions have been interpreted as a kind of DoS attack, but I would like to know if there is any chance to obtain the information that I need without violeting the policy of the server.

Answer 1

1

You could simply extract them from a extract of the OSM data for Italy (see http://download.geofabrik.de/europe/italy.html) for example by using osmfilter.

permanent link

answered 24 Jun '18, 12:04

SimonPoole ♦
44.7k●13●326●701
accept rate: 18%

edited 24 Jun '18, 12:06

Thank you for your answer. I'm trying to understand how to extract the number of amenities for each city. I was thinking about two steps: 1) ./osmfilter italy-latest.o5m --keep="amenity=college =cinema ..." 2) ./osmfilter italy-latest.o5m --out-count=addr:city | grep city_name With the first one I should obtain a .o5m file with nodes, ways and relations that match my interests. With the second I expect the number of amenities in that specific city. However, I think that not all the nodes, ways and relations contain the address field. Therefore, how should I automatically limit the area?

(24 Jun '18, 16:22) GiulioFa

If you are interested in a limited number of areas, I would simply just use extracts for those areas (that you could either get ready mode or extract yourself).

Well actually I wouldn't, because I would simply import the IT extract in to an osm2pgsql schema data base and get the the information from spatial queries (assuming that OSM contains boundary polygons for the cities you are interested in).

(25 Jun '18, 12:49) SimonPoole ♦

Requests limitation and timeout

Follow this question

Related questions