So I have this code that runs through a loop for municipality codes for Denmark, there are about 98 municipalities, using Python. I take all the nodes, ways and relations that match some amenities I want (pub|bar|cafe|restaurant| etc.) and put them into a list. It is very important that I have the municipality code for each amenity, and this is the only way I found out to make it work.

So I am doing 98 requests, I have had a sleep timer of 6 seconds, but now lately I am getting blocked even after just one request, even if I have a sleep timer of 10 seconds. Is there a way to rewrite this code so that I can do this in only one request perhaps, so that I dont get the 429 error? I send each request with my contact details (the mail was wrong I saw, but fixed now).

I have done a lot of requests earlier when testing the code, but then I have done it only for one municipality, checking if I am getting the result I want, and then retesting, but always a limited amount of requests.

for mun in range(len(mun_code)):
            overpass_query = """
            [out:json];
            area['ISO3166-1' = 'DK'][admin_level = 2] -> .a ;
            area["ref" = "{0}"][admin_level = 7] -> .b ;
            out body qt;
            (
             node(area.a)(area.b)["amenity" ~ "{1}"] ;
             way(area.a)(area.b)["amenity" ~ "{1}"] ;
             rel(area.a)(area.b)["amenity" ~ "{1}"] ;
            ) ;
            out {2} qt ;
            """.format(mun_code[mun], amenities, out)

            response = requests.get(overpass_url, 
                                    params={'data': overpass_query}, 
                                    headers = {
                                        'User-Agent': user_agent,
                                        'From': mail 
                                    })

asked 21 Feb, 12:49

Karl_NO's gravatar image

Karl_NO
11112
accept rate: 0%

edited 21 Feb, 14:06

Richard's gravatar image

Richard ♦
27.8k40247369

Did you think about downloading whole Denmark and performing your queries offline? E.g. by using osmium-tool, there are also python bindings available!

(21 Feb, 14:40) scai ♦
Be the first one to answer this question!
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×355
×3

question asked: 21 Feb, 12:49

question was seen: 314 times

last updated: 21 Feb, 14:40

powered by OSQA