I am using wget to request data on the national borders of a few countries in Western Europe from the Overpass API. I use the following request:

wget "http://overpass-api.de/api/interpreter?data=[out:xml][timeout:3600];(relation(45.0,-6.0,56.0,15.5)["boundary"="administrative"]["admin_level"="2"];node(r)->.admincentre;way(r);node(w););out;" -O res.osm

However, each time I get the following error response after a couple of minutes (the first part is in Dutch ;) ):

HTTP-verzoek is verzonden; wachten op antwoord... 504 Gateway Timeout

Is there any way to resolve this error? I set the timeout to 3600 seconds, but it seems to be ignored. Is that on purpose and is my request too large? If this is the case, are there other options to request my data when sticking to the Overpass API?

Thanks in advance.

asked 03 Feb '16, 22:12

Steijn's gravatar image

Steijn
61569
accept rate: 0%

edited 03 Feb '16, 22:17

Did you try one of the other Overpass API instances?

(04 Feb '16, 11:42) scai ♦
2

You may try to get your data country by country and later stitch the results with osmconvert or something else depending what you do with the data. You can use capitals to run your query on.

Reasonability: Some borders are fairly complex. For example Italy (7,600 km border length) is ~10MB due to mostly having marine border and simple land border but a small luxembourg (359 km border length) is already ~3MB. Germany sits on 30MB of data.

(04 Feb '16, 18:13) RM87
1

scai: I tried other overpass instances, but they yield the same error response. RM87: Ah, that's a nice approach. It works!

(04 Feb '16, 18:51) Steijn

The Overpass website gives the following information:

504 Gateway Timeout is sent if the server has already so much load that the request cannot be executed. In most cases, it is best to try again later. Please note that the server decides this based on the timeout and maxsize parameters of the request, so smaller values for them may also make the request acceptable.

So to me the problem seems to be related to the complexity of my request and the amount of requests made to the server at that moment. Cutting the requests in parts, as RM87 suggested earlier, indeed solves the problem.

However, is it possible to show some kind of traffic diagram of the Overpass instances? That way I can do my requests, I have several of them and also more complex, in the more quiet hours. :)

permanent link

answered 04 Feb '16, 18:58

Steijn's gravatar image

Steijn
61569
accept rate: 0%

wget has its own timeout. Try to specify the same timeout via --read-timeout=3600. The default read timeout is 900 seconds according to my manpage.

permanent link

answered 04 Feb '16, 08:07

scai's gravatar image

scai ♦
31.4k18285439
accept rate: 23%

1

hmm, I think the 504 is sent from the server because an upstream server (possibly the overpass interpreter) has not answered yet. See http error codes. So, wget's own timeout should not have mattered - yet.

(04 Feb '16, 09:00) aseerel4c26 ♦

You are probably right. If a read timeout occurs then wget prints a different error message.

(04 Feb '16, 11:40) scai ♦

I added the timeout flag of wget to my request. However it doesn't seem to solve my problem. Thanks anyway. :)

(04 Feb '16, 18:49) Steijn
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×355
×6

question asked: 03 Feb '16, 22:12

question was seen: 4,455 times

last updated: 04 Feb '16, 18:59

powered by OSQA