|
I have this local setup of overpass in a 32core 16Gb-ram server. My requests are not processing parallel, when i trigger multiple threads of the same request, the total time taken is = (time taken to complete one request * no of threads). Can someone point me towards what am missing here!! So i would like to know if overpass handle's requests parallel or sequential ? Tnx in advance - AG |
|
As you can see in the munin stats for the official server, the server handles about 400-700 requests/minute. That for sure wouldn't be possible by processing a single request at a time. Also, the CPU utilization clearaly shows that all 8 cores all completely used up. Conclusion: processing is parallel by default, each request is handled by a different Apache CGI process up to a certain limit. Note that there's a limit of one request per IP at the same time on the official instance. That most likely doesn't apply if you're running your own instance, unless you set it up explicitly. Unless you're adding more detail on your setup and your query, it's impossible to tell what kind of issues your might see on your local instance. It might also be that you're hitting some I/O limit or experiencing CPU saturation, too many parallel requests for your machine, internal scalability issues in the dispatcher, wrong/insufficient configuration of your web server, your load driver may not work properly. There are literally dozens of different influencing factors. thanx for the info @mmd I'll debug my web server configuration's and look for issues anywhere else.
(20 May '16, 12:03)
Arun Gowtham
hi @mmd Here's the case am trying to solve: My query: [out:json];area[name='Tamil Nadu'];node(area)[amenity='atm'];out count; The time taken for this single request is : 1 new Thread started Time taken for req :1 : 1936 (ms) Whereas when i trigger 10 threads of the same request i get such result, 10 new Threads started Time taken for req :1 : 1870 (ms) Time taken for req :7 : 1869 (ms) Time taken for req :2 : 1870 (ms) Time taken for req :3 : 1870 (ms) Time taken for req :8 : 1868 (ms) Time taken for req :6 : 1869 (ms) Time taken for req :5 : 1947 (ms) Time taken for req :10 : 3588 (ms) Time taken for req :4 : 3590 (ms) Time taken for req :9 : 3950 (ms) Is this normal, what i am expecting is all those 10 requests should have completed approximately around 1870 (ms), but it took additional time. When i increase the thread count, say to 50, the total time taken goes upto 14 seconds. When increased ever further to 500, i start to get 504 error's for nearly 370 of the requests and the time taken for entire set including the errors is 35 seconds Please help me sort out this issue, is this functionality normal or have i configured wrong?! :( Thanks-AG
(20 May '16, 12:39)
Arun Gowtham
So it seems to scale fine up to 7 requests in parallel. Your system has 8 cores and surely has some other minor stuff to do while processing your requests. The remaining 3 requests take twice as long since they have to wait for the other requests to finish. Sounds fine to me. However I don't know any internals about Overpass. But just bumping up the allowed number of threads will usually have a negative impact on the system since they will also introduce additional overhead and will lead to processing more quests in parallel than the system can handle, slowing down each individual request.
(20 May '16, 14:21)
scai ♦
@scai actually my machine is 32 core cpu...
(20 May '16, 14:25)
Arun Gowtham
Oh, sorry. I've taken the 8 cores mentioned in mmd's answer for your system. Anyway, as already explained by mmd there can be multiple other reasons like limited I/O. You can for example check your wa (=wait) in
(20 May '16, 14:31)
scai ♦
yeah,i get you, i just wanted to know if what i stated above is not normal in overpass, so that i would start to debug elsewhere..
(20 May '16, 14:35)
Arun Gowtham
showing 5 of 6
show 1 more comments
|
|
We were facing the same problem, using an unofficial Docker image. It probably had a bug in its Nginx configuration. Updating this image solved it. |
