Is there a limit, that following API can handle request per second, if I hosted in our own server?

http://127.0.0.1:5000/route/v1/driving/13.388860,52.517037;13.385983,52.496891?steps=true

I did exactly what has mentioned in the https://hub.docker.com/r/osrm/osrm-backend/ link to run the OSRM docker instruction. (Note I'm not using front-end)

Everything seems working fine, even If I execute the following API (note here the source limit is 3) http://127.0.0.1:5000/table/v1/driving/79.865184,6.856469;79.867051,6.858589;79.862052,6.860068;79.863039,6.862401?sources=1;2;3&destinations=0

I'm getting response in 44ms.

However, if I increase the sources information in above API like for 50 and send 1000 request per second, I'm getting 7 to 8 second response time and the CPU usage goes for 60-70 usage.

Here is my server and OSRM file size info: map.osrm file size: 1.6GB 16 Core CPU 24GB Ram ulimit -n 1024

I tried Nginx load balancing also as follows briefly:

docker run -t -i -p 5000:5000 -v $(pwd):/data osrm/osrm-backend osrm-routed --algorithm mld /data/map.osrm docker run -t -i -p 5001:5000 -v $(pwd):/data osrm/osrm-backend osrm-routed --algorithm mld /data/map.osrm

and accessed the API as:

http://127.0.0.1/table/v1/driving/79.865184,6.856469;79.867051,6.858589;79.862052,6.860068;79.863039,6.862401?sources=1;2;3&destinations=0

nginx.conf brief setup as follows:

----------------------------------------------------- `worker_processes auto;

events { worker_connections 2048; multi_accept on; }

upstream myapp1 { server 127.0.0.1:5000 weight=1; server 127.0.0.1:5001 weight=2;

}

server { listen 192.168.180.93:80;

    location / {
            proxy_pass http://myapp1;
    }

}` -----------------------------------------------------

Now my question is how can I make/keep it the response time very low (below 100ms) even if I increase it to 50 or more?

asked 10 Nov, 05:23

Fazlul-RM's gravatar image

Fazlul-RM
3115
accept rate: 0%

edited 10 Nov, 05:25


I think the 15ms per route that you measured in your first example is realistic. If you want to compute 50 routes that would then take 750ms of CPU time. Asking this in a single matrix request means it will be run on one CPU core alone so it will take 750ms wall clock time. If you split the request in 5 smaller requests with 10 sources each, and if you have 5 CPU cores available, it should be possible to bring down the response time to below 100ms.

There are some economies of scale with the matrix request of course, so you have to find the "sweet spot" - more routes per request means more latency because you have to wait for one core to complete them all, but the total CPU time spent per route will be smaller.

If you want to compute 50k routes per second (you said you were trying 1000 requests per second with 50 sources each), that will probably take somewhere around 500 seconds of CPU time, i.e. you need 500 CPU cores to do that in one second of wall clock time.

Note that you will be able to squeeze a little more performance from the server if you switch to CH instead of MLD. Maybe then you only need 250 cores.

permanent link

answered 10 Nov, 06:48

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
60.6k74564949
accept rate: 23%

edited 10 Nov, 06:50

Hi @Frederik, Thanks for your quick reply can you tell me what's the CH and MLD? sorry if it's sill qq.

(10 Nov, 08:53) Fazlul-RM

Contraction Hierarchies (CH) and Multi-Level Dijkstra (MLD) are different techniques for preprocessing the routing data. The Quickstart explains how to choose between those two.

(10 Nov, 12:23) scai ♦

Hi @Fredrik, Sorry for delay, is it possible to add a cluster for this?

(13 Nov, 14:24) Fazlul-RM
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×118
×9
×7
×3
×2

question asked: 10 Nov, 05:23

question was seen: 123 times

last updated: 13 Nov, 14:33

powered by OSQA