How many request per second OSRM/Backend docker can handle ?

Question

Is there a limit, that following API can handle request per second, if I hosted in our own server?

http://127.0.0.1:5000/route/v1/driving/13.388860,52.517037;13.385983,52.496891?steps=true

I did exactly what has mentioned in the https://hub.docker.com/r/osrm/osrm-backend/ link to run the OSRM docker instruction. (Note I'm not using front-end)

Everything seems working fine, even If I execute the following API (note here the source limit is 3) http://127.0.0.1:5000/table/v1/driving/79.865184,6.856469;79.867051,6.858589;79.862052,6.860068;79.863039,6.862401?sources=1;2;3&destinations=0

I'm getting response in 44ms.

However, if I increase the sources information in above API like for 50 and send 1000 request per second, I'm getting 7 to 8 second response time and the CPU usage goes for 60-70 usage.

Here is my server and OSRM file size info: map.osrm file size: 1.6GB 16 Core CPU 24GB Ram ulimit -n 1024

I tried Nginx load balancing also as follows briefly:

docker run -t -i -p 5000:5000 -v $(pwd):/data osrm/osrm-backend osrm-routed --algorithm mld /data/map.osrm docker run -t -i -p 5001:5000 -v $(pwd):/data osrm/osrm-backend osrm-routed --algorithm mld /data/map.osrm

and accessed the API as:

http://127.0.0.1/table/v1/driving/79.865184,6.856469;79.867051,6.858589;79.862052,6.860068;79.863039,6.862401?sources=1;2;3&destinations=0

nginx.conf brief setup as follows:

----------------------------------------------------- `worker_processes auto;

events { worker_connections 2048; multi_accept on; }

upstream myapp1 { server 127.0.0.1:5000 weight=1; server 127.0.0.1:5001 weight=2;

}

server { listen 192.168.180.93:80;

    location / {
            proxy_pass http://myapp1;
    }

}` -----------------------------------------------------

Now my question is how can I make/keep it the response time very low (below 100ms) even if I increase it to 50 or more?

Accepted Answer

I think the 15ms per route that you measured in your first example is realistic. If you want to compute 50 routes that would then take 750ms of CPU time. Asking this in a single matrix request means it will be run on one CPU core alone so it will take 750ms wall clock time. If you split the request in 5 smaller requests with 10 sources each, and if you have 5 CPU cores available, it should be possible to bring down the response time to below 100ms.

There are some economies of scale with the matrix request of course, so you have to find the "sweet spot" - more routes per request means more latency because you have to wait for one core to complete them all, but the total CPU time spent per route will be smaller.

If you want to compute 50k routes per second (you said you were trying 1000 requests per second with 50 sources each), that will probably take somewhere around 500 seconds of CPU time, i.e. you need 500 CPU cores to do that in one second of wall clock time.

Note that you will be able to squeeze a little more performance from the server if you switch to CH instead of MLD. Maybe then you only need 250 cores.

How many request per second OSRM/Backend docker can handle ?

Follow this question

Related questions