This is a static archive of our old OpenStreetMap Help Site. Please post any new questions and answers at community.osm.org.

Should my Overpass server always be using 100% CPU?

I've recently created my own Overpass server on an m5.large EC2 instance in AWS (2 vCPU, 8GB Memory). It's continually at 100% CPU usage, and I'm wondering if that's normal or if I should be using a larger EC2 instance or if something isn't working properly. I'm about to run many queries against this server for thousands of city's street data, so I want to make sure it's as healthy as possible.

Truncated output of top: %CPU %MEM TIME+ COMMAND 99.0 65.2 953:11.45 ./osm3s_query --progress --rules 73.4 1.1 0:20.51 ./update_from_dir --osc-dir=/tmp/osm-3s_update_O9Zfhz --version=2019-09-06T17\:32\:01Z --meta --flush-size=0

Contents of my rules_loop.log file: 2019-09-04 00:41:05: update started 2019-09-05 00:42:02: update finished 2019-09-05 00:42:05: update started 2019-09-06 00:43:12: update finished 2019-09-06 00:43:15: update started

overpass

asked 06 Sep '19, 18:41

JamesChevalier
151●7●7●13
accept rate: 25%

edited 06 Sep '19, 18:44

One Answer:

I had to resize my EC2 instance, so I stopped the server by running:

bin/dispatcher --osm-base --terminate
bin/dispatcher --areas --terminate
sudo shutdown now -h

Then I started the server back up and ran these commands:

rm -f db/osm3s_v0.7.55_osm_base
nohup bin/dispatcher --osm-base --meta --db-dir="db/" >> osm_base.out &
chmod 666 "db/osm3s_v0.7.55_osm_base"
nohup bin/fetch_osc.sh `cat db/replicate_id` "http://planet.openstreetmap.org/replication/minute/" "diffs/" >> fetch_osc.out &
nohup bin/apply_osc_to_db.sh "diffs/" `cat db/replicate_id` --meta=yes >> apply_osc_to_db.out &
rm -f db/osm3s_v0.7.54_areas
nohup bin/dispatcher --areas --db-dir="db/" >> areas.out &

I did not re-run nohup bin/rules_loop.sh "db" & after reboot. I'm unsure if I need to, and things seem to be working well without it. The server does not run at continuous 100% CPU usage any more, after the reboot.

answered 10 Sep '19, 23:58

JamesChevalier
151●7●7●13
accept rate: 25%

edited 11 Sep '19, 00:00