NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

I've recently created my own Overpass server on an m5.large EC2 instance in AWS (2 vCPU, 8GB Memory). It's continually at 100% CPU usage, and I'm wondering if that's normal or if I should be using a larger EC2 instance or if something isn't working properly. I'm about to run many queries against this server for thousands of city's street data, so I want to make sure it's as healthy as possible.

Truncated output of top: %CPU %MEM TIME+ COMMAND 99.0 65.2 953:11.45 ./osm3s_query --progress --rules 73.4 1.1 0:20.51 ./update_from_dir --osc-dir=/tmp/osm-3s_update_O9Zfhz --version=2019-09-06T17\:32\:01Z --meta --flush-size=0

Contents of my rules_loop.log file: 2019-09-04 00:41:05: update started 2019-09-05 00:42:02: update finished 2019-09-05 00:42:05: update started 2019-09-06 00:43:12: update finished 2019-09-06 00:43:15: update started

asked 06 Sep '19, 18:41

JamesChevalier's gravatar image

JamesChevalier
1517713
accept rate: 25%

edited 06 Sep '19, 18:44


I had to resize my EC2 instance, so I stopped the server by running:

  • bin/dispatcher --osm-base --terminate
  • bin/dispatcher --areas --terminate
  • sudo shutdown now -h

Then I started the server back up and ran these commands:

  • rm -f db/osm3s_v0.7.55_osm_base
  • nohup bin/dispatcher --osm-base --meta --db-dir="db/" >> osm_base.out &
  • chmod 666 "db/osm3s_v0.7.55_osm_base"
  • nohup bin/fetch_osc.sh `cat db/replicate_id` "http://planet.openstreetmap.org/replication/minute/" "diffs/" >> fetch_osc.out &
  • nohup bin/apply_osc_to_db.sh "diffs/" `cat db/replicate_id` --meta=yes >> apply_osc_to_db.out &
  • rm -f db/osm3s_v0.7.54_areas
  • nohup bin/dispatcher --areas --db-dir="db/" >> areas.out &

I did not re-run nohup bin/rules_loop.sh "db" & after reboot. I'm unsure if I need to, and things seem to be working well without it. The server does not run at continuous 100% CPU usage any more, after the reboot.

permanent link

answered 10 Sep '19, 23:58

JamesChevalier's gravatar image

JamesChevalier
1517713
accept rate: 25%

edited 11 Sep '19, 00:00

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×483

question asked: 06 Sep '19, 18:41

question was seen: 1,273 times

last updated: 11 Sep '19, 00:00

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum