NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

Hi,

I'm doing a full planet ingest using osm2pgsql on an Amazon AWS instance. I've given it a 25 gb cache using slim mode, and it's been very speedy. I started the process yesterday morning and now we're at the "Going over pending ways" portion.

I'm concerned however that the process has stalled out. My log file has not updated for 2 hours and I am stuck at

    Going over pending ways...
    191006558 ways are pending

    Using 1 helper-processes

And that's the end of my log file. When it was processing the nodes/ways/rels, at least I got an update of how far it was, but now it seems to have possibly stalled since nothing has updated in 2 hours.

How can I tell how far along this portion of the process is? Or, how can I tell if it has stalled out?

asked 11 Jun '15, 16:23

pixelk8e's gravatar image

pixelk8e
26112
accept rate: 0%

Did you already check CPU usage and I/O load of your osm2pgsql process?

(11 Jun '15, 18:02) scai ♦

Yes, it is using ~10-20% of the cpu at any given time (averaging at about 14%), but i/o seems to be nonexistent (I'm running osm2pgsql to a database on another instance, so that makes sense to me). But I'm still concerned it's chugging away at nothing.

(11 Jun '15, 18:17) pixelk8e

What does "wa" (I/O wait) say in top?

(13 Jun '15, 19:52) scai ♦

There are current versions of osm2pgsql that don't provide any output during that stage.

For estimated times for import see http://wiki.openstreetmap.org/wiki/Osm2pgsql/benchmarks

Note that due to OSM data growth these are always out of date (but can give you a rough indication).

permanent link

answered 12 Jun '15, 18:58

SimonPoole's gravatar image

SimonPoole ♦
44.7k13326701
accept rate: 18%

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×263

question asked: 11 Jun '15, 16:23

question was seen: 3,821 times

last updated: 13 Jun '15, 19:52

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum