NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

Hi there,

I am trying to import an osm planet file in .pbf format into PostGIS and it appears to have just stopped after about 5 days of running. I've read through quite a few of the posts in this forum and not found an answer that seems to explain what has happened. As far as I understand it I should get some information about how long it has taken once it has finished but the last thing it shows me is:

'Processing: Node<1929394k 225.9k/s> Way<186904k 1.37k/s> Relation<477740 9.65/s>'

I've checked the event logs and I can't see anything that says I've run out of memory or space and when I look in PostgreSQL there is only one table called spatial_ref_sys. I also read somewhere that PostgreSQL has to run indexing and what not, so I'm not sure if that is what is happening at the moment.

My server set up is thus:

  • Windows Server 2008 R2 - 64bit
  • 2.00Ghz Dual Core
  • 24 GB RAM
  • 350GB HDD

The command I used to set osm2pgsql running was:

osm2pgsql -S "C:/Program Files/cygwin-package/default.style" -U postgres -d maps C:/Users/jordane/Desktop/Maps/planet-130614.osm.pbf -W --port 5432 --host localhost -s --flat-nodes flat-nodes.bin

Can anyone tell me what happened and how to fix it or is it still running in the background?

UPDATE: So, I figured I may as well run it again and see what happens, in case a process had stopped last time and so killed it. I got exactly the same issue happen again and it stopped at exactly the same point. I was keeping an eye on how much space I had available as it was running and it didn't come anywhere close to filling my disk. It finished overnight so I can't say how much space it was taking up when it stopped but I got no warnings to say it ran out of space. I have however noticed that the used space on my disk has gone down by about 40-50GB since it stopped running, so I can only assume that it has hit a problem and cleaned up the partially converted data.

Am I missing something obvious here? Is there a load of data that it has dumped somewhere other than PostgreSQL? I would really appreciate some pointers as I am all out of ideas bar downloading a new planet file and just trying it again.

Thanks

asked 11 Apr '16, 12:54

Elea%20J's gravatar image

Elea J
26224
accept rate: 0%

edited 15 Apr '16, 09:38


I think that it's fair to say that there are far fewer people using osm2pgsql on Windows than on e.g. Linux - that's probably why you didn't get any answers here.

There has been some Windows activity at https://github.com/openstreetmap/osm2pgsql/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+windows (your question there is at the top of the list).

My first thought is "are you running out of memory?". Have you tried a much smaller data extract (a city or very small country)?

My second is "what do you want to do with the data afterwards"? Your question at https://help.openstreetmap.org/questions/49457/ways-of-loading-a-planet-file-into-a-gui-and-exporting-tiles-from-it suggests that you want to generate tiles. I'm unaware of a well-documented software stack that will do this on Windows.

My suggestions are:

  1. Use Ubuntu Linux 14.04 LTS as the base OS, and follow https://switch2osm.org/serving-tiles/manually-building-a-tile-server-14-04/
  2. Initially use a very small data extract, not a full planet.
  3. When you're happy that you can generate tiles successfully, consider changing to larger data extracts or to a different platform, but bear in mind that for Windows you may need to write something that doesn't exist currently (even if it's only documentation).
permanent link

answered 27 Apr '16, 14:14

SomeoneElse's gravatar image

SomeoneElse ♦
36.9k71370866
accept rate: 16%

Thanks I'll try those. Could I really be running out memory with 24GB of RAM? I kept an eye on it (during the days) and I only ever used about 10-15% of the available RAM.

(27 Apr '16, 14:21) Elea J

Depending on where you got osm2pgsql from, I wouldn't assume that it could address all 24Gb of RAM on Windows.

(27 Apr '16, 14:29) SomeoneElse ♦

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×263
×134
×48

question asked: 11 Apr '16, 12:54

question was seen: 4,560 times

last updated: 27 Apr '16, 14:29

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum