When I try to import planet-latest.osm.pbf I get the following error:

/usr/local/bin/osm2pgsql --create --flat-nodes --slim --username postgres --password --hstore --hstore-match-only --number-processes 6 --cache 50000 --style default.style --input-reader pbf planet-latest.osm.pbf
osm2pgsql SVN version 0.89.0-dev (64 bit id space)

Using built-in tag processing pipeline  
Using projection SRS 900913 (Spherical Mercator)  
Setting up table: planet_osm_point  
Setting up table: planet_osm_line  
Setting up table: planet_osm_polygon  
Setting up table: planet_osm_roads  
Allocating memory for dense node cache  
Allocating dense node cache in one big chunk  
Allocating memory for sparse node cache  
Sharing dense sparse  
Node-cache: cache=50000MB, maxblocks=800000*65536, allocation method=3  
Mid: Ram, scale=100  
Reading in file: planet-latest.osm.pbf  
Using PBF parser.  
Processing: Node(3095464k 2705.8k/s) Way(53703k 29.11k/s) Relation(0 0.00/s)Killed

Here is the message I get from dmesg:

Out of memory: Kill process 23636 (osm2pgsql) score 953 or sacrifice child  
Killed process 23636 (osm2pgsql) total-vm:75687764kB, anon-rss:60639884kB, file-rss:0kB

do I need more that 50GB RAM to import the planet file?

asked 12 Nov '15, 15:42

khamooshi's gravatar image

khamooshi
146101219
accept rate: 50%

edited 13 Nov '15, 14:54

Lightsider's gravatar image

Lightsider
1.5k52129


To import the whole planet, you need to use slim mode, but with your command line you are saving the flat nodes in a file called --slim. Add a file name after --flat-nodes.

Also, -C 28000 is plenty of cache to store all nodes in RAM.

permanent link

answered 13 Nov '15, 22:33

pnorman's gravatar image

pnorman
2.3k52140
accept rate: 19%

--cache 50000 means you are assigning 50000 MB (50 GB) of your memory as node cache. This is too high when having "only" 50 GB of RAM. I'm not familiar with osm2pgsql so I can't suggest a good value for this option, but try to assign less than all your available RAM to it.

permanent link

answered 12 Nov '15, 15:51

scai's gravatar image

scai ♦
31.1k18279434
accept rate: 23%

Actually I have 60GB RAM, and I assigned 50GB. I also tried with 40GB, same thing happened!

(12 Nov '15, 15:58) khamooshi

... and I think (based on the fact that you've asked other questions about rendering) that other (smaller) imports have worked OK in the past?

(12 Nov '15, 16:36) SomeoneElse ♦

I didn't set up the other severs, so I don't know how they did it! btw, I use --flat-nodes and different projection.

(13 Nov '15, 08:42) khamooshi
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×200
×166

question asked: 12 Nov '15, 15:42

question was seen: 2,718 times

last updated: 13 Nov '15, 22:33

powered by OSQA