|
Whenever I try to import the planet.osm.bz2 or california.osm.bz2, I get this: bash-4.2$ ./osm2pgsql -S ./default.style ../california.osm Using projection SRS 900913 (Spherical Mercator) Reading in file: ../california.osm |
|
I would guess this is the "out of memory killer" shooting your process; type 1
I run into the same error by using the --slim option. There are any other solutions?
(10 Oct '12, 09:25)
marco-2012
Have you looked at the switch2osm.org site? While for smaller extracts you can use quite a small machine, you should take the systems requirements for a full planet import seriously.
(10 Oct '12, 09:41)
SimonPoole ♦
Yes, I used switch2osm.org/serving-tiles/manually-building-a-tile-server-12-04/ to set up a tile server on a virtual machine. I can import download.geofabrik.de/openstreetmap/europe/germany/hamburg.osm.pbf successfully and everything works fine. But if I try to import download.geofabrik.de/openstreetmap/europe/germany.osm.pbf I get the "out of memory killer". The virtual machine has 1 GB RAM. Is it to small?
(10 Oct '12, 09:57)
marco-2012
I suspect yes, Germany is something over 5% of all data. Very roughly I would suspect that you need at least 1GB just for the import process.
(10 Oct '12, 10:13)
SimonPoole ♦
Just another data point / anecdote - I struggled to load england (1/6 the size of germany) on a VM allocated 2Gb without fiddling with settings, so I'd have thought that 1Gb was too small. If buying more memory isn't an option, perhaps experiment with VM memory settings - monitor what's being used, try larger or smaller -C values, and try either swap within the VM or a larger memory usage (relying on swap on the host) to see what works best.
(10 Oct '12, 10:39)
SomeoneElse ♦
1
I increased the RAM to 2 GB and tried to increase and decrease the -C value, but I still got an error (not sure what, but no out of memory error). After I leaved the -C value (I don't know what the default -C value is) it works without any errors. Thanks for your help.
(10 Oct '12, 13:49)
marco-2012
showing 5 of 6
show 1 more comments
|
|
I had the same problem with canada-latest.pbf. (2gb file size) Using ksysguard I watched the osm2pgsql process consume first memory (4gb) and then swap (4gb) until all was gone and the process killed. To fix: su dd if=/dev/zero of=/home/RB/temp/extraswap bs=1M count=16384 add 16gb swap file mkswap /home/RB/temp/extraswap swapon /home/RB/temp/extraswap swapon -s and see added swap The pgsql data file was in /usr/share in the root directory (50gb). That was overdosed also so I moved /usr/share/pgsql to /home/RB/proj/Maps where mucho disk space is available. Good to go after that. The 2gb canada-latest required nearly 14gb of memory to process so there is a memory leak or poor architecture someplace in the osm2pgsql program -Rick Brown in Reno, NV The github for osm2pgsql is here: https://github.com/openstreetmap/osm2pgsql/issues I'd suggest explaining the problem there, and include details of the full command line and version that you used (since what osm2pgsql does depends very much on what you ask it to do - whether you ask it to create an updatable database, for example). If you can identify a "memory leak or poor architecture" in the code (or even better, suggest a fix) I'd describe that too.
(28 Aug '15, 17:50)
SomeoneElse ♦
|
