NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

I have a 8G RAM and trying to split the Planet OSM files to multiple continents, countries and regions. It took more than 1 day to split the Planet OSM to continent based OSM files. To split the continent OSM bz2 files further to smaller countries is taking more time. For example, the split of Asia osm bz2 file has been running since 2 days. The max heap value set to JVM is 5G. Any suggestions to speed up Osmosis?

Thanks

Kris

asked 25 Apr '12, 08:41

kris's gravatar image

kris
46447
accept rate: 0%


Never use bz2 with osmosis, it's a performance killer. Use pbf files if you can, and for your intermediate products (i.e. if the only reason you're making asia.osm.pbf is that you want to further split it in a second step) add the compress=none flag to --write-pbf (don't worry, the resulting file will only be twice the size, not ten times).

If you cannot use pbf files, at least use gz instead of bz2; if you really really have to use bz2, use an external program (bzcat planet.osm.bz2 | osmosis --read-xml -).

Generously use --buffer bufferCapacity=12000, and if you have two disks in your system, try to read from one while writing to the other.

The fastest way to split a planet file into components is to use Peter Koerner's OSM history splitter instead of Osmosis.

permanent link

answered 25 Apr '12, 08:48

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
82.5k927201273
accept rate: 23%

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×252
×34
×33
×28
×2

question asked: 25 Apr '12, 08:41

question was seen: 9,275 times

last updated: 25 Apr '12, 08:48

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum