Using Osmosis, bzcat and Postgresql

Question

I'm using Osmosis to load an extract of OSM data for the USA into a Postgresql database. I have previously used osm2pgsql.exe to load data into a postgresql database, but it is taking an extremely long time to load on my computer (Win 7 64 bit) using osm2pgsql 32 bit with the -s argument. I'm a unclear about a couple of things using osmosis and bzcat. On the Osmosis wiki page I see the following instructions:

Import a planet file into a local PostgreSQL rails port database.

osmosis --read-xml file="planet.osm" --write-apidb host="x" database="x" user="x" password="x"

I changed this line to the following, and tried adding a port argument

osmosis --read-xml file="J:\DATA\OSM\massachusetts.osm" --write-apidb host="localhost" database="osm" user="postgres" password="**" port="5436"

The error message I got was the following:

J:\osmosis-0.40.1\bin>osmosis --read-xml file="J:\DATA\OSM\massachusetts.osm" --write-apidb host="localhost" database="osm" user="postgres" password="***" port="5436"
Mar 19, 2012 11:28:16 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.40.1
Mar 19, 2012 11:28:29 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Mar 19, 2012 11:28:32 AM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Argument port for task 2-write-apidb was not recognised.
        at org.openstreetmap.osmosis.core.pipeline.common.TaskManagerFactory.createTaskManager(TaskManagerFactory.java:64)
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.buildTasks(Pipeline.java:50)
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.prepare(Pipeline.java:112)
        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:86)
        at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
        at org.codehaus.classworlds.Launcher.main(Launcher.java:47)

Where I can I pass the port-number? While I am using localhost, my port-number is 5436 for this database. More generally, what does a rails port mean? Do I need to create a set of tables to match what the data is importing?

I am also trying to use bzcat to pull out an extract of the data so that I could use osm2pgsql.exe but this does not work.

bzcat J:\DATA\OSM\massachusetts.osm.bz2 | J:\osmosis-0.40.1\bin\osmosis.bat --read-xml enableDateParsing=no file= --bounding-box top=42.48 left=-71.31 bottom=42.23 right=42.48 --write-xml file=  | bzip2 > J:\DATA\OSM\extracted.osm.bz2

When I ran this, I got an error message saying that bzip.dll was not on my system. I just saved the file linked on this page and tried running that but it seems not to work.

Accepted Answer

First off osm2pgsql is importing to a postgis scheme usfull for rendering, osmosis is importing to a different postgresql scheme that is used by the api. The rails port is the implementation of the api and website at osm.org. These two schemes are not the same.

You are dealing with huge data here and you should give osm2pgsql lots of memmory, compile it in 64 bit mode, tune your database and use fast disks.

As for the dll problems you have, osmosis --read-xml and --write-xml have the option compressionMethod that can be set to gzip, bzip2, none or auto. The default is auto so your command should be

 J:\osmosis-0.40.1\bin\osmosis.bat --read-xml enableDateParsing=no file="J:\DATA\OSM\massachusetts.osm.bz2" --bounding-box top=42.48 left=-71.31 bottom=42.23 right=42.48 --write-xml file="J:\DATA\OSM\extracted.osm.bz2"

Most of the tools in osm are not tested in windows and linux is prefered for development and hosting. You should try out a common linux distro like Ubuntu if you run into more problems.

Using Osmosis, bzcat and Postgresql

Follow this question

Related questions