NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

I am running osm2postgresql on an Ubuntu virtual machine (EC2 micro instance) using the instructions provided on the wiki.

Everything runs fine but I get the following error after the program has been running for a long time:

Mar 04, 2014 2:42:31 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
FINE: Waiting for task 1-read-pbf to complete.
Mar 04, 2014 3:29:56 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-pbf failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error has occurred, aborting.
        at org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:78)
        at org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:134)
        at org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:184)
        at org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.process(EntityBuffer.java:38)
        at crosby.binary.osmosis.OsmosisBinaryParser.parseWays(OsmosisBinaryParser.java:172)
        at crosby.binary.BinaryParser.parse(BinaryParser.java:121)
        at crosby.binary.BinaryParser.handleBlock(BinaryParser.java:68)
        at crosby.binary.file.FileBlock.process(FileBlock.java:135)
        at crosby.binary.file.BlockInputStream.process(BlockInputStream.java:34)
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:37)
        at java.lang.Thread.run(Thread.java:744)

Mar 04, 2014 3:29:56 PM org.openstreetmap.osmosis.pgsnapshot.common.CopyFileWriter release
SEVERE: Unable to close writer.
java.io.IOException: No space left on device
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:345)
        at java.io.BufferedOutputStream.Mar 04, 2014 3:29:57 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 2-buffer failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to write value (SRID=4326;POLYGON((-93.2169649 44.9913991,-93.2169649 44.9914236,-93.2169311 44.9914236,-93.2169311 44.9913991,-93.2169649 44.9913991)))
        at org.openstreetmap.osmosis.pgsnapshot.common.CopyFileWriter.writeField(CopyFileWriter.java:253)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.CopyFilesetBuilder.process(CopyFilesetBuilder.java:187)
        at org.openstreetmap.osmosis.core.container.v0_6.WayContainer.process(WayContainer.java:60)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.CopyFilesetBuilder.process(CopyFilesetBuilder.java:115)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.PostgreSqlDumpWriter.process(PostgreSqlDumpWriter.java:58)
        at org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.run(EntityBuffer.java:74)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: No space left on device
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:345)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
        at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
        at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
        at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
        at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
        at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
        at java.io.BufferedWriter.write(BufferedWriter.java:230)
        at java.io.Writer.write(Writer.java:157)
        at org.openstreetmap.osmosis.pgsnapshot.common.CopyFileWriter.writeField(CopyFileWriter.java:249)
        ... 6 more

Mar 04, 2014 3:29:57 PM org.openstreetmap.osmosis.core.pipeline.common.PassiveTaskManager waitForCompletion
FINE: Task 3-write-pgsql-dump is passive, no completion wait required.
Mar 04, 2014 3:29:57 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
        at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
        at org.codehaus.classworlds.Launcher.main(Launcher.java:47)

Error 1. The command which triggered the error was:
./$osmosis_version/bin/osmosis -v --read-$filetype file="$osmfile" --buffer --write-pgsql-dump enableBboxBuilder=yes enableLinestringBuilder=yes nodeLocationStoreType="TempFile" directory="$temporarydir"
on line 435 of the osm2postgresql script.
## To delete the imported or temporary data, you might need to
 ## run something similar to this (use with care the lines with the '*'):
 # To remove osmosis temporary data:
 rm osmosis-0.40.tgz
 rm -r osmosis-0.40/
 rm -r ./tempgis
 # To delete the imported data, you may need to run:
 dropdb -p 5432 gis
 # If you created a tablespace:
 echo "DROP TABLESPACE ;" | /usr/lib/postgresql/9.1/bin/psql -p 5432
 # If you created a tablespace or used the --install option:
 sudo rm -r /var/lib/postgresql/PostgresPlus/data

Only 20% of the disk space (out of 30 gigs) has been used. How can I resolve this error?

asked 04 Mar '14, 16:40

vr3690's gravatar image

vr3690
664411
accept rate: 0%

edited 04 Mar '14, 16:56

scai's gravatar image

scai ♦
33.3k21309459

1

30 GB disk space is very little. Are you sure you aren't using more than 20% during the run of osm2postgresql? I would expect it to clean up afterwards, even if an error occurs. And how large is the file you are trying to import?

(04 Mar '14, 16:59) scai ♦

How big's the PBF you're using?

(04 Mar '14, 17:08) SomeoneElse ♦

The PBF file is about 144MB in size. After the temp files are removed, around 9% of the disk space shows up as used. Should I be using a bigger disk?

EDIT: just checked again. The PBF file is around 137 MB downloaded from here - http://download.geofabrik.de/north-america/us/minnesota.html

(04 Mar '14, 17:10) vr3690

First of all, are you really sure you need osm2postgresql? The program has been last modified 3 years ago. A lot has happened since then. For most uses, osm2pgsql or imposm are the superior choices.

osm2postgresql will download and install an Osmosis version (0.40) that is known to have issues with 64-bit node IDs in some situations, but I am unsure if this would affect your use case.

Osmosis will write a temporary file to store node locations, and the results of Osmosis will be stored in another temporary file, to be loaded into PostgreSQL after; meaning that the process is likely to temporarily use much more memory than needed in the end. osm2postgresql lets you select where these files are stored; make sure you're not accidentally writing them to a partition that only has a fraction of the total hard disk available.

But again, check if you really want to use osm2postgresql at all.

permanent link

answered 05 Mar '14, 09:06

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
82.5k927201273
accept rate: 23%

edited 05 Mar '14, 09:54

Pieren's gravatar image

Pieren
9.8k2083157

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×263

question asked: 04 Mar '14, 16:40

question was seen: 7,407 times

last updated: 05 Mar '14, 09:54

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum