This is a static archive of our old OpenStreetMap Help Site. Please post any new questions and answers at community.osm.org.

incorrect data check

Yesterday I downloaded planet-latest.osm.pbf.

Today I tried to run:

osmosis --rb file="c:\osm_data\planet-latest.osm.pbf"  
--bp file="c:\osm_data\united_states.poly"  --bp file="c:\osm_data\canada.poly" 
--tf accept-ways highway=* 
--wb omitmetadata=true file="d:\spatialite_data\usa_can.osm.pbf

After running for about 15 minutes I get:

C:\Program Files (x86)\osmosis\bin>osmosis --rb file="c:\osm_data\planet-latest.osm.pbf"  --bp file="c:\osm_data\united_states.poly"
c:\osm_data\canada.poly" --tf accept-ways highway=* --wb omitmetadata=true file="d:\spatialite_data\usa_can.osm.pbf"
Jul 11, 2012 6:47:50 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.40.1
Jul 11, 2012 6:47:50 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Jul 11, 2012 6:47:50 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Jul 11, 2012 6:47:50 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
java.util.zip.DataFormatException: incorrect data check   <<===================     
        at java.util.zip.Inflater.inflateBytes(Native Method)
        at java.util.zip.Inflater.inflate(Unknown Source)
        at java.util.zip.Inflater.inflate(Unknown Source)
        at crosby.binary.file.FileBlockPosition.parseData(FileBlockPosition.java:57)
        at crosby.binary.file.FileBlockHead.readContents(FileBlockHead.java:95)
        at crosby.binary.file.FileBlock.process(FileBlock.java:135)
        at crosby.binary.file.BlockInputStream.process(BlockInputStream.java:34)
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:37)
        at java.lang.Thread.run(Unknown Source)
Jul 11, 2012 7:02:57 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-rb failed
java.lang.Error: java.util.zip.DataFormatException: incorrect data check
        at crosby.binary.file.FileBlockPosition.parseData(FileBlockPosition.java:60)
        at crosby.binary.file.FileBlockHead.readContents(FileBlockHead.java:95)
        at crosby.binary.file.FileBlock.process(FileBlock.java:135)
        at crosby.binary.file.BlockInputStream.process(BlockInputStream.java:34)
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:37)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.util.zip.DataFormatException: incorrect data check
        at java.util.zip.Inflater.inflateBytes(Native Method)
        at java.util.zip.Inflater.inflate(Unknown Source)
        at java.util.zip.Inflater.inflate(Unknown Source)
        at crosby.binary.file.FileBlockPosition.parseData(FileBlockPosition.java:57)
        ... 5 more

Jul 11, 2012 7:02:57 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
        at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
        at org.codehaus.classworlds.Launcher.main(Launcher.java:47)

C:\Program Files (x86)\osmosis\bin>

What is causing this? Is there a work-around?

Regards, Jim

windows data check osmosis

asked 12 Jul '12, 02:22

obrienj
16●3●3●3
accept rate: 0%

edited 12 Jul '12, 02:33

SomeoneElse ♦
36.9k●71●370●866

One Answer:

It looks like it's inside the crosby code (the pbf reader) at the point where it falls over. Perhaps it's incorrect data in planet-latest.osm.pbf (i.e. a download error)? You couldn't double-check the MD5, could you? There's an MD5 of a few-days-old planet here, but of course that may not be the one that you're downloading.

Another possibility, perhaps, may be Java running out of temporary storage, like in this question.

Finally, it's not trying to write any files locally (to "Program Files (x86)") is it? Because if so, and you're on Windows Vista or 7, UAC may try and stop you. I seem to remember that osmosis ships with a batch file on Windows these days - you may want to have a look at that to see if you want any of the settings from it. Also see the "Win 7 64 bit" note at the bottom of this page.

(all of this is, unfortunately, guesswork; I haven't run osmosis for ages and on Windows for even longer).

answered 12 Jul '12, 02:50

SomeoneElse ♦
36.9k●71●370●866
accept rate: 16%

You were correct, I checked the md5 sum of the downloaded file and it didn't match that on the website you referenced.

So, I know the "rest of the story".

Thanks for reminding me about the md5 check.

Regards, Jim

P.S. I won't be downloading the planet file again, it took 12 hours!!

(12 Jul '12, 03:13) obrienj