missing ways while using osmconvert clip

I am using osmconvert to clip countries/regions from the planet.osm.pbf file. My end goal is creating a multimodal transport network, using (mostly) osm data. Mainly for academic purposes. To speed up this creation process, I am splitting up the planet file into countries/regions to create this network parallel. Recently, I have started to realize that I am missing quite some ways in the extracted/clipped files.

For example, openstreetmap.org shows a near complete and connected railway system for tanzania. In my extracted osm.pbf for tanzania (either using a bbox or a .poly file), I am missing quite some railway segments. The same issue is the case for the download.geofabrik.de extracts (at least in the case for Tanzania, where I started to realize this issue).

Perhaps good to emphasize: I am not missing these segments around the borders (this is ofcourse already addressed in the osmconvert wiki), it is really in the middle of the bbox/.poly file.

The osmconvert code I am using is:

osmconvert64 planet-latest.osm.pbf -B=tanzania.poly --complete-ways -o=tanzania.osm.pbf'

Did anyone have this problem with osmconvert before? And more importantly, found a solution to the problem?

Thanks!

osmconvert clip

asked 21 Jun '18, 09:04

ElcoK
11●1●1●3
accept rate: 0%

edited 21 Jun '18, 09:24

Without knowing exactly what you're doing it may be difficult for people to comment. Could you provide a bit more detail and some examples?

(21 Jun '18, 09:17) SomeoneElse ♦

If you are saying that the Geofabrik download for Tanzania is missing data, I would be interested in a way ID so I can check.

(21 Jun '18, 09:37) Frederik Ramm ♦

Hi Frederik,

One example is Way: 396139210

At least, I don't see it in the file I've downloaded today (and yesterday) from geofabrik. It use to be there, and it is also on openstreetmap.org

If you open the lines from tanzania-latest.osm.pbf in QGIS and overlay this with the openstreetmap, you'll see that (for instance in Dar Es Salaam) also many roads are missing.

(21 Jun '18, 09:43) ElcoK

Must be an issue with QGIS then, since the way is definitely present in tanzania-latest.osm.pbf. Perhaps whatever logic QGIS applies is stumped by the fact that your way is part of relation 1183489 which is (at the time of writing with v99 being current) erroneously tagged as highway=residential?

(21 Jun '18, 10:41) Frederik Ramm ♦

I've narrowed down the problem now. The reason why my confusion started was because some part of the railway in TZA (not the way I mentioned above) was missing (this is apparently deleted by someone, even though it is still there).

So the problem is not in the osmconvert clip (which is good to know!), but apparently something goes wrong in the ogr2ogr drivers when trying to directly load in the data (either through python directly or QGIS, as they were missing the quite often the same elements).

After extracting all line features from the pbf file by using the following code in python, I miss quite a lot of segments.

driver=ogr.GetDriverByName('OSM')
data = ogr.Open('C://planet_osm//tanzania-latest.osm.pbf')
layer = data.GetLayer('lines')

features=[x for x in layer]

for feature in features:
    if feature.GetField('railway') is not None:
        osm_id = feature.GetField('osm_id')

(21 Jun '18, 11:21) ElcoK

My default ogr2ogr osm => geojson places railway in other tags for lines. It may be worth checking say against geojson for the same feature using overpass-turbo.

(21 Jun '18, 17:12) SK53 ♦

Hmm interesting point. As this is now turning into a osgeo/gdal question (why different results between ogr2ogr command line and python import), I will post that question at stackoverflow. Thanks for the help!

(22 Jun '18, 08:21) ElcoK

... which is this one, just for reference: https://gis.stackexchange.com/questions/287193/different-results-ogr2ogr-command-line-vs-osgeo-import-ogr-with-reading-osm-data

(22 Jun '18, 10:14) scai ♦

showing 5 of 8 show 3 more comments

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

*italic* or _italic_
**bold** or __bold__
link:[text](http://url.com/ "title")
image?![alt text](/path/img.jpg "title")
numbered list: 1. Foo 2. Bar
to add a line break simply add two spaces to where you would like the new line to be.
basic HTML tags are also supported

learn more about Markdown

Question tags:

osmconvert ×47
clip ×2

question asked: 21 Jun '18, 09:04

question was seen: 1,743 times

last updated: 22 Jun '18, 10:14

missing ways while using osmconvert clip

Follow this question

Related questions