This is a static archive of our old OpenStreetMap Help Site. Please post any new questions and answers at community.osm.org.

How to read a .pbf file in Python (without reading the whole file at once)?

0

I need to parse a .pbf file in Python, but using apply_file() on PyOsmium handlers is not working well for me and so I want to use apply_buffer(). I don't quite know what format is expected from it and documentation on it is lacklustre, but I assume it's to be a binary string encompassing a fileblock.

The problem is that if I open() a file in Python I don't know of a way to find fileblocks to feed them to PyOsmium.

It might have something to do with using a protocol buffer library, but Google's own tutorial only shows how to read a whole file at once. I can't do that, because the files I'm handling are too large for memory (think planet.osm.pbf), so I need to do it piece by piece.

How should I approach this? Am I asking in the right place?

asked 01 Oct '21, 11:53

fsaler's gravatar image

fsaler
16335
accept rate: 0%

Source code available on GitHub .