How to read a .pbf file in Python (without reading the whole file at once)?

I need to parse a .pbf file in Python, but using apply_file() on PyOsmium handlers is not working well for me and so I want to use apply_buffer(). I don't quite know what format is expected from it and documentation on it is lacklustre, but I assume it's to be a binary string encompassing a fileblock.

The problem is that if I open() a file in Python I don't know of a way to find fileblocks to feed them to PyOsmium.

It might have something to do with using a protocol buffer library, but Google's own tutorial only shows how to read a whole file at once. I can't do that, because the files I'm handling are too large for memory (think planet.osm.pbf), so I need to do it piece by piece.

How should I approach this? Am I asking in the right place?

asked 01 Oct '21, 11:53

fsaler
16●3●3●5
accept rate: 0%

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

*italic* or _italic_
**bold** or __bold__
link:[text](http://url.com/ "title")
image?![alt text](/path/img.jpg "title")
numbered list: 1. Foo 2. Bar
to add a line break simply add two spaces to where you would like the new line to be.
basic HTML tags are also supported

learn more about Markdown

Question tags:

python ×78
pbf ×60
planet ×56
parsing ×19
osm.pbf ×12

question asked: 01 Oct '21, 11:53

question was seen: 1,651 times

last updated: 01 Oct '21, 11:53

How to read a .pbf file in Python (without reading the whole file at once)?

Follow this question

Related questions