Fastest way to query an OSM file

I am writing a code in Python to find bus stops inside of different rectangles that depend on incoming data.I am able to this with the overpass Api and the python module overpy but query time is usually around 5 seconds and I would like to have it under 1 second. Moreover, the limit of queries is reached. My second attempt was to save a local osm copy and to cut the area of interest with osmosis, however, trimming the area takes nearly 2 minutes. What is the fastest way to accomplish this ?

asked 02 Mar '21, 17:38

fragodec
16●2●2●5
accept rate: 0%

edited 02 Mar '21, 17:44

One Answer:

active answers oldest answers newest answers popular answers

The command line tool Osmium can do all sorts of things like creating extracts. It is much faster than Osmosis. I recommend the following way of processing OSM data:

Download a suitable extract from Geofabrik
Use osmium extract to narrow this down to a smaller area if needed
Use osmum tags-filter to filter out roughly all the subject data you might need
Use your own Python script (possibly based on PyOsmium) to do any specialized processing you need.

Python code will be slower than the optimized C++ code of the Osmium command line tool, so it makes sense to do pre-filtering with the faster tool (step 2/3) and then use Python for the rest.

permanent link

answered 02 Mar '21, 20:46

Jochen Topf
5.2k●5●50●74
accept rate: 31%

Thank you for your detailed answer. Two-follow up questions:

I did some research and it seems quite difficult to install Osmium in Windows, is there a more or less straight-forward way of doing so?
The rectangles defining the current areas of interest are themselves generated in real-time with a Python code. Ideally I would like the code to call Osmium, do the extraction there and then recover the smaller osm file from Python. Is there a way to do this?

(03 Mar '21, 07:55) fragodec

to 1) No, unfortunately there isn't an easy way to install Osmium on Windows. You might be able to use WSL for this.

to 2) There is no special way for doing this, but you can run any program from Python with os.system(), so you can just build the right command line and run Osmium from it. You can run osmium extract with a bounding box on the command line or use a config file with many bounding boxes in it to efficiently create many extracts. So in your case it might make sense to create that config file from your Python script and then run osmium extract once.

(03 Mar '21, 08:33) Jochen Topf

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

*italic* or _italic_
**bold** or __bold__
link:[text](http://url.com/ "title")
image?![alt text](/path/img.jpg "title")
numbered list: 1. Foo 2. Bar
to add a line break simply add two spaces to where you would like the new line to be.
basic HTML tags are also supported

learn more about Markdown

Question tags:

query ×147
postgis ×134
python ×78
busstops ×39
optimize ×2

question asked: 02 Mar '21, 17:38

question was seen: 3,585 times

last updated: 03 Mar '21, 08:33

Fastest way to query an OSM file

Follow this question

Related questions