NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

I'd like to extract small xml files of tiled osm data from a pbf file. These xml files should contain data square map boxes. I'd like to have one xml file for each tile that is touched by the original pbf file. I used osmosis to extract one tile, which took 10 minutes for a pbf file of 2GB. This is probably due to the size of the pbf file (and IO speed), and not dependent on the size of my bounding box. Is there a tool that lets me extract lots of bounding boxes at once?

asked 10 Apr '15, 17:51

Marian's gravatar image

Marian
11114
accept rate: 0%

edited 10 Apr '15, 17:54


Yes, the OSM History Splitter which processes history files as well as normal PBFs.

permanent link

answered 10 Apr '15, 18:29

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
82.5k927201273
accept rate: 23%

It doesn't compile on my ubuntu machine... (fails to find boost/utility.hpp)

(10 Apr '15, 21:57) Marian

@Marian Please install libboost-dev.

(13 Apr '15, 08:01) scai ♦

I got it to run but it crashes if there are more than 30 bounding boxes in the config file.

(16 Apr '15, 13:25) Marian

Turns out RAM usage grows quickly with the amount of bounding boxes so only a few bboxes can be extracted at a time.

(18 Apr '15, 12:41) Marian
1

You might want to try the --hardcut option which uses less RAM. Else just run the program multiple times. The problem is (you say you want to produce > 100.000 extracts) that a list of nodes and ways needs to be kept in memory for all these, and most programs will use an array-like structure for efficiency. It might be possible to switch osm-history-splitter to a list-like structure using small changes in the code.

(19 Apr '15, 18:46) Frederik Ramm ♦

You can try osmconvert ... it has several clipping features.

Read the instructions in wiki page carefully.

permanent link

answered 10 Apr '15, 19:26

stephan75's gravatar image

stephan75
12.6k556210
accept rate: 6%

Can you provide an example of how to extract two bboxes from one pbf file using osmconvert?

(10 Apr '15, 19:54) Marian

Did you try the clear example at

http://wiki.openstreetmap.org/wiki/Osmconvert#Clipping_based_on_Longitude_and_Latitude

If you want two bboxes extracted, you should call this command twice with diferent parameters ... Success?

(11 Apr '15, 06:48) stephan75

I have to do this more than 100k times. I was wondering if there is a more efficient way than parsing the entire pbf file each time.

(12 Apr '15, 23:13) Marian

If you want to create a huge number of extracts then perhaps installing a local Overpass database and extracting from there could be the fastest way. See http://overpass-api.de/no_frills.html for instructions.

permanent link

answered 19 Apr '15, 18:47

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
82.5k927201273
accept rate: 23%

Why do you recommend creating a local overpass database over using an existing overpass service? Even in production, 10000 requests per day is way beyond what I need. Is the local db faster?

(10 Nov '15, 23:21) Marian

If you're just doing this once for 5 or 10 bounding boxes then I guess it's ok. Else you come close to abusing the existing Overpass server - the local db may not be faster but it sure is fairer!

(11 Nov '15, 01:26) Frederik Ramm ♦

The wiki page says 10.000 requests are ok. If 10 already is abuse, I'll sure set up my own. https://wiki.openstreetmap.org/wiki/Overpass_API

(11 Nov '15, 06:30) Marian
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×252
×83
×47

question asked: 10 Apr '15, 17:51

question was seen: 6,777 times

last updated: 11 Nov '15, 06:30

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum