1
1

I downloaded a world PBF file and going to use osm2pgsql. I'm only interested in a small subset of data:

  • administrative boundaries (type=boundary&boundary=administrative)
  • all kind of cities, towns, villages etc (place=...) - I actually just need their coordinates/polygons, names and populations

Is there a way to configure osm2pgsql in such a way that only those nodes, ways and polygons are created in database which make up administrative boundaries and city/towns/etc with their respective names, populations and coordinates?

In my understand, I should:

  • not use any hstore-related command line options when importing
  • remove everything from the default.style file and only list attributes that I'm interested in:
    • boundary:administrative (how do I only import objects with attributes containing a particular value?)
    • type:boundary
    • place
    • name
    • population
    • population:date
    • source:population

Will this make the osm2pgsql tool to only store objects that I need (and dependent objects) and discard any other objects?

By dependent objects I mean, for example, nodes that make up a a country boundary. The nodes won't have the attributes listed above, but they are still needed for the boundaries. How do I import these, but not the others?

Am I taking the right course of action, or should I do it in a different way?

The final goal is to create a database with minimum amount of data that I can query for simple geocoding by place name, and for rendering of administrative map with administrative boundaries only and no other map features. Does it make any sense?

I'm not concerned about the time it will take to import, i.e. it makes no difference for me whether it will be 2 hours or 24 hours. The only concern is the final amount of data stored in the database, which I would like to reduce as much as possible.

asked 02 Jul '17, 06:06

meglio's gravatar image

meglio
40337
accept rate: 0%

edited 02 Jul '17, 06:11

Hi @meglio, did you find a way to accomplish this? I'm looking to import only a fraction of all the features too.

(30 Oct, 17:24) pierrebonbon

Unfortunately, no.

(31 Oct, 01:23) meglio

@meglio that's a bummer, was hoping that there's was some way to make the import leaner. :-(

(05 Nov, 12:05) pierrebonbon

osm2pgsql manages two sets of tables. The first set, the geoemtry tables, consists of planet_osm_polygon, planet_osm_line, planet_osm_roads, and planet_osm_point. These tables will contain the geometries you are interested in, and because they already contain ready-made geometries, no dependencies on other objects exist. Using the procedure you have outlined you will end up with a fairly minimal data set in these tables, although - and I believe this depends on the osm2pgsql version you use - there might be some duplication with both boundary lines and boundary polygons being generated.

The second set of tables, the slim tables, contain more or less raw OSM objects, in the adequatly named planet_osm_nodes, planet_osm_ways, and planet_osm_rels tables. These tables are only created if you import with --slim, and they will contain nearly all OSM objects, independent of your style file. That's several hundred GB in the case of a full planet import. Using --slim is definitely required if you want to import incremental updates, and even when you don't, --slim reduces RAM usage during the initial import. If you only use --slime to reduce RAM usage, combine with --drop to drop the slim tables after the import.

Long story short, you can reduce the total database size significantly using the process you outline, but only if don't need incremental updates.

permanent link

answered 02 Jul '17, 09:26

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
66.0k806101022
accept rate: 24%

@frederik-ramm "no dependencies on other objects exist" - this is a gotcha moment, many thanks! If I understand correctly, slim tables are populated first as a normalized version of the data file, then geometry tables are populated out of those. If I need to experiment with osm2pgsql and let it populate various content for the geometry tables, is there a way to skip production of the slim tables on every run, and let the tool reuse the existing slim tables instead?

(03 Jul '17, 08:21) meglio

@frederik, may you please also clarify is my understanding is correct: whatever columns I specify in the .style file, if all values are null/empty for a particular object, the object is discarded and will not be present in the geometry tables. If so, how do I save names of cities, but discard all other objects that contain a name attribute?

(03 Jul '17, 09:52) meglio

You could consider using osmfilter and osmconvert to preprocess the .osm.pbf file even before it gets to osm2pgsql.

permanent link

answered 02 Jul '17, 22:03

Richard's gravatar image

Richard ♦
26.8k38237354
accept rate: 20%

Thanks @Richard. Should I consider --ignore-dependencies if I only need my geometries of my target objects and no of separate nodes / ways from which the former consist?

(03 Jul '17, 10:54) meglio
1

I would not advise using the --ignore-dependencies option because you will need nodes as they are of the only OSM object type which holds coordinates. Here is an example how to prefilter OSM data and even update the database with prefiltered data: https://wiki.openstreetmap.org/wiki/Openptmap/Installation#Fill_the_Database

(08 Jul '17, 20:36) Marqqs
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×178

question asked: 02 Jul '17, 06:06

question was seen: 1,177 times

last updated: 05 Nov, 19:32

powered by OSQA