Hi!

I'm working on my bachelor thesis on the topic of how urban areas across the world cluster based on their walkable urban forms and the presence of greenery. For this, I need a large dataset of cities/towns/villages around the planet paired with their country and longitude and latitude values in the .csv format. I've downloaded the north-america-latest.osm.pbf, and I've been trying to extract data with my desired fields out of it for three days with osmfilter and osmconvert, but I'm just not having any luck. I first tried to extract only towns from north-america-latest.osm.pbf with >osmfilter north-america-latest.osm_01.o5m --keep="place=town" --drop-version --ignore-dependencies --drop-relations --drop-ways -o=north-america-cities.osm< and adding the longitude and latitude fields with osmconvert. This worked, but only for towns and I have no idea how to add the countries to it and I can't really find any resources about how to go about this, the wiki isn't really helpful here to a beginner. It seems to be that what I want, isn't really all that complex, so it shouldn't be this hard?

I would really really appreciate any advice and help.

Thank you!

asked 07 Nov '20, 15:11

Tian's gravatar image

Tian
11112
accept rate: 0%


The pbf format stores the OpenStreetMap data in a way that is convenient for multi-user, versioned editing. The country usually isn't attached to the item that represents a city or town (there's not restriction, so it might be). Country boundaries are usually present in the data (but the process used to create a country file might omit them).

The solution is to pass the location to a country level geocoder. I don't know if there is one that is particularly easy to install; for thousands of points a local instance is probably better than trying to run them through a free service.

permanent link

answered 07 Nov '20, 18:27

maxerickson's gravatar image

maxerickson
12.5k1079175
accept rate: 31%

Thank you for responding so promptly! I will look into some python geocoders then. Other than that, what is the best way to query for cities, towns and villages? Is it via addr or place? Thank you!

(07 Nov '20, 21:58) Tian

The place tag. It's probably a worthwhile exercise to examine the values used in some areas you are familiar with, in case you are interested in more than village/town/city.

(07 Nov '20, 23:42) maxerickson

Great, thank you! I've managed to extract the data for north-america-latest.osm.pbf after converting it to 'osm.o5m' format and then using osmfilter, but following the exact same steps with south-america-latest.osm.pbf, osmfilter gives the warning 'Warning: unexpected end of input file: south-america-latest.osm_01.o5m', which causes a write error when using osmconvert to produce a .csv file. I've used the osmconBert to convert the formats from pbf to o5m, so I don't know how this end of input file could be happening, do you know by any chance?

(08 Nov '20, 23:08) Tian
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×56
×46

question asked: 07 Nov '20, 15:11

question was seen: 303 times

last updated: 08 Nov '20, 23:08

powered by OSQA