This is a static archive of our old OpenStreetMap Help Site. Please post any new questions and answers at community.osm.org.

Scraping Open Street Map for UK street names

1

Hi,

I am looking to compile a list of UK street names for an infographic project. Would it be possible to scrape the OSM for every street name, and output that data into a spreadsheet? If anyone out there can help with this I'd be very grateful.

Thanks,

Chris

asked 18 Apr '12, 11:26

chrishall's gravatar image

chrishall
16111
accept rate: 0%


3 Answers:

4

We reserve the word "scraping" for people who, to our dismay, write clumsy scripts that make tons of individual requests against our API or web site. Don't do that - we're an open data project and we make our data available for download!

Grab a data extract for the UK e.g. from the Geofabrik download server, then use a program like Osmosis to filter out only highways:

osmosis --read-pbf file.osm.pbf --tf accept-ways highway=\* --write-xml myfile.osm

From the resulting XML file, extract all names - easiest on Linux with something like

grep 'k="name"' myfile.osm | cut -d\" -f4

and you have your list. (If you prefer DBF files to XML, you could probably download the shp.zip file from the download server and simply open the roads.dbf file.)

Caveats:

  1. This procedure will yield names for everything tagged "highway", including cycleways, footways, steps, roundabouts.

  2. This procedure does not allow you to count how often each name occurs in reality, because a road may consist of several parts in OSM, so the same road might feature multiple times in your file. Should you want to eliminate such double mentions, some programming or GIS magic will be required.

answered 18 Apr '12, 11:41

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
82.5k927201273
accept rate: 23%

edited 18 Apr '12, 11:49

Frederik,

Thanks for coming back to me. Apologies re. 'scraping', I'm not looking to inconvenience anyone!

The second point you make is probably the most relevant - and thanks for bringing it to my attention. I'm not sure I know how to solve this myself - can you help, or recommend anyone who can? If it's time-consuming work I'm willing to pay for the research/make an appropriate donation.

Many thanks,

Chris

(18 Apr '12, 11:54) chrishall

It seems like the data set mentioned by Richard and Ed would conveniently circumvent this problem!

(18 Apr '12, 11:59) Frederik Ramm ♦

3

It might be better to start with something like OS OpenData, particularly the Locator dataset I think.

http://www.ordnancesurvey.co.uk/oswebsite/products/os-locator/index.html

As yet, OpenStreetMap does not have as comprehensive a coverage as the OS data.

answered 18 Apr '12, 11:44

EdLoach's gravatar image

EdLoach ♦
19.5k16156280
accept rate: 22%

1

OpenStreetMap is arguably not the best data source for your application. You would be better served by using OS Locator, from the Ordnance Survey OpenData release, which has a better licence, a simpler file format, more consistent data, and is more complete.

answered 18 Apr '12, 11:44

Richard's gravatar image

Richard ♦
30.9k44279412
accept rate: 18%

Source code available on GitHub .