Hello, I am asking what I realize is technical question about shapefiles rather than OSM but I hope you will excuse me. For what it is worth, I am a newbie but really think I will be involved in OSM for a while, so it won't be a lost cause to help me: I think I will contribute back. Anyway: I have downloaded shapefiles from both cloudmade and geofabrik. I have also downloaded OSM files and converted them. My problem: all the shapefiles have different dbase structures. For example, Cloudmade is nice and neat with four columns and missing a lot of tag information. QGIS saves shapefiles with anywhere from four columns to twenty columns so far, with the tags column having a variety of information such as "bicycle"="yes","hiking"="yes","information"="map","internet_access"="terminal". Some of this information is duplicated in a separate column: e.g. tags list parking/fees/surface... and the column amenity lists 'parking'. This means that I have to parse this data somewhere else. Which means that I not only have to write a parser (ot at least a macro) but it would have to deal with unknowns each time. What is wierd is that the program I am using, NetLogo (for the moment), is oddly sucessful in dealing with some of the data - unfortunatly I don't have details on how this data is handled in order to solve it. However I would like to move forward having the data side solved and then deal with issues on the programming side. So what I am looking for is:
I realize this is only partially a OSM question, but any help would be appreciated. Thanks asked 13 Mar '12, 11:50 jake cimilo |
The shapefile format has a lot of limitations, so it is not possible to create a shapefile with all the information from OSM, so when you convert you have to decide what to include. You don't describe what data you actually need, so it is hard to say for sure what is the best solution for you. If you need more than just simple shapes, you might be better off importing the planet file (or one of the extracts from Geofabrik or Cloudmade) into a PostGIS database. Again depending on your needs, there are several ways to do that. Some of the options are: Osm2pgsql, Imposm and Osmosis With Osmosis, PostgreSQL and PostGIS you can do something like this: Create a PostGIS and hstore enabled database and setup the pgsnapshot schema, use Osmosis to import the data:
Then create and index new qid columns to make QGIS happy:
When you use the pgsnapshot schema, all the tags are stored in a hstore column. To make it easier to work with from QGIS, you can create views like this:
Then in QGIS, you can add the Amenities and Walkable views as PostGIS layers. You should change the views to use your own classifications. You can also use osm2pgsql and change the import style to create a schema that matches your needs. There are no fixed rules for how anything should be tagged in OSM, so your biggest task is probably identifying what tag/value combinations you should use to make your classifications. You can start by looking at some of these keys and especially their link to taginfo on the right side of the page - the "values" tab can give you an idea about values you should consider: highway, foot, bicycle, tracktype, surface, access, mtb:scale and sac_scale. /Jais answered 13 Mar '12, 12:37 Dymo12 I would like to do that. Eventually I will get to the point where I am storing the various layers I need, including the ones I generate, in an external database. For the moment, I wanted the .dbf file to hold just certain layers. For example, I would like to have separate shapefiles with:
To take the first example, bikeable roads have a tag: "bicycle"="yes". All the .dbf files I have right now just list that tag in a text record with all the other tags that apply to that feature. If the tags were separated or if I could download them in a way that the .dbf file showed a column "bicycle" and a column "level" (for 1 to 5 or which ever standard is used, then I could sort the roads using those columns, e.g. "highways" with "bicycle"="yes" and "level"<"3". I admit that my reason is just that my system has built in ways to do this which I prefer over writing my own script. But if I am running hundreds of agents and hundreds of scenarios, I prefer having the data presorted... aye, which i guess is what i will have to do. Still, if you know an answer without me having to code something, I would appreciate it. Thanks
(13 Mar '12, 13:17)
jake cimilo
Are you aware that QGIS can load OSM files directly (through its OSM plugin) ? This may cause problems if your data use multipolygons but would be enough for simple OSM elements (ways, nodes).
(13 Mar '12, 14:55)
Pieren
I think it is difficult to get the level of detail you want without some coding/scripting, but it is not that hard and it also makes it easier for you to keep you data updated. I have added some examples to my answer.
(13 Mar '12, 20:12)
Dymo12
Dymo12 - thanks. I have a lot of sorting out of my project to do before I attempt any of your suggestions - the guts of it is cognitive models of people and OSM is just the playground they will eventually use. For the moment, I can 'fake' a playground without sorting the OSM data and i need to do further work on the cogntiive model rather than the playground. However, I will come back to this because I really like OSM, I even like that "There are no fixed rules for how anything should be tagged in OSM" - I much rpefer a flexible system even at the cost of having to learn how to code. When I get back to it, I will post what happened and how I succeeded. : )
(14 Mar '12, 09:20)
jake cimilo
|