This is a static archive of our old OpenStreetMap Help Site. Please post any new questions and answers at community.osm.org.

Many questions, XML-definition,pbf or xml, extract parts from planet, coastlines, additional data beside osm-data

-3

Hi!
I'm about to write my own renderer.
But there are a few things I don't find myself or that aren't clear to me.

So I have a couple of questions, some easy, some maybe not.

 1. I used on http://www.openstreetmap.org the "Export" link to get a file of "OpenStreetMap-XML"-Data of my area. The area was around 1km x 1km.
Where do I find the official definition for this XML? Which tag-names do exist?
?xml - osm, bounds, node, tag, nd
(In my following questions I might refer to them as tag-objects.)

 2. For example there were these nd ref="[somenumber]" -tags, all contained in some way-objects.
So I just assume the number in the ref-attribute should reference some other tag-object in this xml-file. But it's not like that.
The numbers are either unique or sometimes happen to be in another nd-object used there as ref also.
Is that a bug in the export function? Or a feature? :P 

 3. For general purposes of course I want to extract those datas related to a selected area myself out of the planet-file.
Assuming I use a xml-planet file: Do I have to look for all node-objects (A) with latitude and longitude inside my area, and then in a second run I would have to look for all tags which contain directly or indirectly aforementionened "ref"-attributes that point to those nodeobjects (A)? Do I have to do it a couple of times, in case there are "ref"-attributes pointing to another way-object?

 4. Though I would much prefer to work with a planet file in xml-format, I would switch to pbf-format, if it is much faster in the processing *and* if pbf is definitly state of art in the OSM-project.
Are the current planet files xml, or already pbf? Or is the pbf idea already outdated again?
Assuming I have a planet file in xml and one in pbf, will they both hold the same data, will one hold a subgroup of the other, or may both contain data that isn't in the other? - Or in short: What should be taken as the official planet file?
Assume I have to download the pbf file to get the "official" planet data: Which program to use to convert it all into an xml file?

 5. About coastline files: Seeing there are coastline files, does that mean that the coastlines are not included as tag-objects in the planet file, or are they only implicitly included, or are the coastline files just some sort of handy service for map-makers?
If they are only implicitly included: Is it difficult to generate those coastlines, so that I just should use the coastline files provided here, or is it fairly simply (technically and in CPU-usage) to generate the coastlines myself?

 6. Is it advisable to use the data from http://www.naturalearthdata.com? As far as I have read through the wiki-pages, mapnik uses them, and mapnik is used by openstreetmap.org themselves.
Do I need or is it advisable to use http://www.mapability.com/info/vmap1_index.html ? Are there any advantages using that?

 7. Last but not least: Should I better use the OSM forums (http://forum.openstreetmap.org/index.php - no link on the homepage) or the help-section (https://help.openstreetmap.org - prominent link on the homepage) for questions? If it's ok I would prefer this help-section. :) In the future I will probably try the forum, to avoid the automatic styles being applied here. 

I'm sure I won't run out of questions. :P

Greetings
WS

xml extraction pbf coastlines additional-data

asked 23 Oct '12, 15:43

WhyStand2
8●1●1●3
accept rate: 0%

What you should definitely do is to ask one question at a time - nobody is going to write an answer that covers all these questions.

(23 Oct '12, 16:05) TomH ♦♦

I tried, but it took from before you added your comment ;)

(23 Oct '12, 16:26) EdLoach ♦

3 Answers:

Many of the answers I think are on the wiki.

Does OSM XML cover things?
The <nd ref="292403538"/> lines should be part of a <way> definition. A way is a sorted list of nodes, and the way can (probably will) have tags with key (k) and value (v). Nearer the start of the file you should find a <node> definition for the node with the relevant reference id, which will also have lat and lon values, version details, and may have tags with k/v properties as well.
You might find there is an extract for your area of interest already available. If not look at Osmosis which can cut an area from a larger extract.
If they are generated at the same time they should hold the same information. I understand working with PBF for many uses is faster, but I think usually when rendering the data is imported into a spatial database rather than being processed directly from the OSM data (whether XML or PBF).
I think the Coastline files are something Mapnik uses to speed things up. See the bit about the coastline error checker. Ah, and the bit about Rendering on this old edit of the coastline page.
I think VMAP1 is for vertical elevations. If so then the OSM Mapnik doesn't use it, but if you are writing your own then you might want to use it, or maybe SRTM. The cyclemap includes contours and hill shading for example and this isn't from the OSM dataset.
Tricky. You might want to also look at the mailing lists and irc channels. My preferences are for email and irc, but also watch the help.osm.org rss feed. The forums as well take up too much of my time so I visit those rarely. Other people are active there though.

answered 23 Oct '12, 16:26

EdLoach ♦
19.5k●16●156●280
accept rate: 22%

Hi! And thank you for the answer. I will ask the remaining questions again in single threads. Might be better. =) 1. I found that, and now I found the links to the definitions on that page, too. 2. That's how I imagined it should be. The exported file from the openstreetmap.org export function wasn't like that. ): 3. I hope the definitions will enable me to write my own extractor. 4. True. 5. I will read through. 6. Ok. 7. =) I'll give this here a second try. Greetings WS

(23 Oct '12, 17:05) WhyStand2

(a slightly tangential answer)

What I'd suggest first is have a go at mapping something local to you and adding it to OSM (using either Potlatch in "advanced" mode so that you can see the tags and node numbers, or JOSM). Then have a go with something that renders the data (not necessarily a full rendering database). That way you'll get familiar with the way the data is connected, and answers to questions like "what's an 'nd' in a way definition" will reveal themselves.

On the last point, the questions that you're asking might fit here (though better one at a time!) or on the "dev" mailing list, or (with some context in a pastebin that you can refer to) one of the IRC channels. There is a "Development" area of the forum but it's relatively low traffic.

answered 23 Oct '12, 20:43

SomeoneElse ♦
36.9k●71●370●866
accept rate: 16%

For me (loading the full planet) working with PBF files is faster not because of the file format "per se", but because they do not require to be uncompressed first (either to a separate, 330GB XML file or a memory buffer). It seems that the "official" format is now PBF, at least for older versions of the planet file.

answered 24 Oct '12, 14:22

MCPicoli
2.2k●13●30●47
accept rate: 24%