i have a extremely large xml-file - which is derived from the field of geo informatics. i got it from a German subsite or the OpenStreetMap-Project: the Geograpical-Engineering-site that deilvers a weekly snapshot of OpenStreetMap of a certain area: i took the germany.osm.bz2 from here http://ftp5.gwdg.de/pub/misc/openstreetmap/download.geofabrik.de/ For doing some tests with xslt i want to run a request to find out certain entity - let us take for example the restaurants. we want to find out all the restaurants in the area. now we can run that directly on the bz2 compressed file, that we downloaded - for example if we use the following code:
well i splitted the file with xml_split -which is a great perl-module from CPAN. The problem: with the following xslt-processor i get only bad results - the parsed files werent not parsed enough i only get a minor set of informations when i run the code on a xml-file. see the xslt-processor - and below - a litte data-chunk out of the file i run and parse if you want to check it - just get the little dataset - note it is a splitted file here you can get it: https://rapidshare.com/#!download|643p12|2523227518|germany-001.xml|100000 Note: see therefore the important lines:
Note- you can run a little test - and see how long it takes to parse time xsltproc restaurants.xslt germany-001.xml > restaurants-001.csv
here we have the xslt-processor that contains the code for parsing...:
and here below we have a data-chunk out of the xml-file that we have parsed: see it
see the results - note there are missing some parts - unfortunatly..
well it is somewhat a problem that i get the results - ive tried alot but at the moment i am glueless why i get the little output - that is totally contrary to the tags i have in the xslt -processor - any idea and hint will be greatly appreciatdd btw: after all i want to run approx 5000 files that are the result of the split - and subsequently i want to collect all the results in a mysql-database... here you can get the original-file: http://ftp5.gwdg.de/pub/misc/openstreetmap/download.geofabrik.de ( germany.osm.bz2 01-Apr-2012 14:51 1.7G ) and here a splitted one: https://rapidshare.com/#!download|643p12|2523227518|germany-001.xml|100000 i have to refactor the coed -so the question - is - how can i get the mysql-results on a efficient way? update:thx to the first answer in this thread i startet to refactor the code - but still lack of some better results. i have to retry it again..lots of changes were suggested - i did a quick walktrough on the xslt-parser: with the first trial of refactoring i got some funny results. But i will try again - i go trough all the xslt-processor-code and have a closer look if i find the errors and finally i try to refactor all the xslt-file. - any pointers and subbestions or code-snippets are greatly wellcome. Well to me it looks like our we should consider somewhat changing the context node to make this code clearer and avoid errors like this:
Then we can use XPaths like select="tag/@id" But above all we should consider refactoring this code to make better use of what do you think - any pointer towards the refactoring is grealy appreciated asked 24 Apr '12, 16:33 tagtheworld |
This is not an XSLT help desk. Your problem is not related to OSM except that you use OSM test data; it would probably be better to discuss these things in a forum dedicated to XML processing in general. You write that eventually you want the data in a MySQL database. Is there any reason why you don't just bin the whole XSLT stuff and import the data into MySQL using Osmosis or something like https://github.com/skyebook/OSMGenerator? answered 24 Apr '12, 19:49 Frederik Ramm ♦ hello frederik thx for the answer - well it would be very interesting to import the whole stuff into a mysql-db. at which stage can we do a import into mysql!? At a very early stage of the data-processing,,,,
(24 Apr '12, 20:21)
tagtheworld
hello after a quick look at the options of importing i muse bout the db-shemes. Is it possible to do certain requests on the data - if they are imported into mysql - i heard about the problem of consistency and such issues . i will dig deeper into the issues - come back and report all the findings. greeting
(24 Apr '12, 20:41)
tagtheworld
|