Hi!

I succeeded to import the OSM data for Netherlands (by the way, took 38 hours to finish) and now I want also to import (in the same database) the OSM data for Belgium and Luxembourg.

Do you know how can be done? I have tried using ./utils/setup.php --osm-file belgium.pbf --import-data , but didn’t work (stopped at some point: Setting up table: planet_osm_ways) Should I use ./utils/setup.php --osm-file belgium.pbf --load-data?

Thank you!

Best regards, Roxana

asked 25 Aug '12, 09:31

RoxanaO's gravatar image

RoxanaO
31337
accept rate: 0%

edited 24 Jan '17, 22:15

aseerel4c26's gravatar image

aseerel4c26 ♦
31.9k16237548

@lonvia Unfortunately didn't work! Here is the result:

/var/www/public_html/Nominatim/osmosis-0.40.1/bin/osmosis --read-xml 'belgium.pbf' --read-empty --derive-change --write-xml-change /var/www/public_html/Nominatim/data/osmosischange.osc ... org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion SEVERE: Thread for task 1-read-xml failed org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to parse xml file belgium.pbf.

@Cartinus Thank you for the link! It will help me, but I wanted to know how to do if I want to import countries one by one.

(27 Aug '12, 09:15) RoxanaO

You are using the wrong Osmosis flag, use --read-pbf, not --read-xml, see http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#PBF_Binary_Tasks (I have removed superfluous parts of above error message).

(07 Jan '15, 12:43) SK53 ♦

Hello, I have the same problem with you. And I use the following statement to import the first country data:

./utils/setup.php --osm-file countryA.bz2

However, it shows "countryA.bz2" not readable.

Do you know Why such problems have appeared?

(23 Dec '16, 08:31) PasserbyAmen

Does the file countryA.bz2 exist? Is it located in your current directory? Is it readable by your user?

(23 Dec '16, 18:04) scai ♦

I assume you are talking about importing into a Nominatim DB.

In theory, you could do that by using the update script:

./utils/update.php --import-file belgium.xml.bz2

(pbf files will not work here) and then reindex your data:

./utils/update.php --index

But that will be very, very slow.

The import will go much faster if you first merge the three country excerpts and then import the result in one go.

permanent link

answered 25 Aug '12, 10:47

lonvia's gravatar image

lonvia
5.1k14976
accept rate: 38%

edited 11 Oct '12, 08:48

2

In the case of these three countries, there is even an excerpt available that already contains all three of them: planet-benelux

(25 Aug '12, 12:52) cartinus

I tried also to import the merged made with the country excerpts and got a fatal error and stopped:

COPY_END for COPY planet_osm_rels FROM STDIN; failed: ERROR: duplicate key value violates unique constraint "planet_osm_rels_pkey"

And tried again with ./utils/update.php --import-file belgium.pbf but still have java errors from above.

Any other ideas? Sorry for bother and thank you!

(28 Aug '12, 13:09) RoxanaO

Ages ago someone posted a patch for osm2pgsql to modify instead of create data. If you really must merge data, you could try something like that - I'm sure that it'd be much slower than importing the data that cartinus linked to, though.

(28 Aug '12, 13:21) SomeoneElse ♦

Looks like it cannot read pdf format. Try with the belgium.xml.bz2 file.

(28 Aug '12, 19:37) lonvia

I have tried with belgium.osm (extracted from belgium.osm.bz2) and worked, but when I want to search (on local website) Belgium as a text: No results found. If I search after coordinates, for example: [51.4494967087473,4.46587880577936] should get: Oude Baan, Schildershof, Essen, Antwerpen, Antwerp, Flanders, 2910, Belgium, but instead I get only: Oude Baan, Belgium.

Do you know why is this happening?

(29 Aug '12, 07:18) RoxanaO

Yes, sorry, forgot to mention that you also have to reindex your data. I've added that step to my original answer.

(29 Aug '12, 07:30) lonvia

If you use osmosis to do the merge, it will merge the files correctly when there are duplicate objects in the same files.

(23 Dec '12, 00:04) smsm1

Solution works, but you can speed up the process by using osm2pgsql cahce like this(12G) ./utils/update.php --import-file ../azerbaijan-latest.osm.bz2 --osm2pgsql-cache 12000 And when indexing you can use threads aka index instances depending on your cpus count: ./utils/update.php –index --index-instances 4

(07 Jan '15, 07:57) ttrumm

I approve that I had successfully imported the missing osm file into the existing nominatim database (europe+asia). I took an osm.bz2 file from geofabrik, unpacked it to an osm (to speed things up) and ran --import-file. It took me very reasonable 2 days on 4 core 5GB RAM VM. Next step will be indexing.

(23 Oct '17, 15:16) sunsetjunks

When i try to import a little pbf or osm file, this error appears:

Osm2pgsql failed due to ERROR: Flatnode store cannot save negative IDs.

(18 Apr '18, 22:19) cybercoder

Please ask a new question.

(19 Apr '18, 07:53) scai ♦
showing 5 of 11 show 6 more comments

Hi, the easy way is to do the following:

  1. cd /yourNominatimFolder/build;
  2. mkdir data; cd data;
  3. Download your pbf files (http://download.geofabrik.de) wget http://download.geofabrik.de/south-america/peru-latest.osm.pbf wget http://download.geofabrik.de/south-america/chile-latest.osm.pbf wget http://download.geofabrik.de/south-america/colombia-latest.osm.pbf wget http://download.geofabrik.de/south-america/mexico-latest.osm.pbf
  4. apt-get install osmctools;
  5. Convert your PBF, so you can merge all of them in one file: osmconvert peru-latest.osm.pbf -o=peru.o5m; osmconvert colombia-latest.osm.pbf -o=colombia.o5m; osmconvert chile-latest.osm.pbf -o=chile.o5m; osmconvert peru.o5m colombia.o5m chile.o5m -o=allcountries.o5m; osmconvert allcountries.o5m -o=allcountries.osm.pbf;
  6. cd /yourNominatimFolder/build;
  7. ./utils/setup.php --osm-file data/allcountries.osm.pbf --all --osm2pgsql-cache 8000 2>&1 | tee setup.log
  8. Take a coffee. :)

If you have questions or need support, you can contact me: sistemas@aguilacontrol.com

permanent link

answered 04 Jul '16, 22:15

aguilacontrol's gravatar image

aguilacontrol
1
accept rate: 0%

Step 3:

Step 5:

  • osmconvert peru-latest.osm.pbf-o=peru.o5m;
  • osmconvert colombia-latest.osm.pbf -o=colombia.o5m;
  • osmconvert chile-latest.osm.pbf -o=chile.o5m;
  • osmconvert peru.o5m colombia.o5m chile.o5m -o=allcountries.o5m;
  • osmconvert allcountries.o5m -o=allcountries.osm.pbf;
(04 Jul '16, 22:19) aguilacontrol

Hi aguilacontrol, thank you for the very clear instructions. I'm currently in the process of the last line of point 5. After point 5 succeeds and I want to update the Nominatim database, is your recommendation to do (from the wiki)

rm settings/configuration.txt

./utils/setup.php --osmosis-init

./utils/setup.php --create-functions --enable-diff-updates

./utils/update.php --import-osmosis-all --no-npi

and then have coffee, or do something else?

(07 Jan '17, 14:14) OttoH
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×548
×193
×164
×3

question asked: 25 Aug '12, 09:31

question was seen: 22,978 times

last updated: 19 Apr '18, 07:53

powered by OSQA