Error while extracting merged database for Nominatim. Help!

Question

I merged Europe and North America .os.bz2 files from http://download.geofabrik.de/ using the command osmium merge country1.osm.bz2 country2.osm.bz2 -o together.osm.bz2. Then I started extraction of the merged file using the command ./utils/setup.php --osm-file together.osm.bz2 --all 2>&1 | tee setup.log. After running this command I got the error below:

2020-08-05 01:40:59 == WARNING: resetting threads to 1
2020-08-05 01:40:59 == module path: /srv/nominatim/build/module
2020-08-05 01:40:59 == Create DB
2020-08-05 01:40:59 == Setup DB
Postgres version found: 10
Postgis version found: 2.4
 set_config
------------

(1 row)

2020-08-05 01:41:04 == Import data
osm2pgsql version 1.2.0 (v3.5.0-67-g8201c7f)

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=46413MB, maxblocks=742608*65536, allocation method=11
Mid: pgsql, cache=46413
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Parsing gazetteer style file '/srv/nominatim/Nominatim/settings/import-full.style'.
Using projection SRS 4326 (Latlong)
NOTICE:  table "place" does not exist, skipping

Reading in file: /srv/nominatim/build/europe-north-america.osm.bz2
Using XML parser.
Processing: Node(3824539k 228.0k/s) Way(413495k 49.59k/s) Relation(6198427 5019.0/s)  parse time: 26344s
Node stats: total(3824539161), max(7771142999) in 16771s
Way stats: total(413495830), max(832369740) in 8338s
Relation stats: total(6198427), max(11366925) in 1235s
result COPY END for planet_osm_rels failed: ERROR:  duplicate key value violates unique constraint "planet_osm_rels_pkey"
DETAIL:  Key (id)=(11980) already exists.
CONTEXT:  COPY planet_osm_rels, line 3143

DB copy thread failed: Ending COPY mode
ERROR: No Data
string(7) "No Data"

After this extraction stopped. What could have had caused it and how to solve this issue?

Answer 1

The error message isn't easy to read, but the important detail is this part: "planet_osm_rels failed: ERROR: duplicate key value violates unique constraint". And it even says what the id is: 11980. So presumably the relation 11980 was in together.osm.bz2 twice. This relation is the France relation and France has many overseas departments etc. so it might well be that it was in the Europe and North America extracts.

Now the question is why was it in the merged files twice? The most probably reason is that you downloaded the two original extracts at different points in time and they both contained different versions of the same relation. In that case osmium merge will leave both in there, as documented in the man page: "Do not use this command to merge non-history files with data from different points in time. It will not work correctly." But this is guess work, because I don't know what's in the files you downloaded. You can check this with "osmium getid".

Error while extracting merged database for Nominatim. Help!

Follow this question

Related questions