No space left on device with setup.php in Nominatim

Question

I'm trying to use the north-america-latest.osm.pbf file, around 40GB. I expect a total disk usage of 400(ish) GB.

./utils/setup.php --osm-file /3TB/nominatim/north-america-latest.osm.pbf --all --osm2pgsql-cache 24000 --threads 8 2>&1 | tee setup.log

When running the setup, I get

./utils/setup.php   --osm-file /3TB/nominatim/north-america-latest.osm.pbf   --all --osm2pgsql-cache 24000 --threads 8 2>&1   | tee setup.log
2018-03-16 20:35:16 == Create DB
2018-03-16 20:35:17 == Setup DB
Postgres version found: 9.6
Postgis version found: 2.3
2018-03-16 20:35:19 == WARNING: external UK postcode table not found.
2018-03-16 20:35:20 == Import data
osm2pgsql version 0.93.0-dev (64 bit id space)

Using projection SRS 4326 (Latlong)
NOTICE:  table "place" does not exist, skipping
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=24000MB, maxblocks=384000*65536, allocation method=11
Mid: pgsql, cache=24000
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels

Reading in file: /3TB/nominatim/north-america-latest.osm.pbf
Using PBF parser.
Processing: Node(966941k 786.8k/s) Way(72862k 84.72k/s) Relation(0 0.00/s)COPY_END for COPY planet_osm_ways FROM STDIN;
 failed: ERROR:  could not extend file "base/292375/295704.6": No space left on device
HINT:  Check free disk space.
CONTEXT:  COPY planet_osm_ways, line 15312860

Error occurred, cleaning up
ERROR: Error executing external command: /srv/nominatim/Nominatim-3.1.0/build/osm2pgsql/osm2pgsql -lsc -O gazetteer --hstore --number-processes 1 -C 24000 -P 5432 -d nominatim /3TB/nominatim/north-america-latest.osm.pbf
Error executing external command: /srv/nominatim/Nominatim-3.1.0/build/osm2pgsql/osm2pgsql -lsc -O gazetteer --hstore --number-processes 1 -C 24000 -P 5432 -d nominatim /3TB/nominatim/north-america-latest.osm.pbf

I've already created a default tablesplace on the 3TB drive for PostGre, but somehow my (tiny) root partitions gets filled after 30ish minutes. How can I change this apparent tmp path?

Accepted Answer

It looks like your default tablespace setting was ignored. You can tell Nominatim to explicitly move part or all of the database into a different tablespace through its configuration. To move all data, add something like that to your settings/local/php:

@define('CONST_Tablespace_Osm2pgsql_Data', 'bigTS'); @define('CONST_Tablespace_Osm2pgsql_Index', 'bigTS'); @define('CONST_Tablespace_Place_Data', 'bigTS'); @define('CONST_Tablespace_Place_Index', 'bigTS'); @define('CONST_Tablespace_Address_Data', 'bigTS'); @define('CONST_Tablespace_Address_Index', 'bigTS'); @define('CONST_Tablespace_Search_Data', 'bigTS'); @define('CONST_Tablespace_Search_Index', 'bigTS'); @define('CONST_Tablespace_Aux_Data', 'bigTS'); @define('CONST_Tablespace_Aux_Index', 'bigTS');

where bigTS should be replaced by the name of your tablespace. See settings/defaults.php for more information on these settings.

No space left on device with setup.php in Nominatim

Follow this question

Related questions