|
Greetings again! For reference, here's my previous thread. Newbie here with this software. https://help.openstreetmap.org/questions/86651/third-time-apologies-if-this-is-a-repeat-nominatim-install-woes I was doing so great! I thought I was gonna break some kind of speed record, because this was going to complete in like one day (or so I thought). But it didn't - crashed and burned. (crys) Below is the output...any help? What do I do now? $ nominatim import --osm-file planet-latest.osm 2>&1 | tee setup.log 2023-02-09 16:09:11: Using project directory: /home/data/1941/nom 2023-02-09 16:09:13: Creating database env.defaults gbpostcodes.csv.gz planet-latest.osm setup.log uspostcodes.csv.gz website wikimedia-importance.sql.gz worldfile 2023-02-09 16:09:14: Setting up country tables 2023-02-09 16:09:15: Importing OSM data file 2023-02-09 16:09:15 osm2pgsql version 1.7.1 2023-02-09 16:09:15 Database version: 12.13 (Ubuntu 12.13-0ubuntu0.20.04.1) 2023-02-09 16:09:15 PostGIS version: 3.0 2023-02-09 16:09:16 Parsing gazetteer style file '/usr/local/etc/nominatim/import-extratags.style'. NOTICE: table "place" does not exist, skipping 2023-02-10 11:05:12 Reading input files done in 68156s (18h 55m 56s). 2023-02-10 11:05:12 Processed 8172948111 nodes in 20393s (5h 39m 53s) - 401k/s 2023-02-10 11:05:12 Processed 915512510 ways in 36157s (10h 2m 37s) - 25k/s 2023-02-10 11:05:12 Processed 10604686 relations in 11606s (3h 13m 26s) - 914/s 2023-02-10 11:05:12 Done postprocessing on table 'planetosmnodes' in 0s 2023-02-10 11:05:12 Done postprocessing on table 'planetosmways' in 0s 2023-02-10 11:05:12 Done postprocessing on table 'planetosmrels' in 0s 2023-02-10 11:05:12 osm2pgsql took 68157s (18h 55m 57s) overall. 2023-02-10 11:05:13: Importing wikipedia importance data 2023-02-10 11:07:11: Importing secondary importance raster data 2023-02-10 11:07:11: Secondary importance file not imported. Falling back to default ranking. 2 023-02-10 11:07:11: Create functions (1st pass) 2023-02-10 11:07:11: Create tables 2023-02-10 11:13:59: Create functions (2nd pass) 2023-02-10 11:14:00: Create table triggers 2023-02-10 11:14:00: Create partition tables 2023-02-10 11:14:01: Create functions (3rd pass) 2023-02-10 11:14:01: Initialise tables 2023-02-10 11:14:01: Load data into placex table Traceback (most recent call last): File "/usr/local/bin/nominatim", line 14, in <module> exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module', File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 264, in nominatim return parser.run(**kwargs) File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 126, in run return args.command.run(args) File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/setup.py", line 121, in run database_import.load_data(args.config.get_libpq_dsn(), num_threads) File "/usr/local/lib/nominatim/lib-python/nominatim/tools/database_import.py", line 195, in load_data conn = DBConnection(dsn) File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", line 74, in init self.connect(cursor_factory=cursor_factory) File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", line 99, in connect self.wait() File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", line 128, in wait wait_select(self.conn) File "/home/jpatterson/anaconda3/envs/jpconda/lib/python3.8/site-packages/psycopg2/extras.py", line 758, in wait_select state = conn.poll() psycopg2.OperationalError: FATAL: sorry, too many clients already |
|
To clarify, I did the recommended psql tweaks recommended here: https://nominatim.org/release-docs/latest/admin/Installation/ shared_buffers = 2GB maintenance_work_mem = (10GB) autovacuum_work_mem = 2GB work_mem = (50MB) effective_cache_size = (24GB) synchronous_commit = off max_wal_size = 1GB checkpoint_timeout = 10min checkpoint_completion_target = 0.9 But i found a reference here to increase max_connections. I could try that...but do I just restart the import after that? |
|
SO, I increased this: max_connections=500 Restarted the DB server, and tried this: /home/data/1941/nom$ nominatim add-data --file planet-latest.osm And it died... 2023-02-10 14:58:08: Using project directory: /home/data/1941/nom 2023-02-10 14:58:08 osm2pgsql version 1.7.1 2023-02-10 14:58:08 Database version: 12.13 (Ubuntu 12.13-0ubuntu0.20.04.1) 2023-02-10 14:58:08 PostGIS version: 3.0 2023-02-10 14:58:09 Parsing gazetteer style file '/usr/local/etc/nominatim/import-extratags.style'. Processing: Node(1210k 1.2k/s) Way(0k 0.00k/s) Relation(0 0.0/s) 2023-02-10 15:15:20 ERROR: DB copy thread failed: Ending COPY mode for 'place' failed: ERROR: relation "place_to_be_deleted" does not exist LINE 1: DELETE FROM place_to_be_deleted pdel ^ QUERY: DELETE FROM place_to_be_deleted pdel WHERE pdel.osm_type = NEW.osm_type and pdel.osm_id = NEW.osm_id and pdel.class = NEW.class CONTEXT: PL/pgSQL function place_insert() line 25 at SQL statement COPY place, line 1: "1 N man_made mast "name"=>"Monte Piselli - San Giacomo" 15 \N "communication:microwave"=>"yes","comm..." . Traceback (most recent call last): File "/usr/local/bin/nominatim", line 14, in <module> exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module', File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 264, in nominatim return parser.run(**kwargs) File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 126, in run return args.command.run(args) File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/add_data.py", line 79, in run return add_osm_data.add_data_from_file(cast(str, args.file or args.diff), File "/usr/local/lib/nominatim/lib-python/nominatim/tools/add_osm_data.py", line 25, in add_data_from_file run_osm2pgsql(options) File "/usr/local/lib/nominatim/lib-python/nominatim/tools/exec_utils.py", line 159, in run_osm2pgsql subprocess.run(cmd, cwd=options.get('cwd', '.'), File "/home/jpatterson/anaconda3/envs/jpconda/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/local/lib/nominatim/osm2pgsql', '--hstore', '--latlon', '--slim', '--log-progress', 'true', '--number-processes', '1', '--cache', '1000', '--style', '/usr/local/etc/nominatim/import-extratags.style', '--output', 'gazetteer', '--append', '--flat-nodes', '/home/data/1941/nom/worldfile', '--with-forward-dependencies', 'false', 'planet-latest.osm']' returned non-zero exit status 2. NO love? |
|
"FATAL: sorry, too many clients already" Increasing the Recovering from that doesn't seem to work. I'd recommend deleting the whole database again and use When you see lines like |
Also my posts do not seem to be formatting correctly. Am I posting things wrong? I'm sorry I don't know why the hard returns are not carrying over.
Thisforum understands Markdown, so indent e.g. server errror logs by 4 spaces.
Also, when editing xml text what you see in the preview does not match what you get on the page!