|
We are facing a problem with a closed connection with osm2pgsql when performing the daily update of the database from "download.geofabrik.de", and the region in question is Brazil. We have three environments with the same settings (CPU, memory and disk), but the problem only occurs in one of them, which is stage, dev and pro work well. Command: osm2pgsql -a --slim -e13-20 -d dbhom --number-processes 4 --cache 3042 --tag-transform-script /opt/programs/openstreetmap-carto/5.3.1/hom_openstreetmap/openstreetmap-carto/openstreetmap -carto.lua -S /opt/programas/openstreetmap-carto/5.3.1/hom_openstreetmap/openstreetmap-carto/openstreetmap-carto.style -o /opt/appfiles/hom_openstreetmap/dirty_tiles.38216 /opt/appfiles/hom_openstreetmap/changes .osc.gz Log: osm2pgsql version 0.92.0 (64 bit id space) ... Reading in file: /opt/appfiles/hom_openstreetmap/changes.osc.gz Using XML parser. Processing: Node(60k 0.2k/s) Way(0k 0.00k/s) Relation(0 0.00/s)node cache: stored: 63588(100.00%), storage efficiency: 48.50% (dense blocks: 2, sparse nodes: 57356), hit rate: -nan% Osm2pgsql failed due to ERROR: stop COPY_END for planet_osm_roads failed: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. On the PostgreSQL side, we have this message below with a few minutes after the connection with the database started. However, the execution in the application part continues and is only finished minutes later. 2022-06-02 09:36:38 -03 [35713]: [1-1] user=usr,db=dbhom LOG: could not receive data from client: Connection reset by peer We have the application server, where osm2pgsql runs, and we have the database with the same settings both.
The software versions used in the 3 environments are the same and have been recently revised. |
|
I'd start with upgrading your osm2pgsql version to the latest one. You say the environment was "revised", but osm2pgsql 0.92 is from December 2016, and is by now heavily outdated. If you upgrade to 1.6.0, you'll get the added benefit of the new "flex" configuration options of osm2pgsql (see https://osm2pgsql.org/doc/manual.html and https://osm2pgsql.org/doc/manual.html#the-flex-output), which gives much greater flexibility in defining the output database schema. Also, 8GB RAM is low end nowadays. Although your Brazil extract may not be that taxing, as being still quite modest in size at 1.4GB PBF, more RAM never hurts for caching purposes on the OS and database level. |
