This is a static archive of our old OpenStreetMap Help Site. Please post any new questions and answers at community.osm.org.

Is it possible to recover from a failed osm2pgsql import?

0

After a week of importing, I hit an error due to an unrelated db issue. Is it in any way possible to pick up and continue the last ~3 hours of the relations import?

Reading in file: /osm/data/planet-220516.osm.pbf
Using PBF parser.
Processing: Node(7685383k 248.6k/s) Way(858972k 2.10k/s) Relation(9501590 59.22/s)WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.
terminate called after throwing an instance of 'std::runtime_error'
  what():  result COPY_END for planet_osm_polygon failed: SSL SYSCALL error: EOF detected

asked 31 May '22, 09:33

JamesPep's gravatar image

JamesPep
16223
accept rate: 0%


One Answer:

2

No, unfortunately it is not possible to pick up where the import crashed.

Note that one week for the import seems to be on the long side. Have you read the chapters on tuning the database in the manual? https://osm2pgsql.org/doc/manual.html#tuning-the-postgresql-server

answered 01 Jun '22, 08:15

Jochen%20Topf's gravatar image

Jochen Topf
5.2k55074
accept rate: 31%

It's painfully slow, but it's on one of those new AWS RDS Aurora-Postgreql "serverless" instance, so I'm not sure how to tune it.

(02 Jun '22, 10:30) JamesPep

Source code available on GitHub .