Hi there. Thank you for your help in advance!

I am new to setting up my own OSM Tile Server. I followed the instructions here: https://switch2osm.org/serving-tiles/manually-building-a-tile-server-18-04-lts/

Question: My question is how to determine if osm2pgsql has finished successfully while running the command in the background?

Note: This is related to importing the Planet OSM file (more details below). The test import of Azerbaijan detailed in the setup instructions did work as documented, so I know the basic setup is working.

I reviewed the "/var/log/syslog" file and I don't see any messages "success" related to "osm". However, several times when the import failed, there were messages related to the failure - mainly that osm2pgsql had been killed.

Can anyone suggest how I can validate that osm2pgsql did indeed finish successfully? Or is there another place I should be looking for logs?

More Details (in case these help):

I was able to get the basic server running and did the test included in the instructions and confirmed that the software was running properly.

I then imported the Planet OSM file. This required several attempts and tweaks and I referred to multiple helpful pages and posts on the web.

I "think" I was finally successful in importing the Planet OSM file with the following command: sudo -u ubuntu osm2pgsql -d gis --create --slim --flat-nodes 'nodes.bin' --cache-strategy dense -G --hstore --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua -S ~/src/openstreetmap-carto/openstreetmap-carto.style ~/data/planet-latest.osm.pbf >out.txt 2>/dev/null &

My machine is an AWS EC2 m5.2xlarge instance (8 vCPUs and 32 GB RAM) with a 1.3 TB Disk - EBS Volume).

I think peak disk usage reached about 1.2 TB and it took 3 days to run.

asked 30 Mar, 17:17

Buzz1000's gravatar image

Buzz1000
11113
accept rate: 0%

2

If anything is going to run for a while, I tend to use "screen" and detach (using ^a^d) from it while doing something else. Then "screen -r" to reconnect to see how the process is getting along. That way you'll always see the last line of any output.

You can also try redirecting output, or using typescript, but osm2pgsql is quite verbose.

(30 Mar, 17:34) SomeoneElse ♦

Thanks for the ideas. I tried redirecting the output to a file initially but as you said, it is quite verbose and I got worried that just the output would eat up all my disk space.

I'll give "screen+detach" a try. In case you know this off the top and if you will indulge a newb question - will "screen+detach" work even if I log off the machine or do I have to keep the same SSH session alive all through the import? Unfortunately I have to log off once I start the job so I need a way that keeps stuff running even then.

(30 Mar, 18:07) Buzz1000
1

will "screen+detach" work even if I log off the machine

yes

or do I have to keep the same SSH session alive all through the import?

If something kills the "SCREEN" process then bad things will happen. Otherwise you'll see something like this:

me@mymachine:~$ ps -ef | grep 1628
ajtown    1628     1  0 Mar02 ?        00:00:08 SCREEN
ajtown    1629  1628  0 Mar02 pts/1    00:00:00 /bin/bash
ajtown   25162 25135  0 20:04 pts/2    00:00:00 grep --color=auto 1628

"pts/1" is running something via screen, and has been since March 2nd. The original session was logged out, but I can reconnect via "screen -r" any time I want. What I can't guarantee is what happens when your machine completely runs out of memory - my experience is that osm2pgsql would be killed and not "SCREEN", but I can't guarantee that.

(30 Mar, 19:08) SomeoneElse ♦
Be the first one to answer this question!
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×230
×53
×1

question asked: 30 Mar, 17:17

question was seen: 198 times

last updated: 30 Mar, 19:09

powered by OSQA