Considering I can spin up any EC2 Instance (even the strongest ones) and tweaking the configuration for maximum import speed, What's the shortest time for a full planet import using is there a bash script that fires asked 16 Sep '19, 14:14  LiorM |
One Answer:
On a 64 GB RAM machine with fast NVMe disks you will typically achieve a full planet import (with slim mode and flatnodes) in about 14 hours. This is for old-fashioned "real" hardware. In the cloud, if RAM is not an issue at all and if you do not need updates, then the fastest imports are probably those run without I am not aware of a script that auto-adapts values. answered 16 Sep '19, 14:40  Frederik Ramm ♦ |
Can you share the
osm2pgsqlconfig so it will run in ~6 hours? Plus - whats the CPU / RAM prerequisites to achieve that?No need for updates.
As I said, the 6 hours is a guess of mine; because of the high memory requirements I haven't run osm2pgsql on a planet file without --slim recently. I think if you have 512 GB of RAM and 4-8 cores you should be fine. The exact command line arguments depend on which map style you'll be using etc. but it would likely be something like:
osm2pgsql -d gis --hstore planet-latest.osm.pbf
some styles will expect you to use a special style file, a LUA transforms file, and perhaps recommend the
-Goption for building proper multipolygons, all of which have the potential of slowing things down a little. If you need LUA transforms, then building osm2gpsql with "luajit" can recover some of the time used for LUA processing. I'm not 100% sure about the--cacheparameter, it used to be irrelevant for non-slim mode but you'll have to see. Do report back your results!