NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

how to optimization the osm2pgsql on Dedicated Server (centos 7)?

I have 2 of this testing server with:

1) 8 core, 32G RAM, NVME 512GB

2) 8 core, 64G RAM, NVME 512GB

software: Linux (centos 7.6.1810)

/etc/sysctl.conf

vm.swappiness = 0

/etc/security/limits.conf

* soft nofile 1024000
* hard nofile 1024000
* soft nproc unlimited
* hard nproc unlimited
* soft core unlimited
* hard core unlimited
* soft memlock unlimited
* hard memlock unlimited

/etc/fstab

/dev/mapper/centos-root / xfs defaults,noatime,nodiratime 0 0

sh run:

export FLATNODESFILE=/var/osm/nodes.cache

export PBFFILE=/mnt/resource/planet-190805.osm.pbf

export CACHESIZE=30000 # how to optimization this para on differencce HW

export PGPASSWORD=< pg_password >

# /usr/local/share/osm2pgsql/default.style

osm2pgsql --create --hstore --hstore-add-index --cache $CACHESIZE --multi-geometry --unlogged \

--flat-nodes $FLATNODESFILE --number-processes 8 -v \

--host < PG_Server_IP > --port 5432 --database gis --username osm $PBFFILE

any other optimization are need?

asked 02 Sep '19, 03:53

Nationals's gravatar image

Nationals
11446
accept rate: 0%

edited 02 Sep '19, 04:20

DateFile Node Way Relation

190722 5330978k 591991k 6917050

(02 Sep '19, 04:15) Nationals

I suggest to set --number-processes to about half your cores, not the full number. Also, 512 GB of disk space will not be sufficient for a full planet import (1 TB is needed and even that will soon run out). This help site also has some recommendations about optimizing your postgresql.conf (e.g. here).

permanent link

answered 02 Sep '19, 08:29

Frederik%20Ramm's gravatar image

Frederik Ramm ♦
82.5k927201273
accept rate: 23%

PG SQL is on another Dedicated Server with 1TB NVME.

(02 Sep '19, 10:53) Nationals
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×263

question asked: 02 Sep '19, 03:53

question was seen: 1,340 times

last updated: 02 Sep '19, 10:53

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum