As a new user I have multiple questions, I will try to break it up by topic. Part 1I am a new user of OSM and started building a tool to locate customers and group them by criteria. I had made some great progress on my project and then came in last Monday to find the OSM tile server very slow. Not knowing how common this is I started putting together my own server. I stumbled across this thread: https://help.openstreetmap.org/questions/65043/osm-slowdown which initially gives me the impression it is an unusual situation, but either way I happen to have a decent server available that isn't doing anything at the moment and so figured I could set it up anyway. Server Specs:
I started setting it up last week and had an issue where it slowed way down which I discussed on StackOverflow here: https://stackoverflow.com/questions/51625815/osm2pgsql-extremely-slow-on-import-on-server-with-192gb-ram The issue initially looked to be related to a HDD that went bad in a RAID setup and I swapped it out for the SSD and using the shell script you see there I was able to import Alabama in about 300 seconds at a rate of between 300-400k points/second during the first command, then running the second command to --append it slowed way down again to 1k points/second. So while I did have an issue with the HDD, it doesn't seem to be the only bottleneck. Is it better for me to import all of North America using the --create vs --append multiple states? Is there supposed to be that dramatic of a difference between them? I opened the database using a GUI (Navicat) and opened the planet_osm_nodes table which opened quickly with 1000 records. I then asked for the last page to get an idea how many records were in it. It took over a minute to pull back page 416,338. Using iotop I noticed something unusual, it was WRITING to the disk drive at a speed of around 120MB/sec to give me the last page. Is this because of how my GUI is requesting those pages The select statement would be of the type I was visiting with a DBA from Micron over the weekend and talking about the import process and the topic of disabling foreign keys and indexes came up as a way to speed up an import into an empty DB. Does the Osm2pgsql script already do this? He was talking about the Postgresql ROLLBACK feature that lets you reverse commits. While this is a great idea during updates, for an initial DB load, it isn't really important for Postgresql to be keeping track of how to rollback, is there options to disable that within the import script rather than tweaking the server config? This is a brand new installation of Ubuntu 16.04 without any data anyone needs to worry about preserving. With that in mind, what is the fastest method we could devise of importing these records. What is the postgresql.conf file supposed to look like for a server with those stats? Here is what I came up with, but I am sure it could be much better:
asked 07 Aug '18, 18:46 AlanHalls |