NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum

Hi! I am using generate_tiles.py script to generate tiles but it is taking awfully long time for whole Europe. It was generating tiles really fast for just one town, but whole Europe seems to be taking forever. Basicaly I followed this tutorial: http://openstreetmapserverwindows.blogspot.ie/2015/11/osmopenstreetmap-tile-server.html to set up everything on Windows Server 2012 64-bit and we made it. Importing whole Europe from http://download.geofabrik.de/ took 15 days (wow!) probably because we have HDD instead of SSD disk on our server. Our processor is Xenon 3.5 GHz and 32 GB RAM. I started generate_tiles.py 10 minutes ago for zoom level (3-16) 10 minutes ago and I got only 135 tiles so far ;-( (it is processing zoom level 6 now...). This is terribly slooow. How can we speed up this process so that we get all tiles for zoom level (3-16) in at least 1 month? :-) This is serious question.

Looking in Windows "Resource Monitor" I see 4 postgres.exe processes reading data with average speed between ~ 500 and ~ 2500 B/s - isn't this terribly slow ? Maybe we should tweak Postgres to speed it up ?

I can provide further technical details and settings I used in files if needed. Thank you.

asked 08 Feb '17, 23:05

Koles500's gravatar image

Koles500
68337
accept rate: 0%

edited 08 Feb '17, 23:10


OK, nevermind. I noticed there is generate_tiles_multiprocess.py script , I changed NUM_THREADS to 8 and used and it generated half million tiles over the night already :-) It is processing zoom level 12 now, so I think it should be done over next few days.

permanent link

answered 09 Feb '17, 13:58

Koles500's gravatar image

Koles500
68337
accept rate: 0%

The style your using should provide recommendations for partial indexes. Beyond that, general PostgreSQL tuning is applicable, mainly work_mem. https://www.geofabrik.de/media/2012-09-08-osm2pgsql-performance.pdf has some recommendations, but mainly as they apply to importing, and for an older PostgreSQL version.

Most styles are slowest per tile at low zooms, particularly zoom 6-9.

If you are using https://svn.openstreetmap.org/applications/rendering/mapnik/generate_tiles.py you need to adjust the default NUM_THREADS to reflect your CPU. Unfortunately, generate_tiles.py generates tiles one by one instead of rendering a larger area and cutting the image to size like mod_tile. This probably makes it 4 to 16 times slower than mod_tile.

Pre-rendering to zoom 16 isn't practical with most styles. The OSMF rendering servers pre-render to zoom 12, and this takes about a day. Each zoom has 4x the number of tiles, so you're looking at about 300x as many tiles as zoom 12.

permanent link

answered 09 Feb '17, 00:21

pnorman's gravatar image

pnorman
2.4k52140
accept rate: 18%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×341
×263
×48
×33
×24

question asked: 08 Feb '17, 23:05

question was seen: 4,933 times

last updated: 04 Aug '18, 08:06

NOTICE: help.openstreetmap.org is no longer in use from 1st March 2024. Please use the OpenStreetMap Community Forum