Hi, I've installed my own tileserver and imported the whole France to test (the goal being to serve the entire planet). But the rendering is VERY VERY slow, like even for zoom 12 tiles it takes sometimes over than one minute to render (and I'm the only one on the server, so one minute to render a few zoom 12 tiles, it is really slow). I really don't know why it takes so long, I've followed the instructions on switch2osm, the only difference being that I choosed to use the OSM-Bright theme, customized for my needs. But to be sure it wasn't my custom style which impacted the perfs, I've also tried with the original OSM-Bright and it is the same rendering time. Here is my configuration:
I've imported the data with this osm2pgsql command:
But I've also tried without my custom style and without flat-nodes, it doesn't make any relevant difference. While monitoring the server process and ram usage, I've noticed that it was postgresql and not renderd which was taking a long time, and using all process cores at there maximum (which make sense because from what I understood, the postgresql queries are the slowest part of the process). Following the small tutorial of Paul Norman, I've logged theses queries, returning over and over (the log file is almost only composed of queries like theses ones):
With an average time of 1/2secs by queries, so it's probably what slows down the process. For the record, I haven't enable update yet, so the only queries running are the ones I use for testing rendering. Also, I've tried multiple postgresql confs (following again Paul Newman and Frederik Ramm recommandations), but haven't noticed any big improvment (the time being actually around 1 minute for a zoom 12, I expect it to be closer to a second at this level, am I wrong?). For higher zoom levels, it become faster (but still a bit slow I think):
Does anybody have any idea of why even with the basic OSM-Bright style and this specs I'm facing this very slow rendering time? It doesn't make any sense to me... Thanks! asked 07 Aug '18, 10:49 voharunado |
I've successfully imported the whole planet but now I'm still having the slow rendering issue. I'm just trying to load one tile after the server just started, and I'm seeing the requests Frederik told me before that it was normal at the renderd start:
This one * 5 when I'm looking in In the renderd logs, I see that when the rendering start:
But after that, nothing. Nothing in the postgresql logs either. Does somebody have an idea of what could cause that? Thanks! EDIT: After almost an hour (yeaaah) the tile was done:
But then I started another (random) tile at zoom 16, which should be fast... It was a few minutes ago, I'm still waiting... And I'm still seeing the same requests in answered 02 Oct '18, 13:40 voharunado |
The answered 07 Aug '18, 12:24 Frederik Ramm ♦ Thanks a lot Frederik, I didn't know that, but it make sense because when a launch render_list, it seems to process way faster than from an apache request, probably because the process isn't killed with that. It may be linked to my docker installation, I'll investigate on this.
(07 Aug '18, 13:50)
voharunado
Where should I look to see if the children are killed? Or even when it is fully started? I'm watchning the process list with htop but I can't see any new child process related to renderd, I see some postgres children spawning but that's it. However I've noticed something strange, after some queries, it become faster (still slow but optimizable slow, like 15 seconds for a zoom 12 tile which is way better than the previous time) and I don't see the SELECT ST_XMin queries anymore... Looks like it's random to me, I've reproduced that twice but it didn't append at the same amount of queries/time...
(07 Aug '18, 14:54)
voharunado
Well the problem is really only for the first tiles, so it may be normal I don't know... You said that these queries should only happen at renderd startup but how long does this startup takes? Because I see in renderd log that it is started like 2/3 seconds after I launched it. It takes a very long time (and log the above psql requests, but only when I request some tiles, it does not launch the requests alone at startup) only for the first tiles rendered, but I can't see any logic here... Sometimes it's the first 20, sometimes more (in time, it can take more than 10 minutes of manual requests). But anyway, after a certain number, these requests stops and it starts to be fast... Not really a big issue if this is just at the server startup, but still, is this the normal behavior? Thanks.
(08 Aug '18, 10:37)
voharunado
Did you find a solution in the end? Did you just need to wait? If so, how much time? And what rendering speed did you get? Thanks
(20 Aug '18, 16:34)
timautin
|
Lower zoom tiles tend to be slower as there is much more data per tile / meta-tile to retrieve (from the database) and render (this is naturally offset by zoom dependent style changes), think a factor of 4 per zoom level. It is normal to pre-render at least to level 12 for this reason as SomeoneElse has already pointed out. answered 07 Aug '18, 11:18 SimonPoole ♦ Thanks for your answer. Yes I know that and as I just sayed to SomeoneElse, we intend to pre-render those tiles. So for you, more than one minute at zoom 12 is normal? I don't know much about rendering so maybe I panic for no reason... I've added some rendering times for higher zooms, it's way faster but still not acceptable for a production server...
(07 Aug '18, 11:28)
voharunado
|
Just a couple of thoughts:
1) I'd pre-render tiles to zoom 12 so that the time to do those isn't an issue that users notice.
2) Are you using Docker for a particular reason? Surely any unnecessary redirection will slow things down?
Thanks for your answer.
1) Yes I intend to, but still it seems to me that this rendering tile is really high, even on higher zooms (I've edited my question), the rendering tile can be problematic, ~6 seconds for a zoom 14 tile seems too much...
2) The original reason was to be able to deploy easily (in dev or prod) and to facilitate an eventual use of multiples servers. I'm just starting with Docker, so I don't know all the inconveniants but from what I understand, unlike a virtual machine, the process are actually runned on the host machine, so it may slow down things a bit but not that much I think...