|
Hello All, In a Windows environment we were able to successfully loaded our data into a PostGIS database using osm2pgsql. Also successfully downloaded all the requisite shapefiles. Were able to create .png tiles using generate_tiles.py and mapnik-stylesheets-master's osm.xml file. We are wondering what is the best way to custom render .png tiles? Should we use Komstik or TileMill? Or just manually edit mapnik-stylesheets-master's .xml.inc files? Please let us know your opinion and why. The more details the better. We have to work in a Windows environment and in the end must have the .png tiles in the standard OSM directory structure - z/x/y.png. Thanks, |
|
Hello, I had the same problem on rendering the tiles as MapViking, can anyone help with the link to setting up the current stack to run under windows to work with 64 bit integers? Thank you
permanent link
This answer is marked "community wiki".
1
Possible options include using Docker or Windows subsystem for Linux.
(08 Dec '18, 11:34)
SomeoneElse ♦
Finally got all your tutorials and directions right SomeoneElse, from installing Ubuntu on Win 10 (lot of win upgrades and time) until switch2osm tutorial for Ubuntu 18.04 (I am new on linux so yes, it got me time to get it right). Thank you very much for the hard work, great tutorials and fast answer.
(13 Dec '18, 10:31)
questionosmuser
|
|
Hi MapViking. I did the export in pieces. So, I would select a medium sized geographic extent and export several zoom levels. Then, I would export the same geographic extent with several other zoom levels. After I ran mbutil for both exports I combined the resultant .pngs into one directory. This was easy to do. From the first export mbutil might produce directories 9, 10 and 11. From the second export mbutil might might produce directories 12 and 13. When I copied directories 9, 10, 11, 12 and 13 all into the same directory it was just as though TileMill had generated those .pngs all at the same time. I too found that TileMill bombed when I had it generate too many .pngs at once. Geographically dividing the area up and doing different zoom levels at a time was a solution that worked for me. |
|
I have now imported the osm file for South America into the PostGis DB: http://download.geofabrik.de/south-america-latest.osm.pbf When I open TileMill I can zoom and pan within the imported data. It takes a while for the UI to update, but eventually TileMill show the map for selected zoom-level and pan area nicely. I now want to export all of the data in the DB (for South America) to an mbtiles file. If I set a larger area (bounds) than is showing on the geofabrik site for South America (which I have to do (I guess) if I want to cover the whole area that is in the osm file), what will TileMill do with all the area which is outside the area in the osm file? Will TileMill generate tiles for none-existent data in the DB? How do I specify export bounds that make sure I only generate mbtiles data for the data (area) in the database, and not area outside the OSM data from geofabrik ? Would be nice to get this clarified before I start the export process, since I suspect that process will take many hours, zoom level 1 - 11. UPDATE: I now set bound so I covered the main land of South America, and activated the export to mbtiles in Tilemill. After a short while I got the message in TileMill that the export failed. At the end of the TileMill log I see this: [Error: socket hang up] code: 'ECONNRESET' So what can be the reason for this? Any tips on what I could do to resolve this issue? What I want to do is to render tiles region by region, and eventually end up with a map (png tiles) for the whole world (zoom level 1-11) except for Antarctica region. I use: Windows 7 8 x Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz 12.9GB RAM / Then I imported norway-latest.osm.pbf into database. Opened a new project with the same OSMBright style (as I used for Iceland). After a while I get a detailed map visible in editor and exporter window of TileMill. Then I create a bounding box just around Norway and export. After a minute or so, TileMill report Export process failed. In the tilemill log I find this at the end: bad allocation [tilemill] Export process failed [tilemill] [tilemill] no crash logs found SECOND UPDATE I do not know why tilemill complain about bad allocation and fail in the export process of mbtiles file. But in this last test I'm still using data fra the datbase (as I did when I managed to create an mbtilesfile for Iceland). I started with South America (high in spirit), but that failed when exporting from TileMill. Then I imported Norway into the datase and selected norway bounds, and that failed. Now in my last try to get something usable out of tilemill for something else than Iceland, I thought I could try to export just a tiny part of Norway. That also resulted in failed export and this message at the end of the tilemill log file: bad allocation [tilemill] Export process failed [tilemill] [tilemill] no crash logs found [tilemill] Closing child process: core (pid:11088) Exiting TileMill due to fatal error... Here is the tiny area that I selected. If I have to select smaller areas than this in the export process, then I believe it is going to be a very long way to generate tiles for all the regions and countries I want. I have looked at memory and CPU usage in the export process. Memory usage are quite low, but CPU reaches 100% in a short time. If anyone has some suggestions on how find out the cause for tilemill keeps failing the export - it would be awesome. THIRD UPDATE: Now I used the same tiny area as below, but selected zoom level 1-8. Then it generated the mbtiles file without any errors. I will now try to move the export dir for TileMill to an SSD disk, to see if that make it possible to export the whole zoom range I want (1-11), and perhaps choosing larger bounding box. If not, I will have to find another way to pre-generate the image tiles. FOURTH UPDATE: I moved the export dir of TileMill to an SSD disk. Then I used same area (bounds) as in the image below, and selected zoom level 9-11. Then that generated the mbtiles file whithout errors. Then I dragged the right edge of the selected area, to the left side of Norway, to get the western part of this lower area of Norway. Here I tried first with zoom level 1-11. Which failed. Then I tried with zoom level 1-8. Which also failed. Memory usage is around 3-4 GB. CPU usage is popping up to around 99-100% when I activate the export. Since it is not possible for me to get consistent exports here without to much trouble, then I probably have to find another solution. Another thing that might be an issue for me, is that In order to get tiles across borders to match in the final map (map for most of the world), I will have to import region by region to the database, and then select my bounding boxes for the tile generation carefully. I suspect that if I import e.g. only Norway, and then produce tiles for that country. Then import Sweden and produce tiles for that country. Then there will be some tiles which will miss data for the area which fall outside the data in the database and still are within the set bounding box - causing some of these neighbouring tiles to loose data (please correct me if this is wrong). Shapefiles are not available from the geofabrik.de for other regions than Antarctica. So working with shapefiles from geofabrik - is not an option for me. I have seen the suggestion to use Kosmtik. Maybe I will have to look into that as well.
|
|
Hi again Heather, I have now read the tutorial. And have been able to import the osm-bright style into TileMill, as well as connect to the database where I have the planet data (not the whole planet though) from Geofabrik.de. Then I exported to an mbtiles file from TileMill and used mbutil to create the tiles. The map tiles now render without any artifacts, misalignment and such. You can see same area as I added to my first comment in this thread - for comparison (below). In between the work with TileMill, I checked what version of osm2pgsql I was using when importing data to the postgis db, and found out that is was quite an old version. I installed the previos version using a link in an online tutorial on how to get it to work on Windows. After I downloaded the latest version of it, and then imported data into the db - and used mapnik to generate tiles again, then the map was allmost perfect. But still some misalignment of road numbers. I have also exported an xml file from the osm-bright project I imported to TileMill. I would now like to try to use the OSMBright-style (xml) exported from TileMill in my previous workflow - to see if that actually produces the same end result. If it does it would be interesting to see which of the two - mapnik or mbutil, produces the tiles faster, and if there is any difference in the size of the png tiles (in bytes). When I use osm2pgsql to import data to the PostGis db, I specify a style to use. I use something called default.style in the osm2pgsql folder. What is this? Can I produce such a file from the generated OSMBright.xml from TileMill or some other tool? I also get some errors when I try to use the OSMBright.xml with the generate_tiles.py in mapnik. The current error is: RuntimeError: Failed to find font face 'Arundina Sans Regular' in FontSet 'fontset-0' in FontSet at line 15 of 'OSMBright.xml' When I look in the folders of TileMill I have not been able to locate that font face. Do you know how to get this fixed, and where to find the font / fontset? And where do I store these fontset / fonts in mapnik? Anyway... using TileMill and Mbutil, as you suggested, I can now generate png tiles where a comparable area of the previously posted faulty looking map (from my first comment in this thread) - now look like this (so if not able to produce perfect tiles in mapnik under windows with the OSMBright.xml style, then I can definitely use your solution to generate the tiles):
|
|
Hi MapViking, I ended up using shapefiles from http://download.geofabrik.de/index.html, TileMill, MBUtil, Python and my own custom style.mss. This worked and everything lines up correctly. This tutorial outlines everything quite well https://www.e-education.psu.edu/geog585/node/711. My understanding is that TileMill is old and not supported anymore however, it is free and does the work correctly. I also found it much easier to work with shapefiles as opposed to PostgreSQL/PostGIS/QGIS. I added the shapefiles as Layers to TileMill. In the Add layer dialog box I specified them as File and then I only had to enter the ID and Datasource properties and left SRS at Autodetect. All of my shapefiles are in the same projection/datum. I wrote my own style.mss and added it to the TileMill project. My style.mss is much simpler than OpenStreetMap's. There are examples on the Web as to how to render different things with a style.mss and I often referred to these to figure out how to do things. Then in TileMill in the Project settings (wrench tool) I set the following. Format: Optimized png (256 colors) Zoom: 9-11 Center: -121.0988,47.5023,12 Bounds: -125.584,43.3610,-115.723,49.5930 Scale Factor: 1 MetaTile size: 2 I saved these settings. Under the Export menu dropdown I selected MBTiles. I then clicked the Export button.
When the Export was done, I selected 'View exports' in TileMill's Export menu dropdown. Then, I clicked 'Save' in TileMill to save the file. For instance, c:\MyFiles\WaSt.mbtiles. I already had python installed on my machine. I open a command prompt with administrator’s rights. At the command prompt I entered c:\python27\python.exe c:\mbutil\mb-util c:\MyFiles\WaSt.mbtiles c:\MyFiles\MyTiles When mb-util was finished running I had the .png tiles for the specified bounds for zoom levels 9, 10 and 11 in the c:\MyFiles\MyTiles directory in the appropriate format. TileMill can take awhile to render .pngs at the higher zoom levels. Sometimes I had to divide the area up that I wanted .pngs for so TileMill wouldn't bomb. I did this by changing the Bounds in the Project Settings to only generate parts of the whole area at a time. Feel free to ask me any specific questions. Heather 3
Unfortunately TileMill is not supported anymore. It is therefore sensible to use Kosmtik for further use because TileMill does not interface to current versions of Mapnik. If you want static tiles at specific zoom levels QGIS querying your osm2pgsql database is a decent option as you can save the rendered data as MBTiles.
(25 Jul '18, 12:51)
SK53 ♦
Hi Heather, and thank you very much for taking the time to answer my question the way you did. I will read through the walkthrough you linked to, and see if I can get it to work. I'll come back laiter with information on how it went.
(25 Jul '18, 13:50)
MapViking
SK53, Is Kosmtik completely free? Can you use shapefiles with Kosmtik?
(25 Jul '18, 17:09)
SeattleHeather
1
Kosmtik is completely free, and you can use shapefiles: it's pretty much the tool used by developers of the main map style on OSM. It basically offers a similar front-end onto Carto-CSS to that of TileMill. It is open source and was developed by an OSM contributor. Setting up Kosmtik requires more work than TileMill, but there are decent instructions on the github project page, and some notes on switch2osm. (One further fly in the ointment, which I wouldnt worry about too much, is that Carto-CSS is itself getting long in the tooth. There are some interesting discussions on the subject in some OSM forums).
(25 Jul '18, 19:13)
SK53 ♦
1
SK53, Is there a good 'How To' as to how to install Komstik, load a few locally stored shapefiles and export it all to .pngs in the Windows environment?
(25 Jul '18, 21:01)
SeattleHeather
Hello friends, Which one of you could do that service to get tile render png osm?
(11 Oct '18, 19:00)
CLAGOS
The Docker environment that's part of OSM Carto should work on Windows, and does include Kosmtik. Windows Subsystem for Linux (see here) is another option.
(11 Oct '18, 22:52)
SomeoneElse ♦
showing 5 of 7
show 2 more comments
|
|
Hi. Did you have any success in generating png tiles using mapnik on windows? I found this: https://openstreetmapserverwindows.blogspot.com/2015/11/osmopenstreetmap-tile-server.html And managed to generate png tiles using mapnik in the way your are asking. But not really usable since many roads are not rendered correct, and there is also some misalignment of the content internally in the generated tiles. I suspect stylesheet problem, but information about how to accomplish this in a windows environment is not to good. So If you have made any progress and want to share some of your experience - please do so. Here is an example area from the map from my server, using the generated tiles and leaflet. Compared to the next image, which is from same area and same zoom level (but from the Openstreetmap.org server, there is still some more work to be done before it's usable:
3
You are referencing very old technology. The image you produce strongly suggests to me that you are using an old software stack which does not handle 64-bit integers used for OSM elements. There is a relatively recent post by SomeoneElse (one of the maintainers & testers of the switch2osm website) as a diary entry about setting up the current stack to run under Windows.
(25 Jul '18, 12:54)
SK53 ♦
Thanks for tips. Do you have a link to that recent post about setting up the current stack to run under windows?
(26 Jul '18, 08:39)
MapViking
Hello everybody, I wanted to ask if someone got the link mentioned or if you have an answer to handle 64 bit integer, I am having the same issue of missing data on map tiles
(08 Dec '18, 11:31)
questionosmuser
|
meta: hi all, please note that this help site works different from forums with discussion threads. Please stick to real questions, answers (answers to the original question) and comments (on question or on answers) next time - or use the OSM forum instead, for such a discussion.