Custom rendering for paper map, is Mapbox Studio Classic a good choice?

From what I read about Mapbox Studio Classic, it is primarily geared to making webmaps, and only secondarily for print.

All of your requirements (maximum flexibility in design, topographic quality map output, large dimension printing and layout, map legend), really points to a GIS (Geographical Information System) as a first and likely also final step to process and layout your data.

QGIS is of course an obvious choice, as it allows you to download custom sets of data (both geographically and by attribute) and insert this in a SpatialLite or other type of database, but the default option in its interface still doesn't handle multipolygons properly. Therefor, it may be better to use another tool capable of handling multipolygons to process and insert the data in a database, e.g. osm2pgsql and PostGIS, and then use this as the datasource.

Another option is ESRI's ArcGIS Editor for OpenStreetMap, which also does a pretty decent job (and handles multipolygons better than QGIS in the latest release).

Both QGIS and ArcGIS allow you to output your data as vector PDF, meaning you could postprocess it in another application capable of editing or using PDF vector data, and at the same time not being limited by raster output like in Mapbox Studio Classic, as I understood from their website it only outputs high resolution raster for print.

There are two options I would traditionally consider. Both do, however, require a little bit of 'hacking' skill; if you're worried by Mapbox Studio's steep learning curve, you'll probably find these near-vertical!

The first option is to convert raw OpenStreetMap data (in .osm format) to Adobe Illustrator format, and then style the map in Illustrator. There is a Perl script called osm2ai.pl which will convert an .osm file to .ai, which I wrote many years ago. Rather than providing a direct download link, I'd actually suggest you Google it as a number of people have provided helpful tutorials.

The script doesn't do any styling whatsoever. However, it is capable of filtering OSM data into different Illustrator layers - say, a buildings layer, a primary roads layer, a waterway layer, and so on - to make the styling process easier.

I've used this approach many times to create print maps from OSM data and it works well.

The second option is to use Mapnik, which is the core of Mapbox Studio Classic. However, instead of asking for output in a raster format such as .png, you configure it to produce PDF maps. This is achieved by using the Cairo renderer rather than the default AGG.

This way, you do all your styling in CartoCSS, load your data into an OSM database, and then tell Mapnik to make the map. It requires quite a lot of hacking (most usually with Python, though I use Ruby) to get it to work: PDF is a fairly minority pursuit for Mapnik. However, the results can be surprisingly good. The PDF map export feature on cycle.travel is one place I use this approach.

You might also consider the non-open-software but free-of-charge "Maperitive", a windows program that runs fine on Linux under Mono. It has its own styling language, different from that of TileMill, and it can generate SVG output that can be post-processed with Inkscape. This is useful especially if you want to make some changes that go beyond "all features that have the attribute X should be painted like this" but where you want to style a couple of individual features differently - this is easy to do once you have the data in Inkscape.

Mapbox Studio is a good styling app for pre-visualisation. Hebstreits customizer is a new service working with Mapbox styles and deliver clean maps as Open Vector PDFs for artistic and business use. If open SVG format is desired, this would have to be requested.