Should I manually add building contours in Germany?

Yes please!

• The state-owned data you mention is the data of the federal („Bund“) government. It may be made available for free as in money, but I wouldn’t count on that it will be made available free to use for any purpose, including commercial purposes and relicensing under OSM’s license. The interesting cadaster data with streets and houses is collected and traded by the single states („Länder“). Despite year-long lobbying by the Open Data movement, it will not be made available for free in the foreseeable future.
• The OSM data you contribute to can be used freely for many purposes, as opposed to the official data which comes with many restrictions on the use.
• The OSM data has the potential to be enriched with more detail than can be found elsewhere (for free), like house numbers or names, points of interests, etc.

Yes, it will take some years to get good building coverage. Like it took some years to get good street coverage.