A lot of man made structures have facilities for people to walk in and through at least for service purposes: bridges, communication towers, sewers and yes, dams. Larger dams usually have a service tunnel where inspections are made and measuring instruments for leakage and settling are installed. Additionally, there might be machine rooms for water inlets our outlets, power generators etc.
The question is what do we want to convey with the message "this is a building"? For the usual detached house, apartment building, barn, church etc this is clear. For the rest I regret there is no clear line we can draw. There's one big motorway bridge here in the city which has large service corridors. I would still do with highway=* and bridge=yes. Then there is another bridge with which houses some bigger rooms that have been converted to artists' workshops. That might warrant to give this bridge its own building tag.
For the dam I would see it similarly. Is it only a service tunnel that gets inspected once a day, just leave it with dam. But are there also additional facilities like machine rooms in there make it a building (or map both separately if possible).
I guess as so often the answer is "it depends on your gut feeling". <shrug>
