Can I customize a map that only shows locations I create?

The easiest way is probably generating an SVG map either through the Export tab or with a custom renderer like Maperitive, and then open that in Inkscape or Illustrator or so and remove whatever you don't like.

Try using Viking. While designed primarily for use with GPS tracking, it really excels at making quick maps. It allows you to use any tile source (OSM Mapnik, OSM Osmarender, Google Maps, etc.). You can also create your own Waypoints to overlay the map.

Getting Specific, I would use the following procedure.

1. Start Viking
2. Add a "Map Layer"
3. Goto the appropriate area (View --> Goto Location...)
4. Add a "TrackWaypoint Layer"
5. Add the appropriate locations as Waypoints. You can even choose the icon or image file.
6. When you are happy with things, you can then "Generate Image File..." from the File menu.

You may also find you may want to add different map sources. Cloudmade has some nice map styles that may fit your needs.

I should also add that no matter what map tiles you use, you should make sure you follow the appropriate usage (read copyright) restriction and add the appropriate attributions.

A more complicated way would be to design your own rendering style sheet. That way you can completely control what is displayed and how it appears.