How can I get/extract the names of cities with their latitude and longitude details and their boundaries?

I must say that Nigeria will be a tough region. I'm afraid the boundaries you want probably won't be complete in Nigeria.

Secondly, it would depend on which boundaries you want. I see that most rough residential areas are available, but I see no administrative boundaries.

In any case, you can start with the city names, their lat and lon. It's rather simple with overpass, just execute this query: http://overpass-turbo.eu/s/ceG

Getting the bounding box will quite be a bit harder. You could use Overpass to extract all residential landuses from OSM, and bind them to the city names using some GIS software (like qGIS). But Overpass doesn't seem to treat landuse areas as areas, so you can't perform is_in queries on it, which makes it a whole lot harder.

You can go to the webservice from user Wambacher and download some boundary data in different format, like raw OSM XML, shapefile or geoJSON.

Choose "Nigeria" from the tree menu on the left screen side, and open its subtree structure to select smaller regions with higher admin_level.

You have to be logged in to that webservice with your OSM account credentials via OAuth to download data.