This shouldn't be much of a problem. You can for example just include the URL of the umap page in an iframe. There are other websites that have done that, for example [Spanish Walking][1]. Have a look in that page's code to see how it is done if you are unfamiliar with it.
umap.openstreetmap.fr should work. Other installations of umap might prevent the page from running in a frame.
Update:
I now also found what you were originally looking for I think. In the map settings side panel (right side) go to user interface options and make the embed control visible (always or hidden). Then you get a share icon on the left hand side of the screen with the embedding code.
[1]: https://www.spanish-walking.com/mountains/rock-arches