Read safety of std::map / std::unordered_map

I have a std::map (or std::unordered_map, as I assume they behave similarly) that I read and write to. I also have an associated mutex.

I will be both reading and writing (by inserting or deleting elements) to the map. I heard that STL containers were read-safe. If so, it is safe to only use the mutex for write operations?

I'm asking because I need to iterate over the values of the map at one point, and I'd like to only use my mutex when an element requires modification.