How to efficiently delete a key from an std::unordered_set if the key is not present in an std::map?

I have an std::map and an std::unordered_set with the same key.

I want to remove all keys from the set that do not exist in the map.

My idea was to do something like the following:

#include <map>
#include <unordered_set>

int main()
{
    std::map<uint64_t, std::string> myMap = { 
                                                {1, "foo"},
                                                {2, "bar"},
                                                {3, "morefoo"},
                                                {4, "morebar"} 
                                            };

    std::unordered_set<uint64_t> mySet = { 1, 2, 3, 4, 123 };

    for (const auto key : mySet)
    {
        const auto mapIterator = myMap.find(key);

        if (myMap.end() == mapIterator)
            mySet.erase(key);
    }

    return 0;
}

And then invoke it on a timer callback every few seconds, however, the snippet above throws an exception when trying to delete the key 123 from mySet, stating:

List iterator is not incrementable.

The thing is, even if it didn't throw an exception I feel like this idea is far from elegant/optimal. I'm wondering if there is a better way to approach this problem?

Thanks.