Unique elements in unordered_set, erasing and adding while iterating?

I am trying to implement a recursion wherein, I'm passing an unordered_set<int> by reference/alias, to reduce space and time complexity. In order to do this, I must iteratively do the following, remove an element, call recursively, and then remove the element from the unordered_set.

This is a sample code for the recursive function for printing all permutations of a vector<int> A is as follows,

void recur(int s, vector<int> &A, vector<int> &curr, vector<vector<int>> &ans, unordered_set<int> &poses){
   if(s == A.size()){
       ans.push_back(curr);
       return;
   }
   for(unordered_set<int>::iterator it = poses.begin() ; it != poses.end() ; it++){
       int temp = *it;
       curr[temp] = A[s];
       poses.erase(it);
       recur(s + 1, A, curr, ans, poses);
       poses.insert(temp);
       curr[temp] = -1;
   }
}

You can assume that I'm passing curr with all -1s initially.

When adding iterating through an unordered_set it is guaranteed to find all unique elements. I was wondering whether it would be the same if I remove and add elements back while iterating. Will the position of the element change in the hashset, or is it dependent on the implementation. If this is incorrect could someone also suggest some other way to go about this, since I do not want to pass by value, since it would copy the entire thing for all recursive calls on the stack. Any help would be appreciated.