How to resize an std::vector without loosing the existing data?

I was implementing an algorithm that needs me to remove and add the same elements from a vector each iteration of a loop.

Example:

iteration 1 : |1 2 3 4| (size 4)

iteration 2 : |1 3| 2 4 (size 2 with the elements '2' and '4' still there in memory but not accounted for in the size of the vector)

iteration 3 : |1 2 3| 4 (size 3 with the element '4' still there)

Basically, I want to be able to change the value returned by the size() function without affecting the vector for performance reasons.

I know I could use another variable outside of my vector to keep track of its size, but I wanted to know if it could be possible directly inside the std::vector container.

Thanks for any help.