C++ std::unique_ptr with STL container

I got a piece of code which uses a std::set to keep a bunch of pointer.

I used this to be sure of that each pointer will present only once in my container.

Then, I heard about std::unique_ptr that ensure the pointer will exists only once in my entire code, and that's exactly what I need.

So my question is quite simple, should I change my container type to std::vector ? Or It won't changes anything leaving a std::set ?