C++ object needs to update its vector and map of shared pointers to other objects

I have FooManager class with a public member std::map<size_t, std::shared_ptr<Foo>> fmap; for keeping track of Foo instances. There is an additional farray public member of FooManager which is a std::vector<int>. The Foo instances are added as,

FooManager fooman;
int total_foos = 100;
for (int i = 0; i < total_foos; ++i) {
    auto ff = std::make_shared<Foo>(foo_data, i);  // calls Foo constructor
    fooman.farray[i] = ff;
    fooman.fmap.emplace(ff->GetID(), ff);  // note that ff.id != index i
}

Each Foo instance runs a boost::thread process, and I'm using FooManager to keep track -- initially I found this approach more straightforward than a boost::thread::group, but I could be mistaken.

At a barrier, where each Foo instance is waiting for fooman to do a computation that decides which Foo instances continue and which don't, I have a result foo_continues (a std::vector<int>) that corresponds to the indices of fooman.farray. If, for example, foo_continues[3]=2 and foo_continues[7]=0, I need the Foo instance corresponding to index 7 to be deleted, and that of index 3 to continue with two entries being track in fooman. Importantly, I need the containers fooman.farray and fooman.fmap to update accordingly. How can this be done in a FooManager::CommputeContinues method?

And if you have an alternative approach than to keeping both fooman.farray and fooman.fmap, I'm open to suggestions. Thanks in advance for the help!