c++ Multiple worker threads and promises

I have a thread safe queue which is prepopulated with work by the main thread; when I launch my worker threads they pop a task from the work queue, but also, can push a new task back to the work queue (sometimes worker threads do push back more work, sometimes they don't). Brief glimpse into code:

auto work_queue = safe_queue{};

static void handle_task(T task) {
  // process the task
  // might push a new task to a work queue using work_queue.push()
}

int main() {
  //some work is done to prepopulate work_queue

  auto handle_work = [](){
    while (!work_queue.empty) {
      T task = work_queue.pop();
      handle_task(task);
    }
  };

  std::vector<std::thread> threads;
  for (int i = 0; i < NUM_OF_THREADS; i++) {
    threads.push_back(std::thread(processing));
  }

  std::for_each(threads.begin(), threads.end(), [](std::thread &t) {
    t.join();
  }
}

I understand that this code won't work correctly since in some cases, queue might be empty while some worker threads are processing work when another thread comes in, doesn't find work and exits (although threads processing tasks might push back new work to queue). My question is, how to prevent threads from exiting prematurely like that? Would that work using std::promise<void> to allow threads to communicate to other threads that they still might be working? If so, how would that work with multiple threads (I'm new to c++ and only used promises with single thread)?