Interruptible sleep with a std::thread

I have a simple C++11 thread program like below.

Code:

#include <iostream>
#include <thread>
#include <chrono>

int main(int argc, char *argv[]) {
   std::cout << "My program starts" << std::endl;
   std::atomic<bool> exit_thread(false);
   std::thread my_thread = std::thread([&exit_thread]{
        do {
            std::cout << "Thread is doing something..." << std::endl;
            std::this_thread::sleep_for(std::chrono::seconds(5));
        } while (!exit_thread);
    });

   std::this_thread::sleep_for(std::chrono::seconds(12));
   exit_thread = true;
   std::cout << "Might have to wait to exit thread" << std::endl;
   my_thread.join();
   return 0;
}

As you can see above, there is a loop which has a sleep_for which makes the thread sleep for 5 seconds and then it wakes and loops again provided that exit_thread is set to false. Main thread waits for 12 seconds and prepares to exit firstly by setting exit_thread to true and then does a join on the thread. All good until now.

Problem:
Above is okay and works for objective. But there is a "potential problem". If the thread has just now started to sleep then it would take it 4 seconds more before it gets out of sleep to discover that it now needs to exit. This delays the exit process and destruction.

Question:
How to can I make the thread sleep in an interruptible way? So that I can interrupt the sleep and make the thread exit right away instead by cancelling out of sleep instead of waiting for the potential 4 or 3 or 2 seconds.

I think that the solution to this might be achievable using a std::condition_variable? Probably? I am looking for a piece of code to show how.

Note that my code runs on both clang and gcc.