reasoning for set behaviour

In one of our university projects we are working on a benchmarking project. What we essentially do is generate a random no. and then insert it into a std::set (RBTree) and then delete it. We do this for a specific time interval. Essentially this is what the code looks like

        auto randomNo = std::random_device{}();
        
        //Measure insertion
        auto start = std::chrono::steady_clock::now();
        myset.insert(randomNo ); //log(n)
        auto end = std::chrono::steady_clock::now();
        auto difference = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count();
        //record the diff.

        //Measure erasures
        start = std::chrono::steady_clock::now();
        myset.erase(randomNum); //log(n)
        end = std::chrono::steady_clock::now();
        difference = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count();
        //record the diff.

Any idea why in some fresh runs insertion and deletion takes way more time than normal ?