Initializing std::vector outside of main() causes performance drop (multithreading)

I'm writing a path tracer as a programming exercise. Yesterday I finally decided to implement multithreading - and it worked well. However, once I wrapped the test code I wrote inside main() in a separate renderer class, I noticed a significant and consistent performance drop. In short - it would seem that filling std::vector anywhere outside of main() causes threads using its elements to perform worse. I managed to isolate and reproduce the issue with simplified code, but unfortunately I still don't know why it happens or what to do in order to fix it.

This is the original code that performs better:

// Create a new context for each thread
int render_threads = 6;
bool active = true;
std::vector<rt::path_tracer> contexts;
contexts.reserve(render_threads);
for (int i = 0; i < render_threads; i++)
    contexts.emplace_back(cam, scene, bvh, width, height, rnd());

auto render_task = [&active](rt::path_tracer &ctx)
{
    while (active)
        ctx.sample_image();
};

// Spawn rendering threads
std::vector<std::thread> threads;
threads.reserve(render_threads);
for (int i = 0; i < render_threads; i++)
    threads.emplace_back(render_task, std::ref(contexts[i]));

This foo struct mimics behavior of my renderer class. This is the slower version:

struct foo
{
    std::vector<rt::path_tracer> &contexts;
    std::vector<std::thread> threads;
    bool active = true;

    foo(const rt::camera &cam,
        const rt::scene &scene,
        const rt::ray_accelerator &bvh,
        int width,
        int height,
        int render_threads,
        std::vector<rt::path_tracer> &ctx):
            contexts(ctx)
    {
        // Doing this in main fixes the problem
        contexts.reserve(render_threads);
        for (int i = 0; i < render_threads; i++)
            contexts.emplace_back(cam, scene, bvh, width, height, 1000 + i);    
    }

    void run(int render_threads)
    {
        auto render_task = [this](rt::path_tracer &ctx)
        {
            while (this->active)
                ctx.sample_image();
        };

        threads.reserve(render_threads);
        for (int i = 0; i < render_threads; i++)
            threads.emplace_back(render_task, std::ref(contexts[i]));
    }

};

...

std::vector<rt::path_tracer> contexts;
foo F(cam, scene, bvh, width, height, render_threads, contexts);
F.run(render_threads);

Performance drop is quite visible and consistent each time I run those two versions:

  97 samples - time = 28.154226s, per sample = 0.290250s, per sample/th = 1.741498
  99 samples - time = 28.360723s, per sample = 0.286472s, per sample/th = 1.718832
 100 samples - time = 29.335468s, per sample = 0.293355s, per sample/th = 1.760128

vs.

  98 samples - time = 30.197734s, per sample = 0.308140s, per sample/th = 1.848841
  99 samples - time = 30.534240s, per sample = 0.308427s, per sample/th = 1.850560
 100 samples - time = 30.786519s, per sample = 0.307865s, per sample/th = 1.847191

The interesting thing is, when I remove the body of foo's constructor and instead do this:

std::vector<rt::path_tracer> contexts; // Can be on stack or on heap, doesn't matter
foo F(cam, scene, bvh, width, height, render_threads, contexts);

contexts.reserve(render_threads);
for (int i = 0; i < render_threads; i++)
    contexts.emplace_back(cam, scene, bvh, width, height, 1000 + i);

F.run(render_threads);

the performance is back to normal. But then, if I wrap these three lines into a separate function and call it from here, it's worse again. The only pattern I can see here is that filling the contexts vector outside of main() causes the problem.

I initially thought that this was an alignment/caching issue, so I tried aligning path_tracers with Boost's aligned_allocator and TBB's cache_aligned_allocator with no result. It turns out that this problem persists even when there's only one thread running. I suspect it must be some kind of wild compiler optimization (I'm using -O3), althought that's just a guess. Do you know any possible causes of such behavior and what can be done to avoid it?

This happens on both gcc 10.1.0 and clang 10.0.0. Currently I'm only using -O3.

Also, I believe that my path_tracer class may be relevant here as well. It's basically just a per-thread container guaranteeing that threads don't share any writeable data.

class path_tracer
{
public:
    path_tracer(const rt::camera &cam, const rt::scene &sc, const rt::ray_accelerator &accel, int width, int height, unsigned long seed);

    glm::vec3 sample_pixel(const glm::vec2 &pixel_pos) const;
    void sample_image();
    void clear_image();
    const std::vector<glm::vec3> &get_image() const;
    int get_sample_count() const;
    float get_rand() const;

private:
    mutable std::mt19937 m_rng;
    mutable std::uniform_real_distribution<float> m_dist;

    const rt::camera *m_camera;
    const rt::scene *m_scene;
    const rt::ray_accelerator *m_accelerator;

    std::vector<glm::vec3> m_pixels;
    glm::ivec2 m_resolution;
    int m_sample_count;
};

path_tracer::path_tracer(const rt::camera &cam, const rt::scene &sc, const rt::ray_accelerator &accel, int width, int height, unsigned long seed) :
    m_camera(&cam),
    m_scene(&sc),
    m_accelerator(&accel),
    m_rng(seed),
    m_dist(0.f, 1.f),
    m_pixels(width * height, glm::vec3{0.f}),
    m_resolution(width, height),
    m_sample_count(0)
{
}

As requested, I'm also including the code used for benchmarking. It's the main loop of my program that gathers data from all the path tracers in other threads.

// Start time
auto t_start = std::chrono::high_resolution_clock::now();

// While the preview is open
for (int i = 1; preview_task_fut.wait_for(0ms) != std::future_status::ready; i++)
{
    sample_count = 0;
    std::fill(buffer.begin(), buffer.end(), glm::vec3{0.f});
    for (auto &c : F.contexts)
    {
        std::transform(
                c.get_image().cbegin(), c.get_image().cend(),
                buffer.begin(),
                buffer.begin(),
                std::plus<glm::vec3>()
        );

        sample_count += c.get_sample_count();
    }


    {
        std::lock_guard lock{pixels_mutex};

        // Tonemapping code here
        // reads `buffer` and writes `pixels` array
    }

    auto t_now = std::chrono::high_resolution_clock::now();
    std::chrono::duration<double> t_total = t_now - t_start;

    std::cerr << std::setw(4) << sample_count << " samples - time = " << std::setw(8) << std::fixed << t_total.count() 
        << "s, per sample = " << std::setw(8) << std::fixed << t_total.count() / sample_count 
        << "s, per sample/th = " << std::setw(8) << std::fixed << t_total.count() / sample_count * render_threads << std::endl;
    }

Thank you