Calling another class` member functions via smart pointers

In a program I'm writing I have a class that creates and handles some threads. After construction, an instance of this will be given an object of another class that the threads will be able to call member functions of.

I've gotten this to work with raw pointers (just replace the smart pointers), but since I have access to smart pointers, I tried to use them instead. Although without much progress .

Some searching led me to use shared_ptrs, so here's what I'm trying to do:

Obj.hpp:

#pragma once

#include "Caller.hpp"

class Caller;

class Obj : std::enable_shared_from_this<Obj> {
public:
    Obj(Caller &c);

    void dothing();
};

Caller.hpp:

#pragma once

#include <memory>

#include "Obj.hpp"

class Obj;

class Caller {
public:
    void set_obj(std::shared_ptr<Obj> o);

    std::shared_ptr<Obj> o;
};

main.cpp:

#include <iostream>
#include <memory>

#include "Caller.hpp"
#include "Obj.hpp"

void Caller::set_obj(std::shared_ptr<Obj> o)
{
    this->o = o;
}

Obj::Obj(Caller &c)
{
    c.set_obj(shared_from_this());
}

void Obj::dothing()
{
    std::cout << "Success!\n";
}

int main()
{
    Caller c;
    auto obj = std::make_shared<Obj>(c);

    c.o->dothing();
}

Running this code leads to a throw of std::bad_weak_ptr, but I don't understand why. Since obj is a shared_ptr, shouldn't the call to shared_from_this() be valid?

Compiled with g++ main.cpp with gcc 7.1.1.