Getting weak pointer to derived class

I have a bunch of derived classes stored as shared pointers, I was wondering if there is any way of getting a weak_ptr to the object from inside the object?

I've tried using the shared_from_this() function but the problem is that since it's a derived class, when I make the base class inherit from enable_shared_from_this, when the derived class calls shared_from_this() it gets a shared_ptr of the base class not the derived class which I can't turn into a shared_ptr of the derived class

Any suggestions?