C++ constructor with private initialization

Not sure if this question has been asked before. I am implementing a factory class. Every instance should inherit from a base class InstBase and should be initialized through a centralized factory.

class InstBase {
  friend class Factory;      

  private:
    InstBase() = default;
    factory* factory;
}


class Factory {

   template <typename Derived, typename... ArgsT>
   InstBase* get(ArgsT&&... args) {
     return new Derived(std::forward<ArgsT>(args)...)                     
   }

}

class MyInst : public InstBase {
   public:
   MyInst(int a, int b) {...};
}

factory.get<MyInst>(1, 2);

As you can see in this example, the interface takes a Derived as a template and initialize an instance with user-defined argument list passed to the constructor of Derived. However, what I want to add here is assigning the pointer to the factory in which the instance is created. This can be done by:

   template <typename Derived, typename... ArgsT>
   InstBase* get(ArgsT&&... args) {
     auto ptr = new Derived(std::forward<ArgsT>(args)...);
     ptr->factor = this;
     return ptr;                    
   }

I felt this is a bit redundant because the base class InstBase is always created before the derived and can be only newed by the factory class. Is there any trick to achieve this goal without changing the interface of get? It doesn't make sense for me to ask user to attach another pointer to factory in the argument list (e.g., factory.get<MyInst>(1, 2, &factory)).