What is the best way to keep a pointer to different types when only one of them is needed at a time?

This is my first question here, so Hurray! I finally started learning!!

I'm in a situation where I need to have a pointer to an object of unknown type as a member of another class. But the list of valid object types are known at compile-time.

Say I have:

// A class templated on T
template<class T> class obj;

// Now assume there are some objects of types obj<...> somewhere else
obj<int> obj1;
obj<double> obj2;

// Now, manager needs to create a pointer for these objects
// in a specific manner at construction-time

// Say, we have to choose a canonical object (and set the other as secondary)

// manager shouldn't be templated on T because It needs
// to have multiple members of type obj<...>
template<class R>
class manager
{
    shared_ptr<...> canObjPtr;
    shared_ptr<...> secObjPtr;
 public:   
    // once set, canonical obj is not expected to change
    explicit manager(string canonicalObj);
}

How can I achieve this?

Some initial thoughts that don't really work:

1. The most promising approach I can think of is to add T1 and T2 template arguments to manager and construct like this: manager<R, obj<int>, obj<double>>(). I get the feeling that I should grab the "canonicalObj" string with a static function before manager construction and then decide on which manager to create manager<R, obj<int>, obj<double>> or manager<R, obj<double>, obj<int>>.

2. How about "templating" manager on obj1 and obj2 objects directly. Does it seem feasible? Note that I'm resilient to adding template parameters to manager because it's involved in some RunTime selection mechanism that doesn't like working multi-param templates.

3. Instead of 2 member pointers, create 4 (See below, but this is lousy and will surely drive me crazy in the implementation: Need to always check if one pointer is null before using either of them)

template<class R> manager
{
    sharedPtr<obj<int>> canObjPtrI;
    sharedPtr<obj<float>> canObjPtrF;

    // same for secObjPtr above
 public:
    explicit manager(string canonicalObj);
}

4. std::any and std::variant (and their boost equivalents) are out of the question because I want to keep using c++11 and can't use boost by policy. If I were to break one of the rules I would consider upgrading to c++17.

5. I don't think the use of shared_ptr<void> for example can give any benefits because I would have to cast the pointer to the correct type anyway and can't use the interface of the object from the void pointer.

6. The same can be said for union. It provides little to no improvement over 3.

Also, if you see this as a potential design problem, please don't hold back. I invite you to point out any flaws/improvements you notice.