Can I abstract away specific details about templated classes?

I'm working on creating a factory library (similar to http://ift.tt/VAY9E3 that is available for Java) in C++11 to familiarize myself with template programming, as well as create a useful dependency reduction tool. The idea being that the factory would abstract away the details of the creation and destruction of the object, and hide the implementation details. Ideally, I'd like to have something similar to:

InterfaceClass
{
    public:
    virtual void doSomething () = 0;
    virtual ~InterfaceClass () {};
}

// Might need custom deleter depending on how the class was allocated
// (might come from a pool, etc)
ImplementationClass : public InterfaceClass
{
    public:
    // Some (possibly) complicated constructor.
    ImplementationClass(Dependency one, Other dependency) {}

    virtual void doSomething ()
    {
        // Implementation
    }

    virtual ~ImplementationClass ()
    {

    }
}

Ideally, I'd like the end user of the library to just be able to (or something similar):

std::unique_ptr<InterfaceClass> object = factory<InterfaceClass>();

This works great if all classes use the default deleter, but in the case of custom deleters, the type for the unique_ptr changes from:

std::unique_ptr<I> 

to:

std::unique_ptr<I, deleter> 

-- and as far as I can tell, these types aren't compatible.

Is there a way I can define some sort of higher level "unique pointer" that doesn't care about the deleter in it's type signature? Other possible workarounds to keep the API agnostic to the creation / deletion of the object?

Thanks!