Partial specialization for pointer as function return type

I have a template wrapper function that returns a value like this:

template<class T>
T foo(Bar& bar, const char* key) {
    return bar.value<T>(key);
}

But I want it to handle pointer types a bit different, like this:

template<class T>
T foo(Bar& bar, const char* key) {
    return (T)bar.value<void*>(key);
}

So that I can do:

int x = foo<int>(bar, "x");
Baz* b = foo<Baz*>(bar, "b");

Writing it like above obviously gives me an error because of multiple definitions. Is there any other way to do this? I would prefer do not add a cast to each function that uses a pointer.

I've tried the following:

template<class T>
T foo(Bar& bar, const char* key) {
    if(std::is_pointer<T>::value)
        return (T)bar.value<void*>(key);
    else
        return bar.value<T>(key);
}

But that doesn't work either because there are QVariants involved and they produce an error by just instantiating one of its template functions with an unknown pointer type (the bar.value<T>).