Special overload for pointer types

Consider the following snippet:

template <class T>
struct remove_pointer
{
};

template <class T>
struct remove_pointer<T*>
{
    typedef T type;
};

template <typename T>
T
clone(const T& v)
{
    return v;
}


template <typename T, typename U = typename remove_pointer<T>::type>
T
clone(const U& v)
{
    return new U(v);
}


int main() 
{
    auto foo = clone<double>(42.0);
    return 0;
}

This code generates compilation errors:

 In function 'int main()':
30:34: error: call of overloaded 'clone(double)' is ambiguous
30:34: note: candidates are:
14:1: note: T clone(const T&) [with T = double]
22:1: note: T clone(const U&) [with T = double; U = double]

Why is the compiler deriving T=double, U=double in line 22? I thought it only should pass if T is a pointer type.