Initializer list issue in constructor

I have a hard time understanding how std::initializer_list works. I checked other questions, but found nothing relevant (or maybe I didn't see it?).

Say I have this:

template<typename T> 
struct Point
{
    T x,y;
};

template<typename T> 
struct A
{
    std::vector<Point<T>> v;
};

Then I can build with:

int main()
{
   A<int> a{ std::vector<Point<int>> { {4,4}, {5,5},{6,6} } };
}

But I'd like to make thing simpler, so I can write:

int main()
{
   A<int> a( { {4,4}, {5,5},{6,6} } );
}

I tried:

template<typename T> 
struct A
{
    std::vector<Point<T>> v;
    template<typename U>
    A( const std::initializer_list<Point<U>>& il ) : v{il}
    {}
};

But this fails, see live demo.

How can I write a constructor allowing this? Is this even possible?