Infinite recursive template instantiation expected?

I am trying to understand why a piece of template metaprogramming is not generating an infinite recursion. I tried to reduce the test case as much as possible, but there's still a bit of setup involved, so bear with me :)

The setup is the following. I have a generic function foo(T) which delegates the implementation to a generic functor called foo_impl via its call operator, like this:

template <typename T, typename = void>
struct foo_impl {};

template <typename T>
inline auto foo(T x) -> decltype(foo_impl<T>{}(x))
{
    return foo_impl<T>{}(x);
}

foo() uses decltype trailing return type for SFINAE purposes. The default implementation of foo_impl does not define any call operator. Next, I have a type-trait that detects whether foo() can be called with an argument of type T:

template <typename T>
struct has_foo
{
    struct yes {};
    struct no {};
    template <typename T1>
    static auto test(T1 x) -> decltype(foo(x),void(),yes{});
    static no test(...);
    static const bool value = std::is_same<yes,decltype(test(std::declval<T>()))>::value;
};

This is just the classic implementation of a type trait via expression SFINAE: has_foo<T>::value will be true if a valid foo_impl specialisation exists for T, false otherwise. Finally, I have two specialisations of the the implementation functor for integral types and for floating-point types:

template <typename T>
struct foo_impl<T,typename std::enable_if<std::is_integral<T>::value>::type>
{
    void operator()(T) {}
};

template <typename T>
struct foo_impl<T,typename std::enable_if<has_foo<unsigned>::value && std::is_floating_point<T>::value>::type>
{
    void operator()(T) {}
};

In the last foo_impl specialisation, the one for floating-point types, I have added the extra condition that foo() must be available for the type unsigned (has_foo<unsigned>::value).

What I don't understand is why the compilers (GCC & clang both) accept the following code:

int main()
{
    foo(1.23);
}

In my understanding, when foo(1.23) is called the following should happen:

1. the specialisation of foo_impl for integral types is discarded because 1.23 is not integral, so only the second specialisation of foo_impl is considered;
2. the enabling condition for the second specialisation of foo_impl contains has_foo<unsigned>::value, that is, the compiler needs to check if foo() can be called on type unsigned;
3. in order to check if foo() can be called on type unsigned, the compiler needs again to select a specialisation of foo_impl among the two available;
4. at this point, in the enabling condition for the second specialisation of foo_impl the compiler encounters again the condition has_foo<unsigned>::value.
5. GOTO 3.

However, it seems like the code is happily accepted both by GCC 5.4 and Clang 3.8. See here: http://ift.tt/2bTKTIe

I would like to understand what is going on here. Am I misunderstanding something and the recursion is blocked by some other effect? Or maybe am I triggering some sort of undefined/implementation defined behaviour?