C++ Implicit Cast from R-Value

I'm trying to define an implicit cast operator that will move a wrapper type into the implicit type as such:

struct SomeStruct
{
    int a;
    int b;
    int c;
};

struct AnyStruct
{
    template <typename T>
    operator T&() &
    {
        return Get<T>();
    }

    template <typename T>
    operator T() &&
    {
        return std::move(Get<T>());
    }

    // details
    // ...
};

int main(void)
{
    SomeStruct s1 { 1, 2, 3 };
    AnyStruct s2 = std::move(s1);

    SomeStruct& s3 = s2;           // implicit cast working
    SomeStruct s4 = std::move(s2); // implicit cast not working (copying)
}

The first implicit cast works and calls the right cast operator operator T&() &. I would have expected the second implicit cast to call the right cast operator operator T() && and move s2 into s4, but it calls the cast operator operator T&() & instead and actually copy the struct.

What am I doing wrong?