Compile error when trying to use std::result_of

I want to deduce the return type of a function coming as a template parameter. Consider the following code:

#include <type_traits>

struct Result {};

Result foo() { return Result{}; }

template<typename Factory>
void check(Factory) {
    using ActualResult = typename std::result_of<Factory()>::type;
    static_assert(std::is_same<Result, ActualResult>::value, "");
}

int main() {
    check(foo);
}

This works as expected. However, if I change the parameter of check() to const Factory&, then it does not compile. The error from gcc is:

prog.cc: In instantiation of 'void check(const Factory&) [with Factory = Result()]':
prog.cc:14:14:   required from here
prog.cc:9:66: error: function returning a function
     using ActualResult = typename std::result_of<Factory()>::type;
                                                                  ^
prog.cc:10:65: error: function returning a function
     static_assert(std::is_same<Result, ActualResult>::value, "");
                                                                 ^

What's the problem here? How can I make it work?