std::hash for std::chrono::duration

I'm trying to key a set of objects on a std::chrono::duration. This will not compile:

#include <unordered_map>
#include <chrono>

class Foo final
{
public:

    Foo() {}

    int y;
};


int main(void)
{
    auto map = std::unordered_map<std::chrono::duration<float, std::milli>, Foo>();

    map[std::chrono::duration<float, std::milli>(5)].y = 0; // Does not compile.

    return 0;
}

/usr/include/c++/4.9/bits/hashtable_policy.h: In instantiation of 'struct std::__detail::__is_noexcept_hash >, std::hash > > >':

I'm guessing the problem here is there's no std::hash implementation for std::chrono::duration? If not, is there a way of doing this without resorting to keying on the eminently breakable count() and hashing on that?