SFINAE to limit instantiated type to std::chrono::duration types

I want to do something like

template <typename T>
class TemporalAlarmDelay<T, std::enable_if<std::is_base_of<std::chrono::duration< , >, T>::value, T>::type>
{
public:
    explicit TemporalAlarmDelay(T delay)
        : mDelay(delay)
    {

    }

private:
    T mDelay;
    std::chrono::steady_clock::time_point mTriggerTime;
};


int main(int argc, char** args)
{
    TemporalAlarmDelay<std::chrono::nanoseconds> nanosecondDelay;   // this should work
    TemporalAlarmDelay<std::chrono::milliseconds> millisecondDelay; // this should work 
    TemporalAlarmDelay<std::chrono::seconds> secondDelay;           // this should work
    TemporalAlarmDelay<int> failDelay;                              // fail to instantiate
}

The intent is to limit to types of std::chrono::duration (e.g std::chrono::milliseconds, std::chrono::seconds). What is the best way to go about this? As per my example, I was thinking I could use std::is_base_of, but I now realise that the helper types (e.g. std::chrono::milliseconds, std::chrono::seconds etc) do not use inheritance - derrr, what a silly thought.