Initializer list as a variadic template argument

I have a couple of classes A, B, and F, all having different constructor signatures, like these:

class A {
public:
     A() {}
};

class B {
    double x_, y_;
public:
    B(double x, double y) : x_(x), y_(y) {}
};

class F {
    vector<unsigned> factors_;
public:
    F(std::initializer_list<unsigned> factors) : factors_(factors) {}
};

The objects of these classes need to be registered somewhere, so the construction should go through a factory, like this:

template<class R, typename... Args>
inline R & factory(Args &&... args)
{
    R * rp = new R(std::forward<Args>(args)...);
    /* do fancy stuff with rp... */
    return *rp;
}

This works fine for classes A and B, but not for F, because typename... wants to take one or more types:

A & a = factory<A>();   // okay     
B & b = factory<B>(0.707107, 0.707107);  // okay
F & f = factory<F>({2, 3, 5, 7, 11});  // error: no matching function for call to ‘F::F()’ in factory

Question: Is there any way to make it work with the general factory<R>(args...) syntax?

I tried a full specialization of the factory function for <F, std::initializer_list>, but either messed up the syntax or, when it compiled, wasn't picked up as factory by the compiler.

Any other ideas?