How can I fake a class with templated methods without recompiling?

I want to fake (i.e. swap with a testing implementation) a class like this:

class Foo {
public:
   ...
   template <typename T> void Bar(T& baz);
   ...
}

Currently, I achieve this by making an unrelated class with the same interface:

class FakeFoo {
public:
   ...
   template <typename T> void Bar(T& baz);
   ...
}

And swapping the class used throughout the codebase via a macro:

#ifdef USE_FAKE_FOO
using ClientFoo = FakeFoo;
#else
using ClientFoo = Foo;
#endif

The issues with this is that I am required to re-compile the codebase depending on whether or not I want to run tests. Even worse, when I have multiple FakeFoos which implement different functionality for different tests, I have to re-compile the codebase for each specific FakeFoo I wish to use.

Is there any way I can work around this?

NOTE: templating the code that uses a Foo does work, but I don't want to require that all client code does this just so tests can be more easily run.