C++ template implicit abstract cast

Suppose I have the following function, which chooses between a std::ifstream and std::cin, and returns a std::istream:

std::istream& get_stream(std::string * filename, std::ifstream * fs){
    if(!(filename->empty()){
        std::ifstream& s = get_open_filestream<std::ifstream>(filename,fs);
        return s;
    } else {
        return std::cin;
    }
}

Now, I go to template, and I can't get the same abstract base-class conversion behavior:

template<class Istream, class IFstream>
Istream& get_stream(std::string * filename, IFstream * fs){
    if(!(filename->empty()){
        IFstream& s = get_open_filestream<IFstream>(filename,fs);
        return s;
    } else {
        return std::cin;
    }
}


get_stream<std::istream,std::ifstream>(...) // does not compile

How should I template this such that the base class conversion occurs?