Using templates in std::conditional to determine function argument types

I want all my saving and loading of data to go through the same functions to reduce the chance of bugs. To do this I used a lot of templates (and much function overloading). It worked, my code is now much cleaner, but I was unable to use const for saving (because it goes through the same functions as the loader does, where the data is kept non-const).

I'd like to use the const correctly, so here is an attempt to get a simple version working, where the data (in this case std::vector) is non-const for std::ifstream, and const otherwise:

#include <iostream>
#include <fstream>
#include <vector>

template <class Foo>
void Overload(const Foo & foo)
{
    std::cout << "went to const" << std::endl;
}

template <class Foo>
void Overload(Foo & foo)
{
    std::cout << "went to non-const" << std::endl;
}

template <class StreamType, typename... Arguments>
void ReadOrWrite (

    /* for 1st argument */ StreamType & filestream,

    /* type for 2nd argument */ typename std::conditional< 
        /* if */    std::is_same<StreamType, std::ifstream>::value,
            /* then */  std::vector<Arguments...>,
            /* else */  const std::vector <Arguments...>
            >::type

        /*2nd argument name */ & vector
        )
{
    Overload(vector);
}

int main ()
{
    std::ofstream output_filestream;
    std::ifstream intput_filestream;

    std::vector<int> vector;

    ReadOrWrite(output_filestream, vector);
    ReadOrWrite(intput_filestream, vector);

    return 0;
}

I know that it will compile/run properly if I edit the function calls to this:

ReadOrWrite<std::ofstream, int>(output_filestream, vector);
ReadOrWrite<std::ifstream, int>(intput_filestream, vector);

But I don't want the user of the function to need to list the types during the function call.

Is there a clean way to do what I'm suggesting?