Read bitset from file using istream_iterator

I'm trying to read a text file into a vector of std::bitset-objects. The file consists of 1's and spaces (for indicating 0 or false) and I'm trying to do this by overloading the input operator>> and using istream_iterators. I start by changing the spaces in the string to 0's and then construct a bit that I push to the vector with that string. However when I print the output looks very strange and it seems that I'm reading in more elements than lines in the file (the vector of bitsets is longer than the number of lines). The code looks as follows:

  
  template<long unsigned int N>
  std::istream& operator>>(std::istream& is, std::bitset<N>& b){
    std::string line;
    std::getline(is,line);
    for(unsigned long int i =0;i<N;++i){line[i]==' '? line[i]='0':'1';};
    std::cout << line << std::endl;
    b = std::bitset<N>(line);
    return is;
  }
 
 int main(){
  const unsigned long int N = 40;
  std::ifstream file("activities.txt");
  std::istream_iterator<std::bitset<N>> begin(file),end;
  std::vector<std::bitset<N>> bits;
  
  for(auto it = begin;it!=end;++it){
    bits.push_back(*it);
  }
  
  for(auto bit: bits){ std::cout << bit << std::endl;}
 }

I succeeded in doing the same thing without using the istream_iterator and the overloading of operator>> but I'm just trying to figure out why this way is not working.