C++ Bad Allocator Exception When Pushing to Vector

So I'm trying to read the mnist hand written digits dataset into c++, but I keep getting a bad allocator exception when I try to push the line of each image into a 2d vector. So below is the method I'm running and no exceptions are thrown in here.

std::vector<Image> ImageSet::ReadImages(const std::string& image_file_path) const{
  std::vector<Image> temp_image_set;
  std::string image_line;

  std::ifstream image_file(image_file_path, std::ios_base::in);
  Image temp_image = Image();

  while(image_file.good()){
    image_file >> temp_image;
    temp_image_set.push_back(temp_image);
  }

  return temp_image_set;
}

However, in the >> operator overload in my Image class, I get a bad_allocator exception. The image class just contains an int for the number label and a 2d int vector representing the image (each image is 28x28 with each pixel either being shaded or unshaded).

std::istream &operator >> (std::istream& input, Image& image){
  std::string line;
  if(input.get() != 0){
    for(size_t index = 0; index < 28; index++){
      std::getline(input, line);
      std::vector<int> line_data = Image::ConvertLineToInts(line);
      image.kImageData.push_back(line_data);
    }
  }

  return input;
}

std::vector<int> Image::ConvertLineToInts(const std::string& line){
  std::vector<int> line_data;
  for(char pixel : line){
    if(pixel == ' '){
      line_data.push_back(0);
    } else {
      line_data.push_back(1);
    }
  }
  return line_data;
}

I suppose it would take a lot of memory to read through the file, but each vector is pretty small and is overwritten each time, so I don't really understand why I'm getting bad_allocator.