mardi 19 mai 2020

Error using boost serialization with binary archive

I get the following error while reading from boost::serialization::binary_iarchive into my variable:

test-serialization(9285,0x11c62fdc0) malloc: can't allocate region
*** mach_vm_map(size=18014398509486080) failed (error code=3)
test-serialization(9285,0x11c62fdc0) malloc: *** set a breakpoint in malloc_error_break to debug

My serialization and deserialization code are:

template<class Archive>
void save(Archive & archive, const helib::PubKey & pubkey, const unsigned int version){
  BOOST_TEST_MESSAGE("inside save_construct_data");
  archive << &(pubkey.context);
  archive << pubkey.skBounds;
  archive << pubkey.keySwitching;
  archive << pubkey.keySwitchMap;
  archive << pubkey.KS_strategy;
  archive << pubkey.recryptKeyID;
}

template<class Archive>
void load_construct_data(Archive & archive, helib::PubKey * pubkey, const unsigned int version){
  helib::Context * context = new helib::Context(2,3,1); //random numbers since there is no default constructor
  BOOST_TEST_MESSAGE("deserializing context");
  archive >> context;
  std::vector<double> skBounds;
  std::vector<helib::KeySwitch> keySwitching;
  std::vector<std::vector<long>> keySwitchMap;
  NTL::Vec<long> KS_strategy;
  long recryptKeyID;
  BOOST_TEST_MESSAGE("deserializing skbounds");
  archive >> skBounds;
  BOOST_TEST_MESSAGE("deserializing keyswitching");
  archive >> keySwitching;
  BOOST_TEST_MESSAGE("deserializing keyswitchmap");
  archive >> keySwitchMap;
  BOOST_TEST_MESSAGE("deserializing KS_strategy");
  archive >> KS_strategy;
  BOOST_TEST_MESSAGE("deserializing recryptKeyID");
  archive >> recryptKeyID;
  BOOST_TEST_MESSAGE("new pubkey");
  ::new(pubkey)helib::PubKey(*context);
  //TODO: complete
}

template<class Archive>
void serialize(Archive & archive, helib::PubKey & pubkey, const unsigned int version){
  split_free(archive, pubkey, version);
}

template<class Archive>
void load(Archive & archive, helib::PubKey & pubkey, const unsigned int version){
}

The test that calls the code is the following:

BOOST_AUTO_TEST_CASE(serialization_pubkey)
{
  auto context = helibTestContext();
  helib::SecKey secret_key(context);
  secret_key.GenSecKey();
  // Compute key-switching matrices that we need
  helib::addSome1DMatrices(secret_key);
  // Set the secret key (upcast: SecKey is a subclass of PubKey)
  const helib::PubKey& original_pubkey = secret_key;

  std::string filename = "pubkey.serialized";

  std::ofstream os(filename, std::ios::binary);
  {
    boost::archive::binary_oarchive oarchive(os);
    oarchive << original_pubkey;
  }

  helib::PubKey * restored_pubkey = new helib::PubKey(helib::Context(2,3,1));
  {
    std::ifstream ifs(filename, std::ios::binary);
    boost::archive::binary_iarchive iarchive(ifs);
    BOOST_TEST_CHECKPOINT("calling deserialization");
    iarchive >> restored_pubkey;
    BOOST_TEST_CHECKPOINT("done with deserialization");

    //tests ommitted
  }
}

Considerations:

  1. Serialization works both fine with boost::serialization::text_oarchive and boost::serialization::binary_oarchive. They create a file of 46M and 21M respectively (big, I know).

  2. Deserialization with boost::serialization::text_iarchive basically stopped at the execution of archive >> keySwitching; The process gets automatically killed. This is in fact the biggest part of the archive.

  3. I decided to try with boost::serialization::binary_iarchive since the file is half the size, but I get the error shown at the beginning. The error happens while executing the first read from the archive: archive >> context;.

  4. The asymmetry between input and output (save and load_construct_data) is because I could not find another way to avoid the implementation of the serialization of a derived class of helib::PubKey. Using a pointer to helib::PubKey was giving me compilation errors asking for the serialization of the derived class. If there is some other way I'm all ears.

Thank you for your help.

Aucun commentaire:

Enregistrer un commentaire