vendredi 28 octobre 2016

How to write a large binary file to a disk

I am writing a program which requires writing a large binary file (about 12 GiB or more) to a disk. I have created a small test program to test this functionality. Although allocating the RAM memory for the buffer is not a problem, my program does not write the data to a file. The file remains empty. Even for 3.72 GiB files.

    //size_t bufferSize=1000; //ok
    //size_t bufferSize=100000000; //ok
    size_t bufferSize=500000000; //fails although it is under 4GiB, which shouldn't cause problem anyways
    double mem=double(bufferSize)*double(sizeof(double))/std::pow(1024.,3.);
    cout<<"Total memory used: "<<mem<<" GiB"<<endl;

    double *buffer=new double[bufferSize];
/* //enable if you want to fill the buffer with random data
    printf("\r[%i \%]",0);

    for (size_t i=0;i<(size_t)bufferSize;i++)
    {
        if ((i+1)%100==0) printf("\r[%i %]",(size_t)(100.*double(i+1)/bufferSize));
        buffer[i]=rand() % 100;
    }
*/
    cout<<endl;

     std::ofstream outfile ("largeStuff.bin",std::ofstream::binary);
     outfile.write ((char*)buffer,((size_t)(bufferSize*double(sizeof(double)))));

     outfile.close();

    delete[] buffer;

Aucun commentaire:

Enregistrer un commentaire