I am using c++ and I have to process txt/csv files. Lines are basically records and fields are separated with commas. First I read the file and store it as myPoints and then call functions for calculating the modes of each field and finally the mode for each chunk of the data. It works fine and I am pretty sure that the algorithm is working correctly, but the problem is that it doesn't treat same files in the same manner i.e. when storing each record in myPoints, sometimes it reads the last field with one extra character and if I cut and paste the same data in the same file, sometimes the problem is resolved and sometimes not.
I really don't know what is wrong here and that's why I can't fix it. This extra character causes me a lot of troubles as I can't calculate the modes correctly.
Any insights?
The code below is a part of an MPI project and each process is reading its right chunk of data from the same file here, plus as you can see I am using ifstream and stringstream to read and process the file:
unordered_map < int , unordered_map <int , vector<string> > > storeMyShareOfLines(string fileName , int rank , int numPointsLocal){
string dataLine,line2;
ifstream myfile(fileName,ios::in);
vector<string> point;
unordered_map < int , unordered_map <int , vector<string> > > myPoints;
int line_counter = 0;
int start_line = rank * numPointsLocal;
int end_line = start_line + numPointsLocal;
if(!myfile.is_open()){
cout<<"Failed to open"<<endl;
exit(0);
}
int j = 0;
while (getline(myfile, dataLine))
{
stringstream sline(dataLine);
int myShare = 0;
if(start_line <= line_counter && line_counter < end_line)
{
while (getline(sline, line2, '\n'))
{
myShare++;
if( myShare <= numPointsLocal)
{
string word;
stringstream sword(line2);
while(getline(sword,word,','))
{
//cout<<word<<"\t";
point.push_back(word);
}//','
myPoints[rank][j]=(point);
j++;
point.clear();
line_counter++;
//cout<<"\n";
}
else
{ myfile.close();
return myPoints;
}
}//'\n'
}//if-line_counter
else
{
getline(sline, line2, '\n');
line_counter++;
}
}//dataLine
myfile.close();
return myPoints;
}
Aucun commentaire:
Enregistrer un commentaire