I need to read a file in chunks. Since I am reading it in chunks, I thought I could parallelize the communications with computations. I need three active threads, such that while T1 is reading data, T2 could be processing, and T3 could be writing at the same time.
So far, I have managed to create three local buffers where I read the data from a file and store it in these local buffers. I take N elements from the file. Every time I reach N elements, I change thread_id, which means that next thread should store the next elements to its corresponding local buffer.
The #pragma omp critical make sure that while one thread is processing or writing data, no other thread is doing the same. However, if T1 is processing its chunk of data, T2 can write its output. This makes sure that the input is read and written correctly.
What I have so far is this:
int i = 0;
int thread_id = 0;
while (input_file >> stream_element) {
if(thread_id == 0) {
input_buffer_1[i] = stream_element;
}
else if (thread_id == 1) {
input_buffer_2[i] = stream_element;
}
else
input_buffer_3[i] = stream_element;
i++;
if(i == N) {
i = 0;
if(thread_id == 0) {
#pragma omp critical (processing_data)
process(input_buffer_1, output_buffer_1, N);
#pragma omp critical (writing_data)
write(output_file, output_buffer_1, N);
}
else if(thread_id == 1) {
#pragma omp critical (processing_data)
process(input_buffer_2, output_buffer_2, N);
#pragma omp critical (writing_data)
write(output_file, output_buffer_2, N);
}
else {
#pragma omp critical (processing_data)
process(input_buffer_3, output_buffer_3, N);
#pragma omp critical (writing_data)
write(output_file, output_buffer_3, N);
}
thread_id++;
if(thread_id == STREAM_THREADS)
thread_id = 0;
}
What would be the best way to initiate three threads and do these tasks in parallel and synchronized manner?
A simple visualization of the desired result is shown below.
Best regards, Suejb

Aucun commentaire:
Enregistrer un commentaire