I am looking to implement a producer/consumer strategy in C++, where one thread decompresses a huge BZ2 stream (Wikipedia dump) and another one parses the resulting XML file.
I am planning on using a simple std::ifstream for reading, and piping it to a Boost decompressor for the first part of the processing pipeline.
I want to pass the resulting std::ostream to another thread to consume it. I'd prefer the read and writes to these streams to be synchronous (i.e. the producer should wait until the consumer is ready for the next read, since the data is huge), and not to keep all data in memory at once - so std::iostream is not appropiate.
What is the best way to approach this?
For reference, what I would really like is a C++ alternative to Golang's pipe.
Thanks!
Aucun commentaire:
Enregistrer un commentaire