mardi 4 août 2015

C++11 std::thread queue consumer

I'm in the process of multithreading a machine learning platform I've developed, and I'm looking into using a FIFO thread queue, where thread lambdas are created, but processed at X threads at a time.

For example, a Training queue, will have 1200 training examples, and I want to be able to process them 8 threads at a time.

I would really appreciate a simple, minimal example, I've googled around but haven't found anything so I'm guessing its something trivial.

Assume that the threads will capture some local variable, but nothing else.

An example is the following:

107     for ( const auto & id : paradigm_uuids )
108     {
109         std::string uuid ( id );
113         query = "SELECT `json`, `text` FROM `tblParadigm` WHERE `uuid`='" + uuid + "';";
114         res = stmt->executeQuery( query );
115 
116         if ( res->next() )
117         {   
121             auto json = res->getString( "json" );
122             auto text = res->getString( "text" );
123 
124             if ( const auto paradigm = std::make_shared<Paradigm>( json, text ) )
126                 auto res = _learner->Training ( paradigm );
131             else
132                 throw std::runtime_error ( "TrainingSession: failed to construct a Paradigm, using uuid = " + uuid );
133         }
134     }

I'd like to create thread objects/lamdas within that for loop, and then have them run 8 threads max.

Aucun commentaire:

Enregistrer un commentaire