jeudi 3 mai 2018

MySQL select for update, many threads time out

I have a table in MySQL database that holds some "prepared" jobs.

CREATE TABLE `ww_jobs_for_update` (
  `id` bigint(11) NOT NULL AUTO_INCREMENT,
  `status` int(11) NOT NULL,
  `inc` int(11) NOT NULL,
  PRIMARY KEY (`id`)
) 

Now I have a C++1y multithreaded application, where each thread goes to the table and selects a job where status=0 (Not completed), does some computation and sets the status=1 upon completion.

The problem is that many threads will acquire a "job row" concurrently so some locking in the database has to take place.

The C++ method that locks/updates/commit is the following

connection = "borrow a connection from the pool"
std::unique_ptr<sql::Statement> statement(connection->sql_connection_->createStatement());
connection->sql_connection_->setAutoCommit(false);


//statement->execute("START TRANSACTION");
std::unique_ptr<sql::ResultSet> rs(statement->executeQuery("select id from ww_jobs_for_update where status=0 ORDER BY id LIMIT 1 FOR UPDATE"));


if (rs->next()) {

  db_id = rs->getInt64(1);
  DEBUG << "Unlock Fetched: " << db_id;
}

rs->close();

std::stringstream ss;
ss << "update ww_jobs_for_update set status=1 where id=" << db_id;


statement->execute(ss.str());
//statement->execute("COMMIT;");
connection->sql_connection_->commit();

"release the connection to the pool();" 

But this approach seems not be efficient. I always get back

ErrorCode: 1205,SQLState: HY000. Details:

from a lot of threads, especially when there load is increasing.

Why I am getting this back? What is the most efficient way to do this, hard consistency is a requirement.

Aucun commentaire:

Enregistrer un commentaire