mercredi 30 mars 2016

how to put dynamic size of array of threads

As you can see, I want to create few threads.Now number of threads depends on the command line argument I get from console, so basically no of threads to be created has to be dynamic but creating array of objects of thread class in C++11 requires a const size to be given and that is where my problem arises as it is only accepting "num_threads" when it is initialized without using a variable(i.,e using a literal ).
Like: static const int num_threads=10;
But not in case of: static const int num_threads=stoi(argv[1]);

int main(int argc, char *argv[])
{
  if(argc!=2)
  {
     cout << "\n Invalid arguments \n";
     return 0;
  }

  static  const int num_threads = 10;// stoi(argv[1]);//
  thread t[num_threads];


 //Launch a group of threads
 for (int i = 0; i < num_threads; ++i) 
 {
    t[i] = std::thread(call_from_main_to_connect_info_disconnect, i);
 }

 std::cout << "Launched from the main\n";

//Join the threads with the main thread
 for (int i = 0; i < num_threads; ++i) 
 {
    t[i].join();
 }

 getchar();
 return err;

}

Any suggestions in order to have no of threads dynamic through command line input?

Handling multiple std::async calls

I have a requirement, where I need to delete thousands of files efficiently. At present, files are deleted in a sequential manner.

I want to speed up the deletions, by calling delete in an asynchronous manner, using std::async().

Current Flow:

1. Get the list of files
2. For each file call delete()

Desired Flow:

1. Get the list of files
2. For each file:
    2a. Call AsyncDelete() using std::async()
    2b. Store the future object in a vector
3. Wait for each of the deletes to be completed and then return

I will launch each of the async tasks using std::launch::async, so that it runs on a separate thread.

I have following questions:

  1. Is async() suited for workloads involving multiple tasks? Or is it better to use threads for such tasks? I read a chapter (Item 35: Prefer task-based programming to thread-based) in Scott Myer's book "Effective Modern C++", where he recommends using task based programming instead of thread-based.

  2. How costly is each "async()" call? Does it have any overhead like a thread creation overhead? I am planning to control the number of async tasks called per cycle. For e.g. if 10,000 files are to be deleted, I will call just 100 deletes per cycle, instead of spawning 10,000 async() tasks in one go. I hope the standard library implementation efficiently handles multiple async calls (for e.g. using a thread pool).

  3. future() object returned by async() exposes both get() and wait() methods. I read that, get() internally calls wait(). Is it enough to call get() on each of the futures stored in a vector?

  4. What if a get() never returns? Is it advisable to use wait_for() with a time out?

MPI transmission of unknown sub-class with boost::mpi::packed_oarchive and packed_iarchive

I am trying to transmit a class of of unknown subclass, but known base class.

I believe this should be possible using boost::serialization, BOOST_CLASS_EXPORT_GUID and boost::mpi, but I'm pretty new to C++ in general

This is the code that I have:

#include <boost/mpi.hpp>
#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>
#include <boost/serialization/string.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/export.hpp>
#include <boost/serialization/vector.hpp>
#include <iostream>

namespace mpi = boost::mpi;


class Action {
protected:
    int start_rank;
    std::string greeting;

    Action(std::string greeting) {
        mpi::communicator world;
        this->start_rank = world.rank();
        this->greeting = greeting;
    };

private:

    friend class boost::serialization::access;
    template<class Archive> void serialize(Archive &ar, const unsigned int version) {
        ar & this->start_rank;
        ar & this->greeting;
    };

public:

    Action() = default;

    void invoke() {
        mpi::communicator world;
        std::cout << this->greeting << "! I am process " << world.rank() << " of " << world.size()
            << ". I was created on " << this->start_rank << "." << std::endl;
    };
};


class HelloAction : public Action {

public:
    HelloAction() : Action("Hello") {};

};

class GoodByeAction : public Action {

public:
    GoodByeAction() : Action("Good bye") {};

};

BOOST_CLASS_EXPORT_GUID(Action, "Action");
BOOST_CLASS_EXPORT_GUID(HelloAction, "HelloAction");
BOOST_CLASS_EXPORT_GUID(GoodByeAction, "GoodByeAction");

int main() {
    mpi::environment env;
    mpi::communicator world;

    HelloAction *hello = new HelloAction();
    mpi::broadcast(world, hello, 0);
    hello->invoke();

    GoodByeAction *bye = new GoodByeAction();
    mpi::broadcast(world, bye, 1);
    bye->invoke();

    world.barrier();

    if (world.rank() == 0) {
        std::cout << "sending unknown action classes!" << std::endl;
        HelloAction *yup = new HelloAction();
        boost::mpi::packed_oarchive oar(world);
        oar << yup;
    }
    else {
        std::cout << "receiving unknown action classes!" << std::endl;
        Action *action = NULL;
        boost::mpi::packed_iarchive iar(world);
        iar >> action;
        action->invoke();
    }

    return 0;
}

compiling/running with:

mpic++ -g -std=c++1y hello.cpp -lboost_serialization -lmpi -lboost_mpi
mpiexec -np 2 ./a.out

This seems to run just fine:

Hello! I am process 0 of 2. I was created on 0.
Hello! I am process 1 of 2. I was created on 0.
Good bye! I am process 1 of 2. I was created on 1.
Good bye! I am process 0 of 2. I was created on 1.

... until getting to the "sending/receiving of unknown action classes", where I get runtime errors:

receiving unknown action classes!
sending unknown action classes!
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::mpi::exception> >'
  what():  MPI_Unpack: MPI_ERR_ARG: invalid argument of some other kind
[machine-name:20194] *** Process received signal ***
[machine-name:20194] Signal: Aborted (6)
[machine-name:20194] Signal code:  (-6)
[machine-name:20194] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x352f0) [0x7fa685cc22f0]
[machine-name:20194] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37) [0x7fa685cc2267]
[machine-name:20194] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7fa685cc3eca]
[machine-name:20194] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x16d) [0x7fa6862fdb7d]
[machine-name:20194] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d9c6) [0x7fa6862fb9c6]
[machine-name:20194] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8da11) [0x7fa6862fba11]
[machine-name:20194] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8dc29) [0x7fa6862fbc29]
[machine-name:20194] [ 7] ./a.out(_ZN5boost15throw_exceptionINS_3mpi9exceptionEEEvRKT_+0x80) [0x41426d]
[machine-name:20194] [ 8] ./a.out() [0x413802]
[machine-name:20194] [15] ./a.out() [0x4306e6]
[machine-name:20194] [16] /usr/lib/x86_64-linux-gnu/libboost_serialization.so.1.58.0(_ZN5boost7archive6detail19basic_iarchive_impl12load_pointerERNS1_14basic_iarchiveERPvPKNS1_25basic_pointer_iserializerEPFS9_RKNS_13serialization18extended_type_infoEE+0x4d) [0x7fa686df5b8d]
[machine-name:20194] [17] ./a.out() [0x41d08d]
[machine-name:20194] [23] ./a.out() [0x40d293]
[machine-name:20194] [24] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7fa685cada40]
[machine-name:20194] [25] ./a.out() [0x40cfc9]
[machine-name:20194] *** End of error message ***
--------------------------------------------------------------------------
mpiexec noticed that process rank 1 with PID 20194 on node machine-name exited on signal 6 (Aborted).
--------------------------------------------------------------------------

Questions:

  1. Although the question is tagged with boost, should it be possible to transmit an unknown subclass at all with or without boost? I understand I could have a registration of sorts by string name, and an inversion of control guy to handle creation, but I thought that was the intent of BOOST_CLASS_EXPORT_GUID.
  2. Is there a way to make this work with what I have?
  3. Is there a reasonably easy alternative (not using boost) to make this work with regular ol' MPI?

How to mock method returning istream&?

I have mocked virtual method returning istream&. I'd like to use it in a testcase. How to return some value?

The problem is that istream is noncopyable.

Rvalue to forwarding references

I'm reading the reference collapsing rules and I have a question: why if I pass a rvalue A to

template<typename T>
void foo(T&&);

T is deduced to be A?

e.g. if I pass std::string() to the function T is deduced to be std::string, why not std::string&&? It would have made more sense to me, what's the rationale behind deducing T to the type itself?

How to instantiate a new classe inside the function calling?

I am doing this code bellow, creating a class and send in the next line by parameter in two lines of code:

InternalMessage im(call.get_id(), CSEV_CALL_NEW);               
send_call_event(im);

I would like to do something like that using only one line, as C#:

send_call_event( new InternalMessage(call.get_id(), CSEV_CALL_NEW));

Any suggest?

Clang's libc++: using long long in the definition of std::chrono::duration

In libc++ on 32-bit platforms, int64_t is defined as alias of long long. On 64-bit platforms: long.

On the other hand in definition of std::chrono::duration aliases, that you can find here long long is carelessly used:

typedef duration<long long,         nano> nanoseconds;
typedef duration<long long,        micro> microseconds;
typedef duration<long long,        milli> milliseconds;
typedef duration<long long              > seconds;
typedef duration<     long, ratio<  60> > minutes;
typedef duration<     long, ratio<3600> > hours;

So for example, when I require type that is strictly 8 bytes long, I would expect

  foo(uint64_t);
  foo(int64_t);

to be a fairly portable solution. But in case of libc++'s chrono it is not true. There is no portable way except to write your own logic similar to <cstdint>. Ie, defining two additional definitions of foo that take long long and unsigned long long.

Or another example:

  foo(int8_t);
  foo(int16_t);
  foo(int32_t);
  foo(int64_t);

Calling foo(duration.count()) would be ambiguous in this case.

So what the point of using long long that is not larger than long but it's rank is greater than long so it cannot be implicitly cast?

Is this an oversight by developers of libc++?

The reason I brought this up is because drivers of mongodb won't compile on x64 FreeBSD installation. And the reason looks rather silly. For example, gcc used int64_t in the definitions of std::chrono::duration and so it compiles, as expected.