samedi 27 mai 2017

How to best approach scalable test infrastructure compatible with API invocation validation?

I am currently working on writing a test helper that would take an object instance and run it through a sequence of API invocations and validate the corresponding results. Please note that the API invocations results can vary based on the current state of the object instance. Currently, my initial stab at this looks like the following:

enum class API
{
    DoWork1,
    DoWork2,
};

struct DoWork1_Data
{
    bool bSlowPath;
};

struct DoWork2_Data;
{
    unsigned Seed;
};

struct APIOperation
{
    API Api;
    void* pData;
    unsigned Size;
};

/* There will be many of this per combination of APIs that we want to validate */
std::vector<APIOperation> BuildAPIOperations()
{
    std::vector<APIOperation> APIOperations;

    DoWork1_Data DoWork1Data = {};
    DoWork1Data.bSlowPath = true;
    APIOperation Op1 = {};
    Op1.Type = API::DoWork1;
    Op1.pData = &DoWork1Data;
    Op1.Size = sizeof(DoWork1Data);
    APIOperations.push_back(Op1);

    DoWork2_Data DoWork2Data = {};
    DoWork2Data.Seed = 0xDEADBEEF;
    APIOperation Op2 = {};
    Op2.Type = API::DoWork2;
    Op2.pData = &DoWork2Data;
    Op2.Size = sizeof(DoWork2Data);
    APIOperations.push_back(Op2);

    return APIOperations;
}

void ExerciseAPIOperations(IUnknown *pObject, std::vector<APIOperation> & APIOperations)
{   
    for(unsigned i=0; i < APIOperations.size(); i++)
    {
        switch (APIOperations[i].Type)
        {
            case API::DoWork1:
                {
                    /* validation of pData */ 
                    DoWork1_Data *pData = static_cast<DoWork1_Data*>(APIOperations[i].pData);
                    pObject->DoWork1(pData->bSlowPath);
                    /* validation of API DoWork1 behavior */
                }
                break;
            case API::DoWork2:
                {
                    /* validation of pData */ 
                    DoWork2_Data *pData = static_cast<DoWork2_Data*>(APIOperations[i].pData);
                    pObject->DoWork2(pData->Seed);
                    /* validation of API DoWork2 behavior */
                }
                break;
        }
    }
}

I would appreciate general feedback on any improvement advice, and more specifically, I am interested to explore improvements in the following two aspects, and would love to hear thoughts on that:

  1. Instead of N number of BuildAPIOperations(), I was hoping to define a global array of all these different cases, so all the permutations will be clearer to understand and can avoid some amount of code duplication. However, this doesn't seem to fit well with the fact that each API can potentially have different set of parameters and defining that all upfront will probably result in a table that's fairly onerous to parse.

  2. I am not a big fan of defining a custom struct per API (i.e. DoWork1_Data) and then casting these pointers to void* and then back. However, I found them necessary to support a test helper that's compatible with any APIs that this object instance can support.

Please let me know if there's any thoughts on this. Thanks in advance.

Aucun commentaire:

Enregistrer un commentaire