mardi 1 novembre 2016

rounding of error in c++

I am having strange rounding off error in xcode/msvc, where division is unexpectedly rounding off certain numbers.

Basically I want to move n digits to the right of decimal point, like 123456 and move 2 digits to make it 1234.56.

I wrote below code and seeing approximations for certain numbers. Can someone please explain whats going on here ?

#include <iostream>
#include <math.h>

double roundoff(unsigned long long num, unsigned short digits)
{
    unsigned long long decimals = 10;
    if (!digits) decimals = 1;
    else while (--digits) decimals *= 10;

    double t = double(num%decimals);
    t = t / decimals;

    double result = floor(num / decimals);
    result += t;

    return result;
}


int main()
{
    std::cout << std::fixed << roundoff(100000000005850000ull, 7) << std::endl;
    std::cout << std::fixed << roundoff( 10000000005850000ull, 7) << std::endl;

    return 0;
}

This prints

10000000000.584999
1000000000.585000

Aucun commentaire:

Enregistrer un commentaire