Forum Discussion

LuukP's avatar
LuukP
Helper I
2 years ago
Solved

Subtracting two equal measures gives wrong result

Hi everyone,   I have a strange situation where I am subtracting two variables of equal value and they provide the wrong result.  (Note that when the two variables are different amounts, the subtra...
Need Help
  • Greg_Deckler's avatar
    Greg_Deckler
    2 years ago

    LuukP Looks like a floating point/accuracy kind of issue. Perhaps try changing the data type from Decimal to Fixed? Or add a check that if value is less than .0000000000001 or something and if so return 0.

  • rsbin's avatar
    rsbin
    2 years ago

    LuukP ,

    Expand your Measures to 18 decimal places and I believe you may see the difference.

    To get around this, use the ROUND function in your Measures to actually round your  values to 2 decimal places, rather than just "display" 2 decimal.

    Hope this helps resolve your issue.

    Regards,

LuukP's avatar
LuukP
Helper I
2 years ago

Greg_Deckler rsbin Thanks for your responses.  Rounding the two variables solved the issue.