Forum Discussion

HennSarv's avatar
HennSarv
Helper I
5 years ago

question about precision

I've created some small tests about precision and one point I don't get

why the result end with 1 not with 3.

A bit surpriced
Henn Sarv

General Comment

3 Replies

  • v-yingjl's avatar
    v-yingjl
    Community Support
    5 years ago

    Hi HennSarv ,

    In power query, the largest precision that can be represented in a Decimal Number type is 15 digits long. The decimal separator can occur anywhere in the number. The Decimal Number type corresponds to how Excel stores its numbers.

    Please refer: Data types in Power Query 

    You can also refer this similar issue: Decimal Values Precision 

     

    Best Regards,
    Community Support Team _ Yingjie Li
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

    • HennSarv's avatar
      HennSarv
      Helper I
      5 years ago

      YEs I undestand this fine - My question is about last funcy number on "rounded" result

      When You look my example - there ise 2 numbers with lot of 1-s and 2-s after comma
      when You preserve only 15 digits - You get:

      0.111111111111111
      0.222222222222222
      sum on those much be

      0.333333333333333
      but looks like
      0,33333333333333331

      1st - there is 17 numbers after and result must finish with 3 not with 1

      actualli 

      0.11111111111111111 // 17 digits
      +
      0.22222222222222222
      produces 0,33333333333333331