Forum Moderators: open

Message Too Old, No Replies

Javascript Multiplying Wrong

1.2 * 3 is not giving 3.6

         

tsteps

5:04 am on Sep 9, 2011 (gmt 0)

10+ Year Member



My script does a lot of the math correctly, however, when I have it multiply 1.2 & 3. Instead of getting 3.6, I get: 3.5999999999999996

This is the code. Any suggestions on why this is happening and how I can avoid?
<form method='post' name='transform'><table>
Amount: <input type='text' name='v_amount_total' onChange='totcalc()' />
Quantity:<input type='text' name='v_quantity' onChange='totcalc()' />
Sales Tax: <input type='text' name='v_tax' onChange='totcalc()' /></td>
Total:$<input type='text' name='v_calc_total' disabled='disabled' value='#.##' />
<script type="text/javascript"><!--

function totcalc()
{
document.transform.v_calc_total.value =
((document.transform.v_amount_total.value -0) *
(document.transform.v_quantity.value -0))
+ (document.transform.v_tax.value - 0);
}

//--></script>

penders

3:32 pm on Sep 9, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Unfortunately this is the way computers work... and the problems with storing floating point numbers on a binary system. The numbers 1.2 and 3 need to be converted to base 2 in order to be stored and processed. The problem with this is that 1.2 (decimal) does not convert exactly to a precise binary value - it is rounded, some precision is lost. In the same way 1/3 can't be converted exactly to decimal... 0.333333... (however, it is exactly 0.1 in base 3)

In order to output the 'correct' value you need to round the answer to the required number of decimal places. Presumably you want whole pennies, so
answer.toFixed(2)
should suffice.

Or... use integer arithmetic, which can be stored precisely... (12 * 3) / 10 == 3.6 exactly.

rocknbil

7:17 pm on Sep 9, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



And, it's not just Javascript, it can happen in any language and is specific to the device hosting the calculations. Previous thread [webmasterworld.com] with some very good explanations and solutions.

tsteps

3:47 pm on Sep 10, 2011 (gmt 0)

10+ Year Member



thanks for the suggestions. I assumed there was probably a thread on here.

Because users will be entering information, I think I'll just use .toFixed(2) for now. Dealing with dollars, I was going to have to anyway.

Have you ever seen it round the wrong way though? Is there a chance it'll be really bad and it'll give me $1.59 and not $1.60?

thanks

g1smd

4:03 pm on Sep 10, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Depends on whether you round up, round down, or round to nearest.

penders

9:43 pm on Sep 11, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Is there a chance it'll be really bad and it'll give me $1.59 and not $1.60?


Unless you are doing millions of iterations and compounding the error each time then you should not get an error this massive.

rocknbil

4:43 pm on Sep 12, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've also found a more reliable way to do this is multiply initial numbers by 100, apply parseInt() to the calculations, then divide the total by 100 before applying toFixed. <aside>It's interesting it's called "floating point precision" with so many inaccuracies. :-) </aside>