Wednesday, September 30, 2015

Floating Point Math

In most cases, I tend to write my own code to do simple tasks. In some cases it's not a simple task but I need more control. Recently I've realized that when it comes to math, specifically floating point operations, I will always rely on someone else's code to do the math for me.

Floating point operations tend to have side-effects due to the nature of binary operations and the way numbers are stored. There is a lot to learn if you want to know why this is, but others have already explained it better than I ever will. If you want to, you can read about it here.

Suffice it to say, if you write something like this:
i = 0.1;
j = 0.2;
n = i + j;
...and you expect n to be 0.3, you're gonna have a bad time.

Javascript:
<div id="output"></div>

<script>
 div = document.getElementById('output');

 i = 0.1;
 j = 0.2;
 n = i + j;

 div.innerHTML = n;
 div.innerHTML += (n == 0.3) ? "true" : "false";
</script>
Output:
 0.30000000000000004
 false

ColdFusion:
i = 0.1;
j = 0.2;
n = i + j;
x = 0.3;
writeOutput(n);
writeOutput( ( (n eq x) ? "true" : "false" ) );
Output:
 0.3
 false
As you can see, each language handles the output differently but comes to the same conclusion: n is not equal to 0.3.

To fix this issue, there are multiple approaches. I found the following to work for me.

Javascript:
n = (i + j).toFixed(3);

toFixed() will round the last digit and precison should be taken into consideration. In some cases, you will use a higher percision in the calculation and then a lower percision to display the result. You will need to convert it in order to use toFixed(). In Javascript, variables are automatically stored as strings and not their numeric value. In order to operate on a float as a number, you should use parseFloat() to convert it like so:
div.innerHTML = parseFloat(n).toFixed(1);
In calculations:
n === 0.3  // is false
parseFloat(n) === 0.3  // is true

ColdFusion:
n = precisionEvaluate(i + j);
In conclusion: if you can find a library to do the math for you, USE IT! Frameworks and libraries exist to make our lives easier and this is one of those situations where they should be utalized. Bootstrap and other CSS frameworks make resizing the viewport easy without needing to calculate sizes or anything else. I'm all for making things easier and learning Bootstrap was much easier to learn for me than struggling with odd bugs in logic that might go weeks without being found.

I did some other testing in other languages too, which can be found here.

No comments:

Post a Comment