Next: Arithmetic Answer 2, Previous: Modes Answer 4, Up: Answers to Exercises

#### 3.7.12 Arithmetic Tutorial Exercise 1

Dividing two integers that are larger than the current precision may give a floating-point result that is inaccurate even when rounded down to an integer. Consider ‘`123456789 / 2`’ when the current precision is 6 digits. The true answer is ‘`61728394.5`’, but with a precision of 6 this will be rounded to ‘`12345700. / 2. = 61728500.`’. The result, when converted to an integer, will be off by 106.

Here are two solutions: Raise the precision enough that the floating-point round-off error is strictly to the right of the decimal point. Or, convert to Fraction mode so that ‘`123456789 / 2`’ produces the exact fraction ‘`123456789:2`’, which can be rounded down by the `F` command without ever switching to floating-point format.