Chapter 11 Language of Descriptive Statistics

Section 11.1 Terminology and Language

11.1.3 Remarks on the Rounding Processes

As the following considerations and examples will show, we have to be very careful when calculating with rounded results. Let us consider the set $M=ℝ{}_{\ge 0}$ of all non-negative real numbers. On this set, let us define the multiplication

${\displaystyle M\times M\mathrm{\hspace{0.5em}\hspace{0.5em}}\to M\mathrm{\hspace{0.5em}\hspace{0.5em}}\mathrm{\hspace{0.5em}\hspace{0.5em}},\mathrm{\hspace{0.5em}\hspace{0.5em}}\mathrm{\hspace{0.5em}\hspace{0.5em}}(a,b)\mathrm{\hspace{0.5em}\hspace{0.5em}}⟼\mathrm{\hspace{0.5em}\hspace{0.5em}}a\odot b}$

by the calculation rule

${\displaystyle a\odot b\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}\frac{1}{{10}^2}·\text{round}({10}^2·a·b)\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}\frac{1}{{10}^2}·\lfloor {10}^2·a·b+\frac{1}{2}\rfloor \hspace{0.5em},}$

i.e. the product $a\odot b$ is calculated by calculating the usual product $a·b$ first and subsequently rounding the result mathematically to two fractional digits.  
The law of associativity no longer applies to the rounded multiplication. For the numbers $a=2.11$, $b=3.35$, and $c=\mathrm{2,61}$, we have, for example,

${\displaystyle a\odot b\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}2.11\odot 3.35\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}7.07\mathrm{\hspace{0.5em}\hspace{0.5em}}\text{and}\mathrm{\hspace{0.5em}\hspace{0.5em}}(a\odot b)\odot c\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}7.07\odot 2.61\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}18.45\hspace{0.5em}.}$

Changing the brackets, however, results in

${\displaystyle b\odot c\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}3.35\odot 2.61\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}8.74\mathrm{\hspace{0.5em}\hspace{0.5em}}\mathrm{\hspace{0.5em}\hspace{0.5em}}\text{and}\mathrm{\hspace{0.5em}\hspace{0.5em}}\mathrm{\hspace{0.5em}\hspace{0.5em}}a\odot (b\odot c)\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}2.11\odot 8.74\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}18.44\hspace{0.5em}.}$



Likewise, false results can be caused by careless rounding. Let us consider the numbers $a=4.98$ and $b=1.001$. Then, we have

${\displaystyle a·b\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}4.98·1.001\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}4.98498\mathrm{\hspace{0.5em}\hspace{0.5em}}\mathrm{\hspace{0.5em}\hspace{0.5em}}\text{, i.e.}\mathrm{\hspace{0.5em}\hspace{0.5em}}a\odot b\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}4.98\odot 1.001\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}4.98\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}a\hspace{0.5em}.}$

Furthermore, we have

${\displaystyle a·b^{1000}\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}4.98·1.{001}^{1000}\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}a·\underset{1000\mathrm{\hspace{0.5em}\hspace{0.5em}}\text{factors}}{\underbrace{b·\dots ·b}}\approx 13.53028118\hspace{0.5em}.}$

Rounding after each multiplication to $2$ fractional digits results (due to $a\odot b=a$) in the wrong result:

${\displaystyle (\dots ((a\odot \underset{1000\mathrm{\hspace{0.5em}\hspace{0.5em}}\text{factors}}{\underbrace{b)\odot b)\dots \odot b}})\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}a\mathrm{\hspace{0.5em}\hspace{0.5em}}=\mathrm{\hspace{0.5em}\hspace{0.5em}}4.98\hspace{0.5em}.}$