6.1. Significant Digits and Rounding

Results should only be reported to the proper number of significant digits, because the number of significant digits and associated error are indications of the precision of the analytical results. Correct handling of significant digits (and error) and retention of the available precision requires an understanding of the propagation of significance in calculations.

Generally, if not specified, the precision may be assumed to be ±1 in the last reported digit, which is termed the least significant digit. However, some values effectively have an infinite number of significant digits. For instance, 1 inch is defined as exactly 2.54 centimeters (2.54 with an infinite number of zeros following) and each value is infinitely precise for purposes of conversion. In addition, for practical purposes, many constants (speed of light, Planck's constant, etc.) are comparably precise and do not limit the precision of the results of calculations involving them.

The number of significant figures is defined as the quantity of digits in the number excluding leading or trailing zeros. For example, 3.142 has 4 significant figures;  23,459,000 has 5 significant figures; 0.31910 has 4  significant figures (the last zero does not count); and 0.0004086 has 4 significant figures (the zero between 4 and 8 is not a leading or trailing zero and so is counted). Trailing zeros are a main source of confusion, but use of scientific notation allows the writer to indicate the precision by only showing significant figures. Consider the number 2000 (which when written this way has 1 significant digit). The best way to indicate the number of significant digits is to use scientific notation:

2 x 103        1 significant digit
2.0 x 103     2 significant digits
2.00 x 103    3 significant digits

 

One should retain all digits when performing calculations and when finished, round the result to the appropriate number of significant digits. For addition and subtraction, the result should have the same number of significant digits as the least precise number in the calculation. For example,

 

14.72 + 1.4331 + 0.00235 = 16.16

In contrast, theoretically the only way to determine the correct number of significant digits for the results of calculations involving multiplication and division is to propagate significance as one would propagate error. Thus, the precision of the result cannot be better than the square root of the sum of the squares of the relative errors. For example, a measurement of 52.3 has an implied error of ±0.1, corresponding to a relative error of 0.0019. If we wished to square this value, the relative error of the result is:

e = 0.00268.

Now, 52.32 = 2735.29, so the relative error corresponds to an absolute error of:

2735.29 x 0.00268 = 7.3.

The limit on precision is thus 7.3 (rounding to 1 in the tens place), and the result should be presented as 2.74 x 101. In practice this procedure is cumbersome, and usually, unnecessary. Note that the result has the same number of significant digits as the two numbers, which were multiplied. Generally, one can simply follow the rule of rounding the result to the same number of significant figures as the least precise quantity used in the calculation. As examples, the limiting factors for some common calculations are given below: 

Calculation

Limiting Factor

Molecular Weight, Weight Fraction, Energy from Wavelength, Wavelength from Energy

smallest number of significant digits in atomic weights used and smallest number of significant digits in the conversion factor or known quantity

Spectrometer Position

smallest number of significant digits in the 2d values or spectrometer position

Absorption Coefficient

smallest number of significant digits in the elemental absorption coefficients

 

finally, when it is necessary to reduce the number of digits in a result this should be accomplished by rounding. If the number after the last significant digit is greater than 5, one should round the final digit up; if less, round down. If the digit is exactly 5, round up if the digit preceding it is odd (and down if it is even) to average out the effects of rounding.

 


Next: 6.2.1. Calculating Moles of Cations or Anions | Home: Course Overview


Copyright 1997-2003, James H. Wittke

Last update: 01/18/2006 01:47 PM.