|
Precision - Calculation of conversions
is based on the accuracy of the factors for the 'From' and 'To' units, and that of the user
entered 'From' value. Certain factors are always treated with 15 digit accuracy regardless of
the choice of accuracy selected by the user. These include the S.I. Basis units, secondary basis
units such as cubic inch or yard, and some units which are defined as exact values. Many non-metric units other than the major imperial units of length, volume and mass are treated as general business or historic units of measure and include old English measures and figures which denote general magnitudes such mega and kilo. These are treated as exact values and are also calculated using 15 digit accuracy. Scientific Accuracy - With the exception of Basis and exact units (see above), selection of this format will force calculation based on the significant digits of factors. Absolute Accuracy - Accuracy of calculations are based on 15 significant digits for all factors. Display Formats - Scientific accuracy results in display of results in scientific format except for values with a zero exponent. Absolute accuracy results in fixed digit display format unless the number of significant digits exceeds the ability of javascript to display in which case display is converted to scientific format. Also, values with exponents > 12 or < -12 are displayed in scientific format. Display Standard - Result display follows the formats recommended by N.I.S.T. This separates values either side of a decimal point into groups of three with a space separator between groups in place of the comma commonly used. For example, instead of '123,456.7890123', this would be displayed as '123 456.789 012 3'. Exponents are displayed as 'e+nn' or 'e-nn'. These formats are compatible with Windows calculator, thus allowing users to copy values directly from the conversion calculator into the Windows calculator. Rounding Errors - Due to limitations of Javascript, some calculations may result in values with very small differences than expected. For example, a value of 4.99999999999999 instead of 5, or 10.0000000000002 instead of 10. It is left to the user to make the appropriate rounding. Uncertainty in Measurements For an excellent explanation of uncertainty in measurements and significant figures, there is none better than that at the General chemistry Online site. |