XL: Number Formatting Affects Perceived Precision (323625)
The information in this article applies to:
- Microsoft Excel 2000
- Microsoft Excel 2002
This article was previously published under Q323625 For a Microsoft Excel for Macintosh version of this article, see 181918.
SUMMARY
Microsoft Excel calculates a number based on its stored value not its
displayed value. When a formula or worksheet function performs a
calculation, Microsoft Excel uses the values in cells that are referenced
by the formula. Note that the displayed value may be different because of
number formatting.
MORE INFORMATION
When you format numbers by clicking Cells on the Format menu, you alter the way Excel displays the numbers; however, when you calculate numbers, Excel performs the calculations by using the stored values. When you calculate formatted values, you may receive unexpected results because the displayed numbers may be slightly different from the stored values.
For example, if two cells each contain the value 10.005, and the cells are
formatted to display values as currency, the value $10.01 is displayed in
each cell. If you add the two cells together, the result is $20.01 because
Microsoft Excel adds the stored values (10.005 + 10.005), not the displayed
values ($10.01 + $10.01).
If you want the calculations to be based on the displayed values, you can
change the precision of calculations so that they use the displayed values
instead of the stored values. To do this, use the Precision As Displayed
feature or make sure that the values are rounded to the same precision as
the number formatting.
Rounding Values to the Same Precision as Number Formatting
You can use the ROUND worksheet function to round values to a specific
precision.
Example:
For this example, type the following into a new worksheet:
$A$1: 5
$A$2: 200%
$A$3: =A1*A2
The value in cell $A$3 is 10.005. The value in cell $A$3 displays $10.00 if
you format the value as currency. To round the precision correctly, replace
the formula in cell $A$3 with the following:
$A$3: =ROUND(A1*A2,2)
When you do this, the worksheet function takes the result of the expression
in the first argument and rounds it to the hundredth's place (two digits to
the right of the decimal).
Rounding All Values in a Worksheet by Using Precision As Displayed
If you want the calculations to be based on the displayed values, change
the precision of calculations so that they use the displayed values instead
of the stored values. To do this, follow these steps:
- On the Tools menu, click Options, and then click the Calculation tab.
- Under Workbook Options, click to select the Precision as displayed check box.
CAUTION: When you change the precision of the calculations in a workbook by using the displayed (formatted) values, Excel permanently changes any constant values on the worksheets in the workbook. If you subsequently calculate values with full precision, Excel cannot restore the original underlying values.
REFERENCESFor more information about calculation precision, click Microsoft Excel Help on the
Help menu, type precision calculation in the Office Assistant or
the Answer Wizard, and then click Search to view the topics
returned.
Modification Type: | Major | Last Reviewed: | 10/6/2003 |
---|
Keywords: | kbhowto KB323625 |
---|
|