OFF97: Rounding Errors in Visual Basic For Applications (165373)



The information in this article applies to:

  • Microsoft Word 97 for Windows
  • Microsoft PowerPoint 97 for Windows
  • Microsoft Excel 97 for Windows
  • Microsoft Office 97 for Windows

This article was previously published under Q165373

SYMPTOMS

When you run a Visual Basic for Applications macro that performs numeric calculations that result in a decimal value, the result that is returned may be incorrect. For example, for the following calculation

n = 301.84 - 301

Visual Basic for Applications returns 0.839999999999975. The correct result is 0.84.

WORKAROUND

Microsoft provides examples of Visual Basic for Applications procedures for illustration only, without warranty either expressed or implied, including, but not limited to the implied warranties of merchantability and/or fitness for a particular purpose. The Visual Basic procedures in this article are provided 'as is' and Microsoft does not guarantee that they can be used in all situations. While Microsoft support professionals can help explain the functionality of a particular macro, they will not modify these examples to provide added functionality, nor will they help you construct macros to meet your specific needs. If you have limited programming experience, you may want to consult one of the Microsoft Solution Providers. Solution Providers offer a wide range of fee-based services, including creating custom macros. For more information about Microsoft Solution Providers, call Microsoft Customer Information Service at (800) 426-9400.

To return the correct result rounded to a specific decimal place, use the Format function.

The following example macro returns the correct result (.84) for the calculation in this article:
   n = CDbl(Format(301.84 - 301, "0.00"))
				
For more information about using the Format function, click the Office Assistant in the Visual Basic Editor, type Format Function, click Search, and then click to view the "Format Function" topic.

NOTE: If the Assistant is hidden, click the Office Assistant button on the Standard toolbar. If Visual Basic for Applications Help is not installed on your computer, please see the following article in the Microsoft Knowledge Base:

120802 Office: How to Add/Remove a Single Office Program or Component

STATUS

Microsoft is researching this problem and will post new information here in the Microsoft Knowledge Base as it becomes available.

MORE INFORMATION

For additional information, please see the following articles in the Microsoft Knowledge Base:

126455 How to Avoid Rounding & Overflow Probs on Pentium Processors

42980 (Complete) Tutorial to Understand IEEE Floating-Point Errors

69333 How to Work Around Floating-Point Accuracy/Comparison Problems


Modification Type:MinorLast Reviewed:9/13/2006
Keywords:kbProgramming KB165373