FIX: Result Differs When Comparing Single w/ Double Precision (93437)


The information in this article applies to:

  • Microsoft Visual Basic Standard Edition for Windows 2.0
  • Microsoft Visual Basic Professional Edition for Windows 2.0
This article was previously published under Q93437

SYMPTOMS

When you compare a real number stored as a Single precision variable to the same real number stored as a Double precision variable, the result may be that they are not equal. Storage of real numbers is different within the two data types. Therefore, the number may be represented differently, so a check for equivalence can return false.

WORKAROUND

In Microsoft Visual Basic version 2.0 for Windows, this problem occurs only on computers that do not have coprocessors. If your computer does not have a coprocessor, add some extra code when comparing data stored in Single data types to those stored in Double data types.

STATUS

Microsoft has confirmed this to be a problem in the Standard and Professional Editions of Microsoft Visual Basic version 2.0 for Windows. It was corrected in Microsoft Visual Basic version 3.0 for Windows.

MORE INFORMATION

Steps to Reproduce the Problem

  1. Start VB.EXE.
  2. Add the following in the Form_Click event procedure of Form1: 
          Print 12.3! = 12.3#  '** Note the '#' sign disappears.
  3. Press the F5 key to run the example, and click Form1. If the result is '0' then a coprocessor is not installed, if the result is '-1' then a coprocessor is installed or you are on a 486 with a built in coprocessor.
Note if you run this same example in Microsoft Visual Basic for Windows, version 1.0, the result is '0' with or without a coprocessor installed.
Modification Type:MajorLast Reviewed:10/30/2003
Keywords:kbbug kbfix KB93437
©2003 Microsoft Corporation. All rights reserved.