MORE INFORMATION
Different programming languages handle conversions from floating point to
integer in different ways. C, C++, and FORTRAN programs are most likely
safe; that is, they are not likely to be affected without explicit
modification of rounding modes by the programmer. Basic programs are more
likely to be affected by the bugs.
Application programs in general are not affected. Applications should
include appropriate error checking and take appropriate actions to recover
from exceptions. To do this, applications need to do explicit range
checking prior to conversion or implement explicit exception handling. If
an application relies on explicit exception handling, the possibility of
encountering the bug exists. Applications written in languages that use
exception handling will most likely use range checking, thereby allowing
the application to be compiled with a no-check option for performance
reasons.
FIST Overview
The FIST instruction is used to convert floating point numbers to signed
integers. For example, in a C or FORTRAN program, when an integer variable
is assigned a floating point value, that value must be truncated to an
integer before being stored. Other conversions from floating point to
integer are also possible; for example, Basic and Pascal round instead of
truncating the fractional part.
The FIST instruction operates in one of four rounding modes: chop (also
known as truncate), nearest (nearest or even), up, and down. The default
type conversions done by C and FORTRAN use the chop rounding mode, whereas
Basic uses nearest. The processor indicates whether the input operand is an
integer by means of a precision exception (PE) flag. The PE flag is set to
true if the number to be converted was not already an integer; that is, it
had a fractional part. It is unchanged otherwise. The effect of rounding is
shown in the rounding direction( C1) flag. If an input value is rounded to
a number with a greater magnitude, the C1 flag is set; otherwise, it is
cleared.
FIST converts floating point numbers to 16, 32, or 64-bit signed integers.
Because the range of a floating point number is larger than any of these
formats, some floating point numbers cannot be converted to integers. For
example, the largest 16-bit integer is 32767, so an attempt to convert
32768.5 to a 16-bit integer results in an exception (in this case, the
invalid operation exception (IE)). The processing of an exception is
controlled by the exception mask in the floating point control register. If
an invalid operation exception mask is set, the exception is masked,
otherwise, it is unmasked.
For example, applications running under Microsoft Windows version 3.1 begin
with all floating point exceptions masked. Different applications may
change the exception masks and provide their own methods for handling
exceptions. For example Microsoft Visual C++ sets the invalid operation
exception to unmasked, while leaving the precision exception masked. An
application developed in C++ will see a pop-up window appear when a FIST
instruction attempts to convert a number that is out of bounds, unless the
exception handling is changed.
Undetected Overflow
A bug has been identified by Intel in FIST instructions that convert a
floating point number to either a 16- or 32-bit signed integer. The FISTP
instruction that stores to a 64-bit integer is not affected. The error
occurs when a rounding mode is set to "nearest" or "up" for one limited
range of floating point values in the out of bounds region. For this range,
regardless of the exception mask, the value 0 is written to memory, the PE
flag is set, the IE flag is not set, and no IE exception is raised.
In the figures below, floating point values from A and B can be converted
to integers. Values outside this range should raise an exception. This bug
affects values in the C and D range only.
16-Bit FIST
65535.5 -
-32768.5 0 32767.5 65536.0
<-------------[--------|--------]-------------------|-------->
overflow Normal Range overflow
A B C
32-Bit FIST
4,294,967,295.5 -
-2,147,483,648.5 0 2,147,483,647.5 4,294,967,296.0
<-------------[------------|------------]-----------------------|-------->
overflow Normal Range overflow
A B D
For example, take a number outside the range of a 32-bit signed integer,
such as 4,294,967,295.7. This is rounded mathematically to 4,294,967,296.
Due to the bug, the Pentium processor does not raise an exception to this
number. It simply writes a 0 into memory and does not transfer control to a
handler or raise the IE flag.
Applications that Use Microsoft Visual Basic
Microsoft Visual Basic automatically promotes data types to handle mixed
type operations. This avoids the overflow condition for many cases. For
example:
IntX = SngY*IntZ
IntX = 65535.7*2
However, implicit conversions may be affected. The following Visual Basic
version 3.0 code produces different results depending on the processor:
Dim X As Integer
Dim Y As Single
Y = 65535.5
X = Y
MsgBox Cstr(X)
The above code should result in a run-time error 6 "Overflow" message. On a
Pentium processor-based machine, this code results in a return of zero (0).
A simple way to avoid this error is to include a range check in your code.
For example:
If (x)>=65535.5
Then Error 6
The only programs affected would be those that explicitly trap error 6
(overflow) and involve either explicit assignment of a floating-point
expression to an Integer data type or a Long data type or explicit
conversion of a floating point expression using CInt() or CLng().
Programs that employ explicit bounds checking to prevent overflow
conditions should not be affected.
If bounds checking adversely affects performance in a loop, migrating the
code to a C language dynamic-link library (DLL) may be a better option.
Program Results
At the assembly level, the recommended workaround is the insertion of the
FRNDINT (Floating Point Round to Integer) instruction immediately preceding
the FIST instruction. FRNDINT will correctly round the floating point value
before executing the FIST instruction (as opposed to FISTP); the correction
will also need to preserve its input. Future versions of Visual Basic will
include this fix.
NOTE: Any method that forces the processor to emulate floating point
instructions will avoid the problem, at the cost of reduced performance and
speed.
Rounding Mode Errors
For six specific operands, the results of the FIST instructions are not as
expected. There are flag differences in all four rounding modes. There is
also the possibility of an incorrect number being stored in the "up" and
"down" rounding modes. In the "nearest" and "chop" rounding modes, which
are most frequently used, the value stored to memory is correct. This bug
affects the 16-, 32-, and 64-bit variants of the instruction.
The following table shows the affected numbers, rounding modes, and
expected and actual values:
Operand | Rounding | Exp. | Actual | Exp. | Actual | Exp. | Actual
(any one of) | mode | mode | mode | PE | PE | C1 | C1
-----------------------------------------------------------------------
1/16 (0.0625) | nearest | 0 | 0 | 1 | unch. | 0 | 0
1/8 (0.125) | chop | 0 | 0 | 1 | unch. | 0 | 0
3/16 (0.1875) | down | 0 | 0 | 1 | unch. | 0 | 0
| up | 1 | 0 | 1 | unch. | 1 | 0
-1/16 (-0.0625)| nearest | 0 | 0 | 1 | unch. | 0 | 0
-1/8 (-0.125) | chop | 0 | 0 | 1 | unch. | 0 | 0
-3/16 (-0.1875)| down | -1 | 0 | 1 | unch. | 1 | 0
| up | 0 | 0 | 1 | unch. | 0 | 0
For six numbers (1/16, 1/8, 3/16, -1/16, -1/8, and -3/16) incorrect results
can occur. In all rounding modes, the PE flag is not set consistently with
the rounding that occurred; it remains unchanged. For the three positive
numbers in the "up" rounding mode, FIST stores an incorrect value (0
instead of 1); similarly, for the three negative numbers in the "down"
rounding mode, FIST gives an incorrect result (0 instead of -1). In each of
these cases, the C1 bit is also not set correctly. However, in the
"nearest" and "chop" rounding modes, the value stored to memory is always
correct, as is the C1 bit.
Program Results
Incorrect results from the second bug are only returned when the rounding
mode is set to "up" or "down." Therefore, it should not affect an
application unless the application explicitly changes the rounding mode
from the language default. If an application changes to "up" or "down"
rounding mode, it can no longer rely on language-provided floating-point-
value-to-integer-number conversion facilities and must provide its own by,
for example, using FRNDINT.
The incorrect status information in the PE flag, affecting all rounding
modes, should be insignificant in most applications. If rounding occurs,
the PE flag is already set. The PE exception is also typically masked. The
C1 bit is only used by an exception handler, so the incorrect values are
insignificant.
Workaround
At the assembly level the recommended workaround for the second FIST bug is
the same for the first; inserting the FRNDINT instruction immediately
preceding the FIST instruction.
Application programmers can avoid rounding errors in the second bug by not
overriding the default rounding modes.
NOTE: Any method that forces the processor to emulate floating point
instructions will avoid both problems, at the cost of reduced performance
and speed.