Calculating Stereo Pairs

Introduction

The following discusses computer based generation of stereo pairs as used to create a perception of depth. Such depth perception can be useful in many fields, for example, scientific visualisation, entertainment, games, appreciation of architectural spaces, etc.

Depth cues

There are a number of cues that the human visual system uses that result in a perception of depth. Some of these are present even in two dimensional images, for example:

• Perspective. Objects get smaller the further away they are.
  
• Sizes of known objects. We expect certain object to be smaller than others. If an elephant and a tea cup appear the same size then we expect the elephant to be further away.
  
• Occlusion. An object that blocks another is assumed to be in the foreground.
  
• Lighting, shadows. There a number of subtle cues implied by lighting, the way a curved surface reflects light suggests the rate of curvature, shadows are a form of occlusion.
• Relative motion. Objects further away seem to move more slowly than objects in the foreground.
  

There are other cues that are not present in 2D images, they are:

• Binocular disparity. This is the difference in the images projected onto the back the eye (and then onto the visual cortex) because the eyes are separated horizontally by the interocular distance.
  
• Accommodation. This is the muscle tension needed to change the focal length of the eye lens in order to focus at a particular depth.
  
• Convergence. This is the muscle tension required to rotate each eye so that it is facing the focal point.
  

While binocular disparity is considered the dominant depth cue in most people, if the other cues are presented incorrectly they can have a strong detrimental effect. In order to render a stereo pair one needs to create two images, one for each eye in such a way that when independently viewed they will present an acceptable image to the visual cortex and it will fuse the images and extract the depth information as it does in normal viewing.

Stereographics using stereo pairs is one of the major three dimensional display technologies. Stereo pairs create a "virtual" three dimensional image, binocular disparity and convergence cues are correct but accommodation cues are inconsistent because each eye is looking at a flat image. Note: there are 3D display systems where accommodation is not in conflict, these are generally referred to as autostereoscopic because in general they do not need additional viewing apparatus. Examples of autostereoscopic systems are displays designed around the same principles as lenticular sheets or holograms.

The case where the object is behind the projection plane is illustrated below. The projection for the left eye is on the left and the projection for the right eye is on the right, the distance between the left and right eye projections is called the horizontal parallax. Since the projections are on the same side as the respective eyes, it is called a positive parallax. Note that the maximum positive parallax occurs when the object is at infinity, at this point the horizontal parallax is equal to the interocular distance.

If an object is located in front of the projection plane then the projection for the left eye is on the right and the projection for the right eye is on the left. This is known as negative horizontal parallax. Note that a negative horizontal parallax equal to the interocular distance occurs when the object is half way between the projection plane and the center of the eyes. As the object moves closer to the viewer the negative horizontal parallax increases to infinity.

If an object lies at the center of the projection plane then its projection onto the focal plane is coincident for both the left and right eye.

Rendering

There are a couple of methods of setting up a virtual camera and rendering two stereo pairs, many methods are strictly incorrect since they introduce vertical parallax. An example of this is called the "Toe-in" method, while incorrect it is still often used because the correct "off axis" method requires features not always supported by rendering packages.

In this projection the camera has a fixed and symmetric aperture, each camera is pointed at a single focal point. Images created using the "toe-in" method will still appear stereoscopic but the vertical parallax it introduces will cause discomfort. The introduced vertical parallax increases out from the center of the projection plane and is more important as the camera aperture increases.

Off-axis (Correct)

This is the correct way to create stereo pairs. It introduces no vertical parallax and is therefore creates the less stressful stereo pairs. Note that it requires a non symmetric camera frustum, this is supported by some rendering packages, in particular, OpenGL.

Objects that lie in front of the projection plane will appear to be in front of the computer screen, objects that are behind the projection plane will appear to be "into" the screen. It is generally easier to view stereo pairs of objects that recede into the screen, to achieve this one would place the focal point closer to the camera than the objects of interest. Note, this doesn't lead to as dramatic an effect as objects that pop out of the screen.

The degree of the stereo effect depends on both the distance of the camera to the projection plane and the separation of the left and right camera. Too large a separation can be hard to resolve and is known as hyperstereo. A good ballpark separation of the cameras is 1/15 of the distance to the projection plane, this is generally the maximum separation for comfortable viewing. Another constraint in general practice is to ensure the negative parallax (projection plane behind the object) does not exceed the eye separation.
A common measure is the parallax angle defined as P = 2 atan(DX / (2 d)) where DX is the horizontal separation of a projected point between the two eyes and d is the distance of the eye from the projection plane. For easy fusing by the majority of people, the absolute value of P should not exceed 1.5 degrees for all points in the scene. Note P is positive for points behind the scene and negative for points infront of the screen. It is not uncommon to restrict the negative value of P to some value closer to zero since negative parallax is more difficult to fuse especially when objects cut the boundary of the projection plane.

Viewing

Most stereo pair techniques require some sort of apparatus to present the appropriate image to each eye and are thus not "autostereoscopic". The most common methods in use today are LCD shutter glasses or glasses with polarised lens.
The projection of stereo pairs can be categorised as either time parallel where both image are present simultaneously, or time multiplexed where images are alternatively drawn.
The examples and the system discussed in this report use time multiplexed display with LCD shutters synchronised to the alternating images. The following image shows a "standard" arrangement, the computer has a stereo capable video/OpenGL card that has a refresh rate of 120Hz. The stereo pairs are displayed on each alternate refresh, 60Hz per image. The emitter (on the top of the monitor) sends a signal that is inphase with the monitor and the LCD shutter glasses alternatively switch the lenses transparent and opaque. The net effect is the left eye only sees the left stereo pair, the right eye only sees the right stereo pair.

References

Baker, J.
Generating images for a time multiplexed stereoscopic computer graphics system.
True 3D imaging techniques and display technologies
McAllister, D.F. and Robbins, W.E. (Editors)
SPIE Proc 761, 1987.

Bos, P. et al
High performance 3D viewing system using passive glasses
1988 SID Digest of technical papers, vol 19

Bos, P. et al
A liquid crystal optical switching device (pi cell)
1983 SID Digest of technical papers, vol 15

Grotch, S.L.
Three dimensional and stereoscopic graphics for scientific data display and analysis
IEEE Computer Graphics and Applications 3,8, Nov 1983, 31-34

Hodges, L.F. and McAllister, D.F.
Stereo and alternating pair techniques for display of computer generated images
IEEE Computer Graphics and Applications 5,9, September 1985, 38-45

Lipton, L, Meyer, L
A flicker free field sequential stereoscopic video system
SMPTE Journal, November 1984, pp1047.

Lipton, L., Ackerman.,M
Liquid crystal shutter system for stereoscopic and other applications
US Patent No 4,967,268, Oct 30, 1990

Nakagawa, K., Tsubota, K., Yamamoto, K
Virtual Stereoscopic display system
US Patent No. 4,870,486, Sept 26, 1989

Roese, J.A. and McCleary, L.E.
Stereoscopic computer graphics for simulation and modeling
Proc SIGGRAPH 13, 2, 1979, 41-47

Course notes, #24
Stereographics
ACM SIGGRAPH, 1989

Weissman, M.
3D Measurements from Video Stereo Pairs
SPIE Three Dimensional Imaginf and Remote Sensing Imaging, Vol 902, pp 85