D    Digital AXPvme Single-Board Computers and Digital Alpha VME 2100 Systems

The information in this appendix applies to running the Digital UNIX software on the following Digital Alpha VME systems:

• Digital AXPvme 64

• Digital AXPvme 160

• Digital AXPvme 100

• Digital AXPvme 166

• Digital AXPvme 230

• Digital Alpha VME 2100

Refer to the Release Notes and the Installation Guide for further information on installing the operating software.

This appendix contains information on the following topics:

• Configuration instructions

• VMEbus slave block transfers

• VMEbus master block transfers with local DMA

• Realtime interrupt-handling function

• VMEbus backplane (VB) network driver

D.1    Configuration Instructions

This section describes how to configure the AXPvme Single-Board Computer (SBC) and Digital Alpha VME 2100 systems for use on the VMEbus.

D.1.1    Configuring the VMEbus Adapter

You configure each VMEbus adapter separately by editing the ebv10_vme_data.c file on each machine. After editing this file, you must issue a doconfig command on each machine.

Use the following structure to customize various VMEbus parameters. The default values specified should provide proper VMEbus operation for most applications. Be careful when modifying these values; not all adapters support all fields.

struct    ebv10_vme_data  ebv10_vme_data  =  {
    VME_BR3,            /* Bus Request Level for master cycles             */
    VME_ARB_RR,         /* Arbitration mode                                */
    VME_FRTO_DSABL,     /* VMEbus fair requester disabled                  */
    VME_BUSTO_256US,    /* Local bus acquisition timeout                   */
    VME_BUSTO_512US,    /* VMEbus Arbitration Timeout                      */
    VME_ROR,            /* Release mode                                    */
    VME_SYS_CONTROLLER, /* System Controller                               */
    VME_A24_A32_OVRLAP, /* Inbound A24/A32, if same space, overlap         */
    VME_WRT_POST_DSABL, /* Disable VIC master write posting                */
    0x08000000,         /* A32 Inbound DMA window base address             */
    VME_SZ_128_MB,      /* A32 window sz in kb (131072kb  =  128mb)        */
    0x00C00000,         /* A24 Inbound DMA window base address             */
    VME_SZ_4_MB,        /* A24 window size in kb (4096kb  =  4mb)          */
    0x00000100,         /* A16 Interprocessor Communication base           */
    0x00000000,         /* A16 Interprocessor Communication mask           */
                        /* The following entries pertain to VME adapters   */
                        /* that are not embedded SBC VME designs. Embedded */
                        /* SBC VME designs pass the following information  */
                        /* via another mechanism.                          */
    SPLDEVLOW,          /* VME IRQ level to system SPL map                 */
    SPLDEVLOW,          /* VME IRQ 1 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 2 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 3 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 4 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 5 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 6 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 7 to SPL spldevlow                      */
    SPLDEVLOW,          /* Adapter resource blocking SPL spldevlow         */
    0                   /* Master DMA resource selection                   */
                        /*   sparse = 0 - dense = VME_DENSE                */
};

You can specify values for the following VMEbus parameters:

• VMEbus request level

• VIC arbitration mode

• VMEbus fairness timer

• Local bus acquisition and VMEbus arbitration timeouts

• VMEbus release mode

• VMEbus system controller

• VIC master write posting enable/disable

You can also specify values for the following:

• A32/A24 overlap/unique address configuration

• A32/A24 DMA inbound window base and size

• A16 base for interprocessor communication facilities

The following sections discuss these parameters in detail.

D.1.2    Specifying the VMEbus Request Level

You can specify one of the values shown in Table D-1 for the VMEbus request level (vme_brl). The value is stored in the VIC64 Arbiter/Requester Configuration Register (ARCR).

Table D-1: VMEbus Request Level

D.1.3    Specifying the VIC Arbitration Mode

You can specify one of the values shown in Table D-2 for the VMEbus arbitration mode. The VMEbus arbitration mode (arb mode) is stored in the VIC64 Arbiter/Requester Configuration Register (ARCR). This parameter is applicable only when the VME adapter is configured to be the system controller. (See Section D.1.7 for more information.)

Table D-2: VIC Arbitration Mode

D.1.4    Specifying the VMEbus Fairness Timer

You can specify one of the values shown in Table D-3 for the Arbiter/Requester fair request timeout. The fair request timeout (fairness_to) is stored in the VIC64 Arbiter/Requester Configuration Register (ARCR).

Table D-3: VMEbus Fairness Timer

D.1.5    Specifying the Local Bus Acquisition and VMEbus Arbitration Timeouts

You can specify one of the values shown in Table D-4 for the local bus acquisition timeout and for the VMEbus arbitration timeout. The local bus acquisition timeout (local_bus_to) is specified first, followed by the VMEbus arbitration timeout (vmebus_arb_to). The value is stored in the VIC64 Transfer Timeout Register (TTR). The local bus timeout must be less than the VMEbus arbitration timeout.

Table D-4: Local Bus/VMEbus Acquisition Timeout Values

D.1.6    Specifying the VMEbus Release Mode

You can specify one of the values shown in Table D-5 for the release mode. The release mode (rel_mode) value is stored in the VIC64 Release Control Register (RCR).

Table D-5: VMEbus Release Mode

D.1.7    Specifying the VMEbus System Controller

You can specify one of the values shown in Table D-6 to indicate whether or not the VME adapter is the system controller.

In addition to specifying a value (syscon) from Table D-6, the AXPvme SBC configuration switches must be set accordingly to indicate that it is or is not the VMEbus system controller. See the AXPvme Single-Board Computer Installation Guide for information on setting the AXPvme module configuration switches.

The Digital VME 2100 adapter is always the VMEbus system controller. There are no module configuration switches to disable it from being the system controller.

There must be one and only one system controller in the VMEbus backplane. The system controller must be electrically the first module in the VMEbus backplane and in most systems must be in the first VMEbus slot.

Table D-6: VMEbus System Controller

The values specified interact with the VMEbus initialization code to determine whether a VMEbus reset is issued when the VME adapter is being configured. If the value is set to VME_SYS_CONTROLLER and the system being booted is the system controller, as determined by the VMEbus initialization code, a VMEbus reset is issued. If you do not want a VMEbus reset issued during VME adapter configuration, set the value to VME_NOSYS_CONTROLLER. These values pertain only to the system controller.

If the system controller is configured to issue a VMEbus reset during adapter initialization, and other processor modules are installed in the VMEbus backplane, boot the system controller first to allow devices and processor modules to perform their bus reset actions.

D.1.8    Receipt of VMEbus Resets

The system controller should always be the initiator of VMEbus resets. However, under certain error conditions, other VME adapter modules may invoke a VMEbus reset. Modules installed in the VME backplane react to bus resets differently. Some modules, if configured, perform a module reset. Some VME adapters may have their VME interface reset to a powerup state without notification to the operating system. This could leave the VME adapter in an unconfigured state, cause unwanted effects to the operating system and its device drivers, and cause VMEbus errors to occur. Other VME adapters on the VMEbus may accept VMEbus resets and attempt to reconfigure the VMEbus adapter to the hardware context it was running before the bus reset occurred. However, device drivers expecting interrupts may not receive them and I/O hardware operations may be canceled by the VMEbus reset without notification to the device driver. There is also a potential for data corruption to occur when the VME adapter is reset during an I/O operation.

Digital recommends that the system controller be the initiator of VMEbus resets during adapter initialization. If the system controller is not controlled by a processor, then a powerup sequence should cause all VMEbus adapters and devices to be reset. All modules on the VMEbus should perform a module reset upon detection of a bus reset. VME adapters that are not the system controller and that are running an operating system should be shut down in an orderly fashion prior to the system controller being booted. These VME adapters should be rebooted after the system controller has been booted, providing the system controller is to be utilized and controlled by a processor.

D.1.8.1    Receipt of VMEbus Resets on Digital VME 2100 Systems

The VME adapter can be the initiator of VMEbus resets only. Upon receipt of a bus reset, its VME interface (VIC64) is reset. The reset state of the VME interface (VIC64) is not the VME adapter's configured state. The operating system and device drivers are not notified that a bus reset has occurred. If accessed or an I/O operation is invoked following a bus reset, the access may result in an adapter error, misoperation, or a system crash.

D.1.8.2    Receipt of VMEbus Resets on Digital AXPvme SBC Systems

Digital recommends that nodes that are not the system controller have their AXPvme module configuration switch 3 set to Closed (resets the AXPvme module on VMEbus reset signal). When the VMEbus is reset, and the module switch is set to accept a VMEbus reset, nonsystem controller modules take a boot action and are reset to a powered state.

If the AXPvme module configuration switch 3 is set to Open (does not reset the AXPvme module on VMEbus reset signal), the VMEbus adapter software will receive a VMEbus reset interrupt upon detection of a bus reset. The VMEbus reset signal initializes the VMEbus adapter (VIC64) to its powerup state. The VMEbus reset interrupt service interface displays the following message on the console terminal:

vba0  reset_inter:  VMEbus  reset  detected

The interrupt service interface then initializes the VMEbus adapter to its defaults and enables any previously enabled interrupt enable bits.

Do not set the AXPvme module configuration switch 3 to Open without considering the following side effects of receiving a VMEbus reset: device drivers expecting interrupts may not receive them and I/O hardware operations may be canceled by the VMEbus reset without notification to the device drivers. There is potential risk of data corruption depending upon I/O activity at the time a bus reset occurred.

D.1.9    Setting VMEbus Master Write Posting

Master write posting is not currently supported. Do not change the value from the default of VME_WRT_POST_DSABL or unpredictable results may occur.

D.1.10    Setting VMEbus Address Space Parameters

The VMEbus hardware architecture includes three address spaces:

• 32-bit (A32)

• 24-bit (A24)

• 16-bit (A16)

The following sections provide information on specifying size and segmentation for windows within these address spaces.

D.1.10.1    A32 and A24 Address Spaces

The VMEbus 32-bit address space (A32) and 24-bit address space (A24) are used for direct memory access (DMA) inbound windows.

The A32 space has a maximum size of 4 GB and can be partitioned into 32 128 MB windows. You can further partition each 128 MB window in increments as small as 16 MB. Valid window segments are 16, 32, and 64 MB.

The A24 space has a maximum size of 16 MB, and the base address is always zero. This means that you can partition the address space but cannot move it. The default window size is 4 MB and the base address for a window must be a multiple of the window size. The default inbound window is the top 4 MB out of the 16 MB space.

You can specify whether the A24 and A32 addresses can reside within the same addressing range or whether they must be unique.

D.1.10.1.1    A32 and A24 Address Space Overlapping

Read this section if the A32 direct memory access (DMA) inbound window will be configured with a base address of zero (0x00000000), overlapping the A24 address space. A24 inbound windows are always configured within the first 16 MB of the 4 GB VMEbus address space.

Typically, VMEbus A24 and A32 address spaces overlap each other such that addresses in each address space are unique to that address space. As an example, address 0x200 in A32 address space is not the same address as 0x200 in A24 address space. This is the default configuration and selected by the value VME_A24_A32_OVRLAP.

You can configure some VMEbus devices to recognize the same VMEbus address in both A24 and A32 address spaces. These devices treat the two address spaces as a single entity.

Consult the VMEbus hardware device manuals to determine if any devices installed on the VMEbus follow this model. If so, the autoconfiguration software must be configured to disallow A32 direct memory access allocations within the first 16 MB of VMEbus address space. If this is not done, an A32 direct memory access to the first 16 MB of VMEbus address space by another VMEbus device may not only select the AXPvme module but also select the device that treats the address spaces as a single entity.

Configure the first 16 MB of VMEbus address space as a single entity by specifying the value VME_A24_A32_UNIQUE.

Table D-7 shows the values for overlapping and unique address spaces. These values are valid only when the A32 and A24 address spaces are configured to overlap each other; that is, when the A32 base address equals zero.

Table D-7: Address Space Overlapping

D.1.10.1.2    Configuring Window Size for A32 and A24 Address Spaces

You can specify the DMA inbound window size for the A32 and A24 address spaces. Note that the A32 base address and window size is specified first, followed by the A24 base address and window size.

If you specify an invalid base address in relation to a window size, the autoconfiguration code adjusts the base address to match the window size. The base address is adjusted downward to the next appropriate boundary for the window size.

Table D-8 lists the window size values.

Table D-8: Window Size

D.1.10.1.3    Configuring Base Address for A32 Address Spaces

Table D-9 shows the values used for partitioning 128 MB windows in the A32 address space. Note that the base value is contained in bits 24 through 31, with bits 27 through 31 indicating the window and bits 24 through 26 indicating the partition size.

Table D-9: A32 Address Space Base Addresses

D.1.10.1.4    Configuring Base Address for A24 Address Spaces

Table D-10 shows the base address values for windows in the A24 address space. The base address is stored in bits 16 through 23. The table has been truncated for the smaller window sizes.

Table D-10: A24 Address Space Base Addresses

D.1.10.2    Configuring the A16 Interprocessor Communication Facilities Address Space

The VMEbus 16-bit address space (A16) is used for interprocessor communication and to communicate with A16 VMEbus devices. The A16 space has a maximum size of 64 KB and runs from VMEbus address 0000 hex to FFFF hex. You can configure the VMEbus Interprocessor Communication Facilities (ICF) of the AXPvme SBC or Digital VME 2100 on any 256-byte boundary within the VMEbus A16 address space. The default base address is 0x00000100. The mask value must be left at zero.

D.1.11    VMEbus and VME Adapter Interrupt Request Levels

VMEbus and VME adapter interrupt requests are delivered to the operating system and device drivers at system SPL levels from the following table:

Interrupt	Digital AXPvme	Digital VME 2100
Request Name	SBC SPL Levels	SPL Levels
VMEbus IRQ 1	SPLDEVLOW	SPLDEVLOW
VMEbus IRQ 2	SPLDEVLOW	SPLDEVLOW
VMEbus IRQ 3	SPLDEVLOW	SPLDEVLOW
VMEbus IRQ 4	SPLDEVHIGH	SPLDEVLOW
VMEbus IRQ 5	SPLDEVHIGH	SPLDEVLOW
VMEbus IRQ 6	SPLDEVHIGH	SPLDEVLOW
VMEbus IRQ 7	SPLDEVRT	SPLDEVLOW
Autovector IRQ 1	SPLDEVLOW
Autovector IRQ 2	SPLDEVLOW
Autovector IRQ 3	SPLDEVLOW
Autovector IRQ 4	SPLDEVHIGH
Autovector IRQ 5	SPLDEVHIGH
Autovector IRQ 6	SPLDEVHIGH
Autovector IRQ 7	SPLDEVRT
VMEbus Reset	SPLDEVRT
Module Switches	SPLDEVRT	SPLDEVLOW
VME IACK	SPLDEVLOW	SPLDEVLOW
DMA Status	SPLDEVRT	SPLDEVLOW

The Digital VME 2100 system does not support autovector and VMEbus reset interrupt requests.

As the previous table indicates, the Digital AXPvme Single-Board Computers (SBCs) have interrupts that can be preempted by higher-level interrupt requests, while the Digital VME 2100 interrupt requests are all delivered at the same SPL level and cannot be preempted.

On the Digital AXPvme SBC, device drivers must use the rt_post_callout function for interrupts delivered at SPLDEVRT. Interrupt requests for which this is needed are VMEbus IRQ7, Autovector IRQ7, and any of the four module switch interrupts. Device drivers written for the Digital AXPvme VME that use the rt_post_callout function will also run on the Digital VME 2100 system without modifications.

Note

VME device drivers written for Digital VME 2100 systems, and other platforms that deliver VME interrupts at the same SPL level, may be affected when running these device drivers on the Digital AXPvme SBC platform. If these device drivers are using SPL levels to protect common resources between thread and interrupt service interfaces, the preempted interrupts of the Digital AXPvme SBC systems may have unwanted effects on these device drivers. If these device drivers are servicing interrupts for VMEbus IRQ7, Autovector IRQ7, or module switch interrupts, then these device drivers must be modified to use the rt_post_callout function. Device drivers cannot invoke normal thread wakeup mechanisms at SPLDEVRT.

D.1.12    Setting VMEbus Interrupt Vector Parameters

Refer to the Autoconfiguration Support section of this book for an example of adding and enabling VME interrupts. Refer to the vme_handler_info structure in this book for interrupt handler information.

The following sections discuss how to specify vectors and interrupt requests (IRQs) for a device driver. The configuration file uses the keywords vector and priority.

Device drivers are passed this information in the controller structure elements ivnum and bus_priority.

D.1.12.1    Specifying VMEbus Interrupt Vectors and Interrupt Request Levels

VMEbus interrupt vectors 24 to 255 are available to device drivers. Vectors 0 to 23 are reserved by the VME adapter. When you specify a vector, you must also use the priority keyword to specify an IRQ. Valid IRQ specifications are values 1 through 7. These values correspond to VMEbus levels IRQ1 through IRQ7.

Note that if a VMEbus device uses an IRQ, that same IRQ cannot be used for autovectored interrupts.

D.1.12.2    Specifying Autovector Interrupt Vectors

The Digital VME 2100 system does not support autovectors.

VMEbus devices of the type Release of Register Access (RORA) use autovectors. RORA devices do not have the capability of presenting a status/ID vector as do the Release On Acknowledge (ROAK) VMEbus devices.

RORA devices present an interrupt request to the system at a specified VMEbus IRQ level. Upon receipt of the interrupt request, the system provides a system-defined status/ID vector and dispatches it to the interrupt service interface installed for the autovector. It is the responsibility of the device driver to dismiss the RORA device's interrupt request by performing a read or write access to the device. Refer to the hardware manual for the RORA device to determine what type of access is needed to dismiss the interrupt request.

To select an autovector, use the priority keyword to specify a vector value of 0 and an IRQ value of 1 through 7 corresponding to VMEbus levels IRQ1 through IRQ7.

If an IRQ is used for an autovector, the same IRQ cannot be used for VMEbus interrupt vectors.

D.1.12.3    Specifying Module Switch Interrupt Vectors

Select one of the following vectors for the module switch interrupt you want. Use the priority keyword to specify 7 as the IRQ level.

0x1140 - module switch 0 interrupt
0x1150 - module switch 1 interrupt
0x1160 - module switch 2 interrupt
0x1170 - module switch 3 interrupt

Module switch interrupt vectors allow a module to issue an interrupt to itself or to another module. The autoconfiguration software provides control status registers (CSRs) for use in module switch interrupts. You can specify two CSRs on the configuration line. At boot time, the system searches for the specified CSRs.

The autoconfiguration software performs the appropriate bus mapping and provides an io_handle_t in the addr and addr2 controller arguments. The addr argument is passed to the driver's probe interface, while the addr2 value must be obtained from the addr2 controller argument.

For example, the following controller structure specifies a CSR for the base address of the A16 Interprocessor Communication Facilities (ICF). Module switch 0 CSR is an offset from this A16 address.

controller xyz at vba0 csr 0x100 vector modsw0_isr 0x1140 priority 7

The driver structure allows you to specify the size, address type, and swap mode for the CSRs. For example, the following members in a driver structure indicate that the first CSR has a size of 256 bytes, is in the A16 address space, and is set to noswap mode:

int     addr1_size      256
int     addr1_atype     VME_A16_SUPER_ACC | VME_BS_NOSWAP

For more information, see Writing Device Drivers: Tutorial, especially the sections on the addr and addr2 members of the controller structure and on the addr1_size, addr1_atype, addr2_size, and addr2_atype members of the driver structure.

In addition, you can use the vba_map_csr function to provide module switch interrupts. After using the vba_map_csr function to create an I/O handle, you write to an address derived from the base address plus an offset. Two write operations are performed, one signifying a clear and one a set. The following code fragment shows how the I/O handle is created:

io_handle_t	ioh;           /* define type of ioh */
vme_addr_t		A16base=0x100; /* base CSR address */
ioh = vba_map_csr(ctlr, A16base, 256,
                 (VME_A16_SUPER_ACC |
                  VME_BS_NOSWAP));

The following code fragment shows how the module switch interrupts are issued:

write_io_port(ioh+0x20, 1, 0, 0)  /* write to A16 base address
                                     plus the offset to clear
                                     module switch 0 */
mb();
write_io_port(ioh+0x21, 1, 0, 0)  /* write to A16 base address
                                     plus the offset to set
                                     module switch 0 */
mb();

D.1.12.4    Specifying Global Switch Interrupt Vectors

Global switch interrupts are not currently supported.

D.1.13    Using VMEbus Byte-Swap Modes

Digital processors are little endian, while VMEbus is big endian. Default mode or VME_BS_NOSWAP causes the transfer of bytes between Digital processors and VMEbus to be arranged correctly. If, however, a 16-bit or 32-bit number is needed in a VMEbus register, the noswap mode rearranges the bytes within the transfer such that the bytes are reversed in significance. Two other modes are provided to handle these situations: VME_BS_BYTE and VME_BS_LWORD. A third mode for swapping words within longwords, VME_BS_WORD, is not portable across VME adapters and is provided for convenience. The definitions for these modes are in the io/dec/vme/vbareg.h file. The flags for these modes are used in vba_map_csr, dma_map_alloc/load, and the driver structure.

D.1.13.1    VME_BS_NOSWAP Mode

VME_BS_NOSWAP mode provides a hardware mechanism for data coherency for byte-data transfers from Digital (little endian) to VMEbus (big endian) processors. The address of any byte as seen on the two busses remains the same. Block transfers of byte information use 16- or 32-bit transfers. The transfer sizes are 8-, 16-, or 32-bits of byte information. Noswap-mode byte addressing is as follows:

        Byte Address      0       1       2       3
        Little Endian   | A     | B     | C     | D     |
                        ---------------------------------
        Big Endian      | A     | B     | C     | D     |

D.1.13.2    VME_BS_BYTE Mode

VME_BS_BYTE mode provides a hardware mechanism for data coherency for 16-bit data transfers across the VMEbus, such as loading a 16-bit counter on a VMEbus device. In this mode, bytes within words are swapped. For portability, only 16-bit aligned transfers should be used. Byte-swap mode byte addressing is as follows:

        Byte Address      0       1       2       3
        Little Endian   | A     | B     | C     | D     |
                        ---------------------------------
        Big Endian      | B     | A     | D     | C     |

D.1.13.3    VME_BS_WORD Mode

VMS_BS_WORD mode provides a hardware mechanism for swapping words within longwords on certain VME adapters. This mode is not portable across VME adapters; on other VME adapters, byte swapping may be data-size dependent. Digital AXPvme SBC and Digital VME 2100 word swap-mode byte addressing is as follows:

        Byte Address      0       1       2       3
        Little Endian   | A     | B     | C     | D     |
                        ---------------------------------
        Big Endian      | C     | D     | A     | B     |

D.1.13.4    VME_BS_LWORD Mode

VME_BS_LWORD mode provides a hardware mechanism for data coherency for 32-bit data transfers across the VMEbus, such as loading a 32-bit VMEbus address register. In this mode, bytes within words are swapped and words within longwords are swapped. The transfer size is 32 bits only. For portability, only 32-bit transfers should be used. Longword swap-mode byte addressing is as follows:

        Byte Address      0       1       2       3
        Little Endian   | A     | B     | C     | D     |
                        ---------------------------------
        Big Endian      | D     | C     | B     | A     |

D.1.13.5    Shared Memory Between Big Endian and Little Endian Processors

Software swap is required to arrange bytes properly for 16- or 32-bit quantities (such as 16-bit counter values or 32-bit VMEbus address values). In a shared memory environment, where packed data structures in common memory are shared between a Digital processor (little endian) and a big endian processor, VME_BS_NOSWAP and software swapping on nonbyte data is recommended for the Digital processor; noswapping is recommended on the big endian processor.

Software swapping could be implemented with read/write macros that perform the swap with the following code. The purpose here is to provide code that would run on both Digital and big endian machines that have shared memory.

#define read_word/long(iohandle,data)                           /
        data = read_io_port(iohandle,sizeof(word/long),0);      /
#ifdef LITTLEENDIAN                                             /
        swap_xx(data);                                          /
#else /* BIGENDIAN */                                           /
#endif
#define write_word/long(iohandle,data)                          /
#ifdef LITTLEENDIAN                                             /
        swap_xx(data);                                          /
#else /* BIGENDIAN */                                           /
        write_io_port(iohandle,sizeof(word/long),0,data);       /
#endif

D.2    VMEbus Slave Block Transfers

The AXPvme Single-Board Computer (SBC) and Digital VME 2100 are configured to accept slave block transfers (SBLTs) with data widths of D16, D32, or D64 during adapter initialization. Once the SBC has mapped its memory onto the VMEbus by using the dma_map_alloc and dma_map_load functions, no other user interaction is needed.

Memory must be mapped to the VMEbus prior to D64 slave access.

Access to memory must coincide with the appropriate access mode. If you specify supervisory mode access when memory is mapped, memory accesses must use supervisory mode. If you specify user mode access, both supervisory and user access are allowed.

D.3    VMEbus Master Block Transfers with Local DMA

The AXPvme Single-Board Computer (SBC) and Digital VME 2100 VMEbus interfaces provide a block-mode DMA engine. This DMA engine is capable of transferring up to 64 KB of data without processor intervention in VMEbus data widths of D16, D32, or D64.

The DMA engine transfers data from the VMEbus to system memory (read) or from system memory to the VMEbus (write). The hardware interface handles the segmentation of the transfer. This ensures that the VMEbus specification is not violated in relation to crossing VMEbus 256-byte boundaries for D16 and D32 or 2-KB boundaries for D64.

The DMA engine is configured to give up the VMEbus during the transfer and to rearbitrate for the VMEbus again to continue the DMA. The time between when the DMA engine gives up the bus and rearbitrates for the bus is called the interleave period. During the interleave period, single-cycle VMEbus cycles, receipt of slave block transfers (SBLTs), or other operations may be performed.

The master block transfer (MBLT) hardware interface presents address modifiers of user block or supervisory block to the VMEbus, based on parameters passed in the Digital UNIX programming interface. The device or system on the VMEbus must be able to interpret these address modifiers; otherwise, bus errors may occur.

You can use the MBLT hardware interface to:

• Transfer data to and from those VMEbus devices that do not have their own DMA engine

• Move data between VMEbus device memory and system memory

• Transfer data to and from other systems that have their memory mapped to the VMEbus

The MBLT hardware interface supports DMA block-mode transfers to and from VMEbus A24 and A32 address space only.

D.3.1    Digital UNIX Interfaces for Master Block-Mode Transfer

To use the master block transfer (MBLT) with the local hardware DMA engine, you must invoke the following interfaces with specific flag values specified:

• vba_set_dma_addr

• dma_map_alloc

• dma_map_load

• vba_dma

• dma_map_unload

• dma_map_dealloc

The vba_dma interface is used to start up and monitor the DMA engine. See the vba_dma() reference page for information on the vba_dma calling interface.

The flag values DMA_IN and DMA_OUT have specific meaning for VMEbus support with respect to the dma_map_alloc, dma_map_load, and vba_dma interfaces. These flags indicate to the low-level VMEbus dma_map_alloc, dma_map_load, and vba_dma interfaces that the MBLT hardware DMA engine is to be used and the direction of the transfer.

Specifying DMA_IN implies a read from the VMEbus to system memory. Specifying DMA_OUT implies a write from system memory to the VMEbus. You use the vba_set_dma_addr interface to pass the flag values and the VMEbus address at which the transfer is to occur.

The VMEbus block-mode DMA engine on the VME adapter is a single entity that must be shared among various device drivers. Specifying DMA_SLEEP causes the device driver to block in the vba_dma interface if the DMA engine is already being utilized. If DMA_SLEEP is not specified and the DMA engine is being utilized, vba_dma returns an error indication.

The following sample code shows how to invoke the MBLT hardware DMA engine for a block-mode read operation. The code uses a VMEbus transfer width of D32 to invoke a 256-KB transfer from VMEbus address A24 0x400000 to system memory. The code also allocates resources to handle transfers up to 1 MB in size. This allows dma_map_load and vba_dma to be invoked multiple times with varying size buffers. You can change the code to perform writes by substituting DMA_OUT for DMA_IN.

  struct controller *ctlr;
  vme_addr_t        vme_addr = 0x40000;
  unsigned long     max_bc = (1024*1024);
  unsigned long     rtn_bc;
  char              *buffer;
  unsigned long     buffer_bc = (1024 * 256);
  sglist_t          dma_handle = (sglist_t)NULL;
  vme_atype_t       flags = (VME_A24_UDATA_D32 | DMA_IN | DMA_SLEEP);
  int               rtn_flags;
  /*
   *  Allocate a buffer (256 KB) to be used for the transfer
   */
  MALLOC(buffer,(char *),buffer_bc,M_DEVBUF,M_WAITOK);
  /*
   *  Specify a VMEbus address of 0x40000
   *  Specify flags
   *     A24 address space
   *     User mode
   *     Select DMA engine for a read (DMA_IN) and
   *       wait for DMA engine (DMA_SLEEP)
   */
  rtn_flags = (int)vba_set_dma_addr(ctlr,flags,vme_addr);
  /*
   *  Allocate DMA resources for up to 1 Mbyte transfer
   *    Specify flags returned from vba_set_dma_addr() above.
   *    The return value from dma_map_alloc() should equal max_bc
   */
  rtn_bc = dma_map_alloc(max_bc,ctlr,&dma_handle,rtn_flags);
  /*
   *  Call dma_map_load() to load the resources for the DMA block-mode engine
   *    Specify the dma_handle returned from dma_map_alloc()
   *    Specify flags returned from vba_set_dma_addr().
   *    The return value from dma_map_load() should equal buffer_bc
   *
   */
  rtn_bc = dma_map_load(buffer_bc,
	                (vm_offset_t)buffer,
	                0,
	                ctlr,
	                &dma_handle,
	                0,
	                rtn_flags);
  /*
   * Call vba_dma() to start up and monitor the VME adapter's block-mode DMA
   *   engine. Specify the dma_handle returned from dma_map_alloc
   *   The return value from vba_dma() is the actual bytes transferred. This
   *   value should be the same as value buffer_bc. If not, then an error was
   *   detected during the transfer.
   */
  rtn_bc = vba_dma(ctlr,dma_handle);
  /*
   *  Unload and free DMA resources
   */
  dma_map_unload(0,dma_handle)
  dma_map_dealloc(dma_handle)

D.3.2    Restrictions on VMEbus Master Block Transfers with Local DMA

Several restrictions apply to using master block transfers (MBLTs) on the AXPvme Single-Board Computer and Digital VME 2100 systems. These restrictions are listed by data-width selection of D16, D32, and D64.

• D08 (unsupported)

• D16 restrictions

  The VMEbus address must be longword aligned (0, 4, 8, and so forth). The memory address must be word aligned (0, 2, 4, and so forth). The requested byte count must be a multiple of 2.

• D32 restrictions

  The VMEbus address must be longword aligned (0, 4, 8, and so forth). The memory address must be longword aligned (0, 4, 8, and so forth). The requested byte count must be a multiple of 4.

• D64 restrictions

  The VMEbus address must be quadword aligned (0, 8, 16, and so forth). The memory address must be quadword aligned (0, 8, 16, and so forth). The requested byte count must be a multiple of 8.

The Digital VME 2100 system in an SMP environment emulates DMA transfers via PIO operations in lieu of using the MBLT hardware DMA engine. The VME adapter on this system requires three I/O accesses to be atomic to start the DMA engine. These I/O operations cannot be guaranteed to be atomic in an SMP environment. Uniprocessor systems use the MBLT hardware DMA engine.

D.4    Realtime Interrupt-Handling Function

Interrupt service interfaces (ISIs) executing at SPLDEVRT (spl 6) must not call kernel interfaces directly. The rt_post_callout function allows the calling process to defer execution of a function until a time when kernel interfaces can be invoked. The function invoked by rt_post_callout runs at an elevated spl and is subject to the same restrictions as an ISI.

The syntax for the rt_post_callout function is as follows:

int rt_post_callout (
   int (*function)(),
   long arg1,
   long arg2 );

The parameters for the rt_post_callout function are as follows:

• function - Name of the function to be invoked

• arg1 - The first argument passed to the function

• arg2 - The second argument passed to the function

If rt_post_callout is called again with the same function and arguments specified, then the duplicate invocation is dismissed before the first invocation has executed.

The following example is for an interrupt service interface (ISI) that runs at SPLDEVRT:

rt_dev_intr(unit)
	int unit;
{
  register struct rt_softc *sc = rt_softc[unit];
  rt_post_callout(  user_wakeup_interface,	   /* user wakeup function */
		  (long) &sc->error_recovery_flag, /* event to wakeup      */
		  (long) NULL);			   /* unused argument	   */
  return;
}

The following example shows a user-written function to wake up an event called by the rt_post_callout function:

void user_wakeup_interface ( arg1, arg2 )
long arg1;
long arg2;
{
   thread_wakeup( (vm_offset_t) arg1);
}

D.5    VMEbus Backplane Network Driver

In addition to supplying drivers for traditional network media (such as the Ethernet), Digital UNIX includes a VMEbus backplane (VB) driver that communicates over a backplane VMEbus.

The backplane driver is compatible with the other parts of the network subsystem; that is, all higher-level network protocols are immediately available over the backplane, just as they are over the Ethernet. Socket communication, remote login, remote file access, NFS, and remote procedure calls are all simultaneously available to and from any processor on the backplane. Using these network facilities over the backplane is indistinguishable from using any other network medium.

Both the Digital UNIX and VxWorks (Version 3.1 or higher) software support the VB driver as well as communication between these systems on the same backplane.

By default, the VB driver is not configured to run when the system is booted and must be explicitly turned on for the node to participate in the backplane network.

The VB driver is supported on AXPvme Single-Board Computers and Digital Alpha VME 2100 systems.

D.5.1    DECnet Support

Currently, DECnet is not supported over the VMEbus backplane (VB) driver.

D.5.2    VB Driver Concepts

Configuring nodes in a VB network can be simple or complex, depending on the specific system needs. At a minimum, the VB driver must be configured to be turned on and the Ethernet hardware address of the target system must be specified. By default, an unconfigured driver will not start up.

You can use all other default VB characteristics without change, as long as you configure the necessary system VMEbus window space correctly. You can also tailor several backplane node characteristics to meet specific system and application needs. The following sections describe how to configure these characteristics.

D.5.2.1    VMEbus Address Use by the VB Driver

VMEbus addresses are used in two ways in the VB driver:

• To map local memory onto the VMEbus

• To interrupt nodes on the VB network when data is sent

The following sections describe how VMEbus addresses are used for mapping data and for interrupting nodes on the VB network.

D.5.2.1.1    VMEbus Addresses Used for Mapping Data

A VB network is made up of two or more nodes in a VMEbus backplane cage that communicate by way of local memory mapped onto the VMEbus. Nodes that participate in the VB network provide local memory for client message queues. Other backplane nodes map to this memory over the VMEbus and write data to this local memory; this is what is meant by "sending" messages to a node on the backplane network.

The VMEbus has three different basic address spaces:

• A16

• A24

• A32

Each space is referred to as a system VMEbus window. Each system in a VMEbus backplane must configure its system VMEbus windows' base address and size in a unique manner, such that the windows do not overlap across the backplane. See Section D.1 for more information on VMEbus address spaces.

The VB driver uses either A24 or A32 space to map its client communication queues (data) to the VMEbus. The specific address within the A24 or A32 space is specified by an offset from the base of the particular window. The size of the queue data is also specified by the user.

Specify the following information regarding the queues for each backplane node:

• The address space in which to map the queues (A24 or A32) as well as other address space modifiers (supervisory/user, program/data)

• The total size of the area used to map the communication queues

• The VMEbus address where the queues will be mapped to the VMEbus, as specified by an offset from the base of the queues' chosen system VMEbus window

Default values are defined for these three items, but you can configure your VB characteristics differently by modifying values in the /usr/sys/data/if_vb_data.c file, as described in Section D.5.3. The arrows in the following code show the members to modify in if_vb_data.c:

struct vb_cfg vb_cfg = {
   /* Configurable on a per-node basis (nodes can have *
    * different values):*/
      VB_STARTUP_OFF,      /* Startup state of this driver. */
1 --> (AM_A24|AM_SUPER|AM_DATA),/* Address modifier for VME space used       */
                                /*  for client node's message queues.        */
2 --> VB_DMA_WINDOW_SIZE,       /* Size of client VME space used for         */ 
                                /*  message queues.                          */
      VB_INTERRUPT,             /* Interrupt or polled driver I/O interface  */
      VB_LIVENESS_TIMEOUT_DEFAULT,/* Milliseconds interval for checking      */
                                /*  remote node liveness.                    */
      VB_A16_OFFSET_DEFAULT,    /* A16 offset for inbound module switch      */
                                /*  interrupts.                              */
      VB_MAXNODES_DEFAULT,      /* Maximum nodes in the network.             */
3 --> VB_L3OFFSET_DEFAULT,      /* Offset from base VME window address to    */
                                /*  where the client queues will be mapped   */
                                /*  onto the VME for other nodes to see.     */
                                /*  The resulting address must be on a       */
                                /*  page boundary (0x2000 bytes).            */
      {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}  /* Ethernet hardware address     */
                                            /*   of node                     */ 
      /* Configurable on a per-network basis (all nodes must match):         */
      VB_WELL_KNOWN_ADDRESS,    /* Well-known VME address of box manager     */
                                /*   global data. Must be on a page          */
                                /*   boundary (0x2000 bytes).                */
      VB_WELL_KNOWN_ADDRESS_TYPE, /* Address modifier of box mgr global data */
      VB_MAXMTU_DEFAULT           /* Maximum mtu size -- DO NOT MODIFY       */
};

Whatever you configure the values to be, you must modify the system VMEbus windows to accommodate the chosen values. The window (A24 or A32) base and size specified must be unique across the backplane, and its size must be big enough to fit the queue size specified, starting at the offset specified.

You can configure the system VMEbus window on a per-system basis by modifying the A24 or A32 VMEbus base address and size in the /usr/sys/data/ebv10_vme_data.c file. See Section D.1 for more information on modifying the base address and size of the system VMEbus window.

Note

If you do not uniquely configure the system VMEbus windows for the backplane nodes on the VB network, unpredictable behavior may occur. An error message similar to the following prints to the console of a node whose system VMEbus window overlaps that of a node that has mapped the window and is actively communicating through it, even if it is with a device other than the VB driver:

vba0 errors_inter: VIP/VIC errors detected
        VIC BESR 0x50 - VIP BESR 0x40400 VIC DMASR 0x8
        VMEbus timeout
        local bus error - LBERR* asserted to VIC
        Inbound error - invalid s/g or VMEbus slave access error

D.5.2.1.2    VMEbus Addresses Used for Interrupting

Module switch (mailbox interrupt) settings regulate interrupt activity in the VB backplane network. When a node sends data to another node, the sending node generates an interrupt on the receiving node by using module switches.

An interrupt is generated by writing to a particular offset from the base of the A16 system VMEbus window of the node to be interrupted. The offset determines the particular module switch to use for interrupting a node. A16 system windows are always 256 KB (0x100 KB) in size. Valid A16 base addresses are 0x0, 0x100, 0x200, and so on.

You must configure the A16 system VMEbus windows to be unique across the nodes in the VMEbus backplane. You can configure the A16 system VMEbus window on a per-system basis by modifying the A16 base address in the /usr/sys/data/ebv10_vme_data.c file. See Section D.1 for more information on modifying the base address and size of the system VMEbus window.

You specify the module switch to use for interrupting (determined by the offset within the unique A16 window for a particular system) by modifying a value in /usr/sys/data/if_vb_data.c. You can use the default offset of 0x23, which is module switch 1.

There are four module switches associated with each 256-KB A16 window. Table D-11 shows valid offsets and corresponding module switch interrupts.

Table D-11: A16 Offset Values for Module Switch Interrupts

Specify the following settings to use the interrupt interface in the VB driver:

• Set the interface to interrupt mode, rather than polling mode.

• Specify the module switch to use by specifying the appropriate offset within the A16 window (as specified in ebv10_vme_data.c ).

Default values are defined for these two items, but you can configure your VB characteristics differently by modifying values in /usr/sys/data/if_vb_data.c, as described in Section D.5.3. The arrows in the following code show the members to modify in if_vb_data.c:

struct vb_cfg vb_cfg = {
      /* Configurable on a per-node basis (nodes can have different values): */
      VB_STARTUP_OFF,           /* Startup state of this driver.             */
      (AM_A24|AM_SUPER|AM_DATA),/* Address modifier for VME space used       */
                                /*  for client node's message queues.        */
      VB_DMA_WINDOW_SIZE,       /* Size of client VME space used for         */
                                /*  message queues.                          */
 -->  VB_INTERRUPT,             /* Interrupt or polled driver I/O interface  */
      VB_LIVENESS_TIMEOUT_DEFAULT,/* Milliseconds interval for checking      */
                                /*  remote node liveness.                    */
 -->  VB_A16_OFFSET_DEFAULT,    /* A16 offset for inbound module switch      */
                                /*  interrupts.                              */
      VB_MAXNODES_DEFAULT,      /* Maximum nodes in the network.             */
      VB_L3OFFSET_DEFAULT,      /* Offset from base VME window address to    */
                                /*  where the client queues will be mapped   */
                                /*  onto the VME for other nodes to see.     */
      {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}  /* Ethernet hardware address     */
                                            /*   of node                     */ 
      /* Configurable on a per-network basis (all nodes must match):         */
      VB_WELL_KNOWN_ADDRESS,      /* Well-known VME address of box manager   */
                                /*   global data.                            */
      VB_WELL_KNOWN_ADDRESS_TYPE, /* Address modifier of box mgr global data */
      VB_MAXMTU_DEFAULT           /* Maximum mtu size -- DO NOT MODIFY       */
};

Whatever you configure the values to be, the A16 system VMEbus window base address must be modified to be unique among the nodes in the VMEbus backplane for interrupting to work. However, the module switch used to interrupt a particular node can be individually configured on a per-node basis (not necessarily uniquely).

To reiterate, you can use the default offset in if_vb_data.c. However, only one of the nodes in the backplane can use the default A16 base address in ebv10_vme_data.c.

D.5.2.2    Box Manager Node

Because a VB network is made up of two or more nodes in a VMEbus backplane cage that communicate via local memory mapped onto the VMEbus, information about which nodes are participating in the network must be stored so that all nodes can access this information. The information is stored in the local memory of a single backplane node called the box manager.

The box manager node is a special client in that it maps this global information onto the VMEbus in addition to mapping its client communication queues.

The box manager maps the global information onto the VMEbus at an address that is known to all other nodes in the backplane network (the well-known address). When nonbox manager nodes boot, they read information from the well-known address to see what other nodes are in the network. The well-known address must reside in the particular system VMEbus window (A24 or A32) with modifiers (supervisory/user, program/data) that are also well known to other nodes in the VB network. The combination of the address and its modifier uniquely specifies where the box manager global data resides on the VMEbus for all nodes to see.

The well-known address is configurable through if_vb_data.c, and defaults to VB_WELL_KNOWN_ADDRESS, which equals 0xA40000. The address space that it is mapped to (A24 or A32) is also configurable through if_vb_data.c and defaults to VB_WELL_KNOWN_ADDRESS_TYPE, which equals (AM_A24|AM_SUPER|AM_DATA). (See Section D.5.3.1.2 for more information on configuring the well-known address and modifiers.)

The network manager must configure only one node to be the box manager node. A node is a box manager if the well-known address is contained within the node's system VMEbus window (either A24 or A32, depending on the configured value of the box manager address modifier). There is no other switch or value to specify to identify a box manager. Note that you do not have to set the base VMEbus window address to the well-known address; the well-known address must simply be contained within a valid VMEbus system window.

When a node boots, it determines whether or not it is the box manager node by comparing the well-known address to its configured system VMEbus window range. A node that is not the box manager node is called a client node.

The box manager node is also just another network client, and it has local communication queues mapped to the VMEbus just like any other client. The difference is in the placement of those queues mapped onto the VMEbus. The box manager has two sets of data that must be mapped to the VMEbus: the box manager global data and the client communication queues.

By default, box manager global data and client communication queues are mapped to the same address space, A24. In the default case:

• The offset of the communication queues from the base window specified in if_vb_data.c for the queues is ignored.

• The communication queues are mapped directly following the global data starting at the well-known address.

In addition, the combined size of the global data and the communication queues is adjusted to be equal to the configured size of the communication queues (whose default is VB_DMA_WINDOW_SIZE). You do not need to deal with the size of the box manager global data when you determine what your system VMEbus window size should be for the box manager node.

However, you can configure the global data and the communication queues to be mapped to different spaces (A24 and A32). In this case, the communication queues are mapped like any other client node. They are mapped at the configured offset from the base of its configured window. The global data is mapped to the well-known address, for a size of 0x6000 bytes. You must be sure that both system windows, A24 and A32, will accommodate either the well-known address or the communication queues.

The box manager node must be the first node in the backplane to boot so that the global memory is mapped to the well-known address before other nodes attempt to read from it.

You must boot the VMEbus system controller for the VMEbus crate (set by the appropriate jumper on the module) before any other node that is participating in the VB backplane or any other node that is using the VMEbus. This is because when the system controller is booted, it can reset the VMEbus registers of all other nodes. If the box manager is not the VMEbus system controller, ensure that the system controller boots before the box manager node, or that the system controller is not booted while the VB network is up and running.

D.5.2.3    Network Participation

Nodes in a backplane network communicate via memory mapped onto the VMEbus. If this memory becomes unmapped, or the VMEbus is reset for any reason, the mapping is no longer valid. Any read or write operations to a remote node that uses the invalid mapping could cause a panic or machine check on the system performing the read or write. To reduce the possibility of this occurring, the nodes in the VB network keep "liveness" with the rest of the network.

Once a node enters the VB network, it begins continually updating a counter in the global memory called its "heartbeat." In addition, all nodes on the network continually check the VB heartbeat of other nodes, including the box manager node, to see if they are still alive and able to participate in the network in a timely manner.

If the heartbeat of a remote node is no longer being updated, communication to that node must stop in anticipation of the remote node's VMEbus mapping becoming invalid. For example, if a node is rebooted, its heartbeat ceases to be updated and the rest of the backplane nodes eventually lose liveness with that node and stop communicating with it.

When a node is shut down in a controlled manner, (using /usr/sbin/shutdown), the VB driver notifies the other VB nodes that it is shutting down, so that they can stop communicating. If a node is shut down in an uncontrolled manner (panic or halt), the current VMEbus mappings remain valid until you reinitialize the system. This allows time for other VB network nodes to lose liveness with the node before an invalid mapping reference occurs.

Once you fully reboot the shutdown node, it can reenter the VB network, and is seen by the other VB network nodes again.

Once node A loses liveness with node B, node B cannot reenter the VB network without rebooting. You cannot restart the VB driver without rebooting. This restriction is due to the need for the restarting node to probe the well-known address to see if a box manager memory is mapped to the well-known address. This probing is supported only during the booting stage.

Response time is an important aspect of liveness. Even if a node is not shut down, it may respond too slowly to VB network traffic to be considered alive. In these cases, it may be in the best interest of the rest of the VB network to cease communication with that node. For example, a node may have a realtime application running at a realtime priority above that of the VB network driver, and in fact higher than many system functions. Without network traffic being processed in a timely manner, backups or message loss could occur on any node attempting to send data to the node.

The liveness feature of the drivers allows remote nodes to notice that the node's heartbeat is not being updated (because the node is devoted to the realtime application) and stop attempting to communicate with it. In addition, you could use a long liveness interval in a stable network configuration (one that does not expect frequent shutdowns) to allow a light load on the VB network to continue in the midst of expectedly high realtime priority usage.

D.5.3    Configuring the VB Network Driver

The VB driver has to be explicitly turned on to be started up. After you boot a system, you can configure the VB characteristics by editing the /usr/sys/data/if_vb_data.c data file. In addition, you must configure the system VMEbus window and the VMEbus A16 window base address (described in Section D.5.2.1) by editing the /usr/sys/data/ebv10_vme_data.c data file.

D.5.3.1    Editing the if_vb_data.c File

The if_vb_data.c file contains the following structure, whose member values may be modified:

struct vb_cfg vb_cfg = {
/* Configurable on a per-node basis (nodes can have different values):      */
    VB_STARTUP_OFF,             /* Startup state of this driver.            */
    (AM_A24|AM_SUPER|AM_DATA),  /* Address modifier for VME space used      */
                                /*  for client node's message queues.       */
    VB_DMA_WINDOW_SIZE,         /* Size of client VME space used for        */
                                /*  message queues.                         */
    VB_INTERRUPT,               /* Interrupt or polled driver I/O interface */
    VB_LIVENESS_TIMEOUT_DEFAULT,/* Milliseconds interval for checking       */
                                /*  remote node liveness.                   */
    VB_A16_OFFSET_DEFAULT,      /* A16 offset for inbound module switch     */
                                /*  interrupts.                             */
    VB_MAXNODES_DEFAULT,        /* Maximum nodes in the network.            */
    VB_L3OFFSET_DEFAULT,        /* Offset from base VME window address to   */
                                /*  where the client queues will be mapped  */
                                /*  onto the VME for other nodes to see.    */
    {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}   /* Ethernet hardware address     */
                                           /*   of node                     */ 
/* Configurable on a per-network basis (all nodes must match):              */
    VB_WELL_KNOWN_ADDRESS,      /* well-known VME address of box manager    */
                                /*   global data.                           */
    VB_WELL_KNOWN_ADDRESS_TYPE, /* Address modifier of box mgr global data. */
    VB_MAXMTU_DEFAULT           /* Maximum mtu size -- DO NOT MODIFY        */
};

The following sections describe how a network manager can configure VB characteristics to successfully start a VB backplane network.

D.5.3.1.1    Per-Node Configurable Data

You can configure the following members on a per-node basis, and each node can have values different from the other nodes:

• Startup state

  You must change this member from VB_STARTUP_OFF to VB_STARTUP_ON to start up the VB network.

• Address modifier for client VMEbus address space

  You can map a node's queue memory to the VMEbus in either the A24 or A32 address space (AM_A24 or AM_A32), either in supervisory or user mode (AM_SUPER or not), and either in program or data space (AM_DATA or not). The default is A24 data space, supervisory mode, for example, (AM_A24|AM_SUPER|AM_DATA). A32 program space in user mode would be represented as (AM_A32).

• Size of this client's VMEbus window

  This member specifies the size (in bytes) of the VMEbus area selected from the system VMEbus window (according to the address modifier member) to be used by the VB driver for its communication queues. The bigger the size, the greater the number of message packets that will be preallocated for communication.

  The default size of VB_DMA_WINDOW_SIZE, which is 0x40000 (or 256 KB) is enough for 150 packets to be reserved for the queues. This means that if there are 10 nodes in the network, the default size will allow for 15 packets to be devoted exclusively for communication between the local node and each of the other nodes.

• Interrupt or polled driver interface

  The VB driver supports two interfaces for determining whether messages have been sent to its queues: polled or interrupt. You should use the default of VB_INTERRUPT for performance reasons, rather than the alternative VB_POLLED. The VB driver uses module switch interrupts. If you use the interrupt interface, you must ensure that the A16 system VMEbus base address for your SBC or Digital VME 2100 is unique among the nodes in the backplane, as configured in the ebv10_vme_data.c file.

• Milliseconds between checking remote node liveness

  As previously described, a node continually checks to see if a remote node is still alive. It does this by default in intervals of VB_LIVENESS_TIMEOUT_DEFAULT milliseconds, which is defined as 10000 (10 seconds). Be careful when modifying this value. An interval that is too short could cause nodes to lose liveness with each other too easily, and the "lost" node (system) must be rebooted for communication to resume. An interval that is too long or an interval of 0, which specifies no liveness checking, could cause a node to take too long to determine that a remote node has gone down. In this case, the node can attempt to communicate with the shutdown node after the VMEbus mapping is invalid.

• A16 offset for inbound module switch interrupts

  You use module switches to create VB driver interrupts on the backplane. You can use four module switches for interrupts for each system VMEbus A16 base address. The specific module switch used is indicated by the particular value configured in the A16 offset member of the vb_cfg structure in if_vb_data.c. Table D-12 shows the valid offset values.

  Table D-12: A16 Offset Values for Module Switch Interrupts

  Value	Module Switch
  0x21	Module switch 0
  0x23	Module switch 1
  0x25	Module switch 2
  0x27	Module switch 3

  The default offset is 0x23 (switch 1). Thus, if the A16 system VMEbus base address for node A is 0x300, remote nodes can use offset 0x23 added to 0x300 (0x323) to cause an interrupt on node A when the VB driver writes to it. The A16 system VMEbus base address must be unique among all nodes in the backplane; however, the offset itself need not be unique.

  Note

  If you change the module switch from the default of 1, this change must be reflected in the vector number that is specified in the controller line for the VB device in the system configuration file (as documented in Section D.5.5). If you do not change the vector number, interrupts will not work on the system. The default module switch 1 has a corresponding default vector of 0x1150.

  Table D-13 shows the vectors that correspond to module switches.

  Table D-13: Vectors for Module Switch Offsets

  Module Switch Offset	Vector
  0x21	0x1140
  0x23	0x1150
  0x25	0x1160
  0x27	0x1170

• Maximum nodes in the network

  This is the maximum number of nodes that can be a part of the VB network. VB_MAXNODES_DEFAULT is defined as 10, and you cannot modify the value to greater than 32. This value is examined by the box manager only, and determines the maximum number of nodes that may enter the VB backplane network for as long as the box manager is booted. All other client nodes adjust their VB_MAXNODES value according to what the box manager's value is, and they do not have to know ahead of time what the exact value of VB_MAXNODES is on the box manager node.

• Offset from base VMEbus window address to map client queues

  VB_L3OFFSET_DEFAULT is the offset from the base system window at which the client queues will be mapped. The default is 0x0 (at the base system window). The system window (either A24 or A32) is determined by the client queue address modifier. The client queues are mapped at a fixed offset from the base window address (which you can modify in ebv10_vme_data.c ). The default is to map the queues at the beginning of the base address; thus, VB_A16_OFFSET_DEFAULT is defined as 0x0.

  You must have the ability to adjust the placement of the queue mappings because if other VMEbus drivers on the system use specific VMEbus addresses to map memory to, there may be conflicts. In this case, you can either adjust the base address of the particular system VMEbus window or modify the offset value such that the queues (of size specified by VB_DMA_WINDOW_SIZE, for example) start at a specific VMEbus address.

  In general, if you are not planning to use any other VMEbus devices in your systems, leave the default offset at 0x0 for all nodes. If you must change the offset, be sure the value is on a page boundary (0x2000 bytes).

• Ethernet hardware address of node

  The user must fill in this field with the Ethernet hardware address of the node. There is no default for this field. The VB network address is derived from the unique Ethernet hardware address and is the shadow Ethernet address. Thus, this field must contain the Ethernet hardware address of the node that if_vb_data.c is being modified for. If this field is not modified, the VB driver will not come up and the following error will be printed to the console:

  VB: Ethernet address contains all zeroes!  DRIVER EXITING...
  

  For example, if the enet address of the node is 08-00-2b-e2-48-48, you would type {0x08, 0x00, 0x2b, 0xe2, 0x48, 0x48} in the field. One way to obtain the enet address of a running system is "netstat -I ln0" (or tu0, and so on, whatever the Ethernet hardware is). You can also obtain the Ethernet address at the console prompt of a nonbooted system as follows:

  >>>  show dev
  

D.5.3.1.2    Per-Network Configurable Data

You can configure the following three members on a per-network basis, meaning that all nodes that participate in the VB network must configure these values to be exactly the same:

• Well-known VMEbus address of the box manager

  This member is the well-known address referred to in Section D.5.2.2. It is the VMEbus address that the box manager maps global VMEbus data to. This value must be exactly the same on all nodes for the nodes to communicate at all. Take care when modifying this value.

  The default of VB_WELL_KNOWN_ADDRESS is 0xA40000. A network manager can modify this value on all nodes in the network if a specific VMEbus configuration is required.

• Address modifier of box manager data well-known address

  You can map the box manager global data to the VMEbus in either the A24 or A32 address space (AM_A24 or AM_A32), either in supervisory or user mode (AM_SUPER or not), and either in program or data space (AM_DATA or not). The default of VB_WELL_KNOWN_ADDRESS_TYPE is A24 data space, supervisory mode, for example, (AM_A24|AM_SUPER|AM_DATA). As with the well-known address values, take care to ensure that all nodes are configured exactly the same. If you do modify this value, be sure that it is on a page boundary (0x2000 bytes).

• Maximum transmit unit (mtu) size

  Do not modify this value; it is not currently configurable.

D.5.3.2    Editing the ebv10_vme_data.c File

As described in Section D.5.2.1, you must configure unique system VMEbus windows for every node in the VB network. Section D.1 describes the valid values that you can configure for the A24 and A32 system VMEbus windows.

Each system participating in the VB network must map its client communication queues to either A24 or A32 space in a unique manner. Thus, there must be at least enough system VMEbus window space to accommodate the size devoted to the communication queues. In addition, the system VMEbus window of the box manager node must encompass the well-known address (default of 0xA40000).

Although the address modifiers of the box manager well-known address and of the client communication queues are the same by default (A24/Super/Data), they need not be the same. If they are not the same, configure the box manager node so that its system windows accommodate both sets of data. If they are the same, configure the box manager node so that the chosen system VMEbus window accommodates both sets of data, starting at the well-known address, for a size equal to the size of the communication queues.

For interrupting, the A16 system VMEbus window base address must also be unique for all nodes in the backplane, but the size is always 0x100.

The arrows in the following code show the members to modify in ebv10_vme_data.c for the A16, A24, and A32 system VMEbus windows (the default values are shown):

struct    ebv10_vme_data  ebv10_vme_data  =  {
    VME_BR3,            /* Bus Request Level for master cycles             */
    VME_ARB_RR,         /* Arbitration mode                                */
    VME_FRTO_DSABL,     /* VMEbus fair requester disabled                  */
    VME_BUSTO_256US,    /* Local bus acquisition timeout                   */
    VME_BUSTO_512US,    /* VMEbus Arbitration Timeout                      */
    VME_ROR,            /* Release mode                                    */
    VME_SYS_CONTROLLER, /* System Controller                               */
    VME_A24_A32_OVRLAP, /* Inbound A24/A32, if same space, overlap         */
    VME_WRT_POST_DSABL, /* Disable VIC master write posting                */
--> 0x08000000,         /* A32 Inbound DMA window base address             */
--> VME_SZ_128_MB,      /* A32 window sz in kb (131072kb  =  128mb)        */
--> 0x00C00000,         /* A24 Inbound DMA window base address             */
--> VME_SZ_4_MB,        /* A24 window size in kb (4096kb  =  4mb)          */
--> 0x00000100,         /* A16 Interprocessor Communication base           */
    0x00000000,         /* A16 Interprocessor Communication mask           */
                        /* The following entries pertain to VME adapters   */
                        /* that are not embedded SBC VME designs. Embedded */
                        /* SBC VME designs pass the following information  */
                        /* via another mechanism.                          */
    SPLDEVLOW,          /* VME IRQ level to system SPL map                 */
    SPLDEVLOW,          /* VME IRQ 1 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 2 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 3 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 4 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 5 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 6 to SPL spldevlow                      */
    SPLDEVLOW,          /* VME IRQ 7 to SPL spldevlow                      */
    SPLDEVLOW,          /* Adapter resource blocking SPL spldevlow         */
    0                   /* Master DMA resource selection                   */
                        /*   sparse = 0 - dense = VME_DENSE                */
};

See Section D.5.4 for an example of how to modify both if_vb_data.c and ebv10_vme_data.c for a valid 3-node VB network.

Note

The size of the system VMEbus window for a node should be larger than what the VB driver needs. If the VB driver uses the entire system VMEbus window, there will be no space in the VMEbus window left for other VMEbus devices on the system to use.

A system manager must carefully configure all nodes on the backplane to have large enough system VMEbus windows to accommodate the needs of each, but not so much that there is little room left for other nodes. The system manager should make a roadmap of each system's VMEbus device addresses and sizes and fit the VB needs around the needs of the other devices because the VB characteristics are user configurable.

D.5.4    Summary of Configuration Steps

The following steps show the easiest way to configure three nodes to run in the backplane network: Node 0, Node 1, and Node 2.

For all three nodes, the client queues are configured to be at offset 0 (the default) from the base of the A24 system VMEbus window for the particular system. In addition, all other default values are used (for example, interrupt mode versus polling), except for the A24 base address and size and the A16 base address in ebv10_vme_data.c and the startup state in if_vb_data.c.

Configure the box manager node first. Make sure that it is either the VMEbus system controller node or that the system controller node is already up.

To configure Node 0, perform the following steps:

1. Modify /usr/sys/data/ebv10_vme_data.c, changing the A24 base address from the default of 0xC00000 to something that encompasses the box manager data well-known address of 0xA40000. For example, set the A24 base address to 0xA00000, and change the A24 size to VME_SZ_2_MB, which brings the window to just below the default window address of 0xC00000. The box manager node now has an A24 window of 0xA00000 to 0xBFFFFF.

2. Change the A16 base address to something other than the default of 0x100, for example, 0x000. The following is an example of ebv10_vme_data.c. The arrows in the code show the modified members.

   struct    ebv10_vme_data  ebv10_vme_data  =  {
       VME_BR3,            /* Bus Request Level for master cycles             */
       VME_ARB_RR,         /* Arbitration mode                                */
       VME_FRTO_DSABL,     /* VMEbus fair requester disabled                  */
       VME_BUSTO_256US,    /* Local bus acquisition timeout                   */
       VME_BUSTO_512US,    /* VMEbus Arbitration Timeout                      */
       VME_ROR,            /* Release mode                                    */
       VME_SYS_CONTROLLER, /* System Controller                               */
       VME_A24_A32_OVRLAP, /* Inbound A24/A32, if same space, overlap         */
       VME_WRT_POST_DSABL, /* Disable VIC master write posting                */
       0x08000000,         /* A32 Inbound DMA window base address             */
       VME_SZ_128_MB,      /* A32 window sz in kb (131072kb  =  128mb)        */
   --> 0x00A00000,         /* A24 Inbound DMA window base address             */
   --> VME_SZ_2_MB,        /* A24 window size in kb (4096kb  =  4mb)          */
   --> 0x00000000,         /* A16 Interprocessor Communication base           */
       0x00000000,         /* A16 Interprocessor Communication mask           */
                           /* The following entries pertain to VME adapters   */
                           /* that are not embedded SBC VME designs. Embedded */
                           /* SBC VME designs pass the following information  */
                           /* via another mechanism.                          */
       SPLDEVLOW,          /* VME IRQ level to system SPL map                 */
       SPLDEVLOW,          /* VME IRQ 1 to SPL spldevlow                      */
       SPLDEVLOW,          /* VME IRQ 2 to SPL spldevlow                      */
       SPLDEVLOW,          /* VME IRQ 3 to SPL spldevlow                      */
       SPLDEVLOW,          /* VME IRQ 4 to SPL spldevlow                      */
       SPLDEVLOW,          /* VME IRQ 5 to SPL spldevlow                      */
       SPLDEVLOW,          /* VME IRQ 6 to SPL spldevlow                      */
       SPLDEVLOW,          /* VME IRQ 7 to SPL spldevlow                      */
       SPLDEVLOW,          /* Adapter resource blocking SPL spldevlow         */
       0                   /* Master DMA resource selection                   */
                           /*   sparse = 0 - dense = VME_DENSE                */
   };
   

3. Modify /usr/sys/data/if_vb_data.c, changing the startup state member from VB_STARTUP_OFF to VB_STARTUP_ON.

4. Modify /usr/sys/data/if_vb_data.c, inserting the node's Ethernet hardware address in place of {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}. For example, if the node's Ethernet address is 08-00-26-E2-48-47, you would insert {0x08, 0x00, 0x26, 0xE2, 0x48, 0x47}.

5. Run doconfig to rebuild the box manager vmunix, as described in Section D.5.5.

6. Reboot the box manager node. You should see VB: messages printed on the console, and the following message if you have correctly configured the data files:

   VB: This is the box manager node
   

When you configure Node 1, do not modify /usr/sys/data/ebv10_vme_data.c. Instead, use the defaults, which do not overlap with the values reconfigured for the box manager node.

You now have the following setup for the two nodes:

	A24 base address	A24 end address	A16 base address
Node 0:	0xA00000	0xBFFFFF (2 MB)	0x000
Node 1:	0xC00000	0xFFFFFF (4 MB)	0x100

To configure Node 1, perform the following steps:

1. Modify /usr/sys/data/if_vb_data.c, changing the startup state member from VB_STARTUP_OFF to VB_STARTUP_ON.

2. Modify /usr/sys/data/if_vb_data.c, inserting the node's Ethernet hardware address in place of {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}. For example, if the node's Ethernet address is 08-00-26-E2-24-50, you would insert {0x08, 0x00, 0x26, 0xE2, 0x24, 0x50}.

3. Run doconfig to rebuild Node 1's vmunix, as described in Section D.5.5.

4. Reboot Node 1. You should see VB: messages printed on the console, and the following message if you have correctly configured the data files:

   VB: Box mgr address space is not configured for this system,
       thus this node is not the box manager node (OK).  Be sure
       that there is a box manager in the network.
   

To configure Node 2, perform the following steps:

1. Modify /usr/sys/data/ebv10_vme_data.c.

   Node 2 cannot use the default VMEbus addresses because they are already used by Node 1. Change the A24 base address from the default of 0xC00000 to something unique among the other two nodes. For example, set the A24 base address to 0x400000, and keep the A24 size at VME_SZ_4_MB. Node 2 now has an A24 window of 0x400000 to 0x7FFFFF, which is the second 4-MB partition of VMEbus A24 space.

2. Change the A16 base address to something other than the default of 0x100, and something other than 0x000, which is Node 0's choice. For example, set it to 0x200.

   You now have the following setup for the three nodes:

   	A24 base address	A24 end address	A16 base address
   Node 0:	0xA00000	0xBFFFFF (2 MB)	0x000
   Node 1:	0xC00000	0xFFFFFF (4 MB)	0x100
   Node 2:	0x400000	0x7FFFFF (4 MB)	0x200

3. Modify /usr/sys/data/if_vb_data.c, changing the startup state member from VB_STARTUP_OFF to VB_STARTUP_ON.

4. Modify /usr/sys/data/if_vb_data.c, inserting the node's Ethernet hardware address in place of {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}. For example, if the node's Ethernet address is 08-00-25-39-74-71, you would insert {0x08, 0x00, 0x25, 0x39, 0x74, 0x71}.

5. Run doconfig to rebuild Node 2's vmunix, as described in Section D.5.5.

6. Reboot Node 2. You should see VB: messages printed on the console, and the following message if you have correctly configured the data files:

   VB: Box mgr address space is not configured for this system,
       thus this node is not the box manager node (OK).  Be sure
       that there is a box manager in the network.
   

Note

Because Node 1 is using the system defaults for the VMEbus A24 window, you must make sure that if you bring up an additional node (Node 4) that you modify the addresses such that the defaults are not used. Even if Node 4 does not turn on the backplane driver at all, its inbound window overlaps with Node 1. Accesses to the window will cause error messages to be printed to the screen of Node 4 because Node 4 is getting inbound VMEbus accesses from other nodes on addresses that it has not mapped inbound to.

In summary, you should always reconfigure the VMEbus addresses to be unique, no matter how you plan to use the VMEbus.

D.5.5    Adding the VB Driver to an AXPvme SBC or Digital Alpha VME 2100

The following steps outline the complete procedure for adding the VMEbus backplane driver to your SBC or Digital VME 2100. To pick up the newly configured ebv10_vme_data.c and if_vb_data.c files, you must run the doconfig utility to create a new configuration file and a new kernel. To have the backplane driver built into the new kernel, you must edit the configuration file during the doconfig process to add the VB controller line.

Note

Configure and boot the box manager node before configuring and booting any other nodes. The box manager node must be running before you configure and boot other nodes, and the VMEbus system controller module must boot before the box manager, if the box manager is not the system controller. This is because when the system controller is booted, it may reset the entire VMEbus network.

1. Save a copy of the vmunix directory:

   #  cp /vmunix /vmunix.save
   

2. Modify the configurable data in the /usr/sys/data/if_vb_data.c file (as described in Section D.5.3), and the /usr/sys/data/ebv10_vme_data.c file (as described in Section D.1.1). For further information, see the System Administration guide.

3. When the doconfig utility asks if you want to edit the configuration file, answer yes and add the following line to the new configuration file:

   controller   vb0     at vba0   vector vbintr 0x1150 priority 7
   

   Interrupt vector 0x1150 implies module switch 1, which is the A16 address space offset 0x23. Module switch 1 is the default and is used for interrupting the driver.

   The VMEbus backplane driver is now available. During the boot it displays on the console diagnostic messages prefixed with the string VB:.

   The box manager node displays the following message at startup:

   VB: This is the box manager node
   

   A client node (nonbox manager node) displays the following message at startup:

   VB: Box mgr address space is not configured for this system,
       thus this node is not the box manager node (OK).  Be sure
       that there is a box manager in the network.
   

   Caution

   Make sure that only one node comes up as the box manager. If more than one node comes up as the box manager, it means that the system VMEbus window has been configured to contain the well-known address (whose default value is 0xA40000) on more than one node. This results in unpredictable behavior and, at a minimum, causes the VB network to fail.

   ---

   

   ---

   D.5.6    Registering the VB Driver as a Network Device

   Once the new kernel is built and booted, you must use the netsetup command to register the VB as a new network driver. Assign each VB node a unique IP address that is in a subnet used exclusively by the VB network to differentiate between the Ethernet network and the VB network. The participating nodes must be specified in the /etc/hosts file. For information on setting up a new network, see the Network Administration manual.