5 Kernel Interfaces That VMEbus Device Drivers Use

This chapter discusses some of the interfaces that VMEbus device drivers use. See Section 1.7 for information on which VMEbus interfaces are obsolete.

When writing device drivers for the VMEbus, you need to be familiar with kernel interfaces that:

Load and set allocated system resources for direct memory access (DMA) data transfers
Unload system DMA resources
Obtain the VMEbus address
Read and write data from a device register
Wait on or post events as a means of synchronization

The final section in this chapter discusses the mechanism for passing information to the busphys_to_iohandle interface.

5.1 Allocating Resources for Direct Memory Access Data Transfers

Direct memory access (DMA) describes the ability of a device to directly access (read from and write to) CPU memory, without CPU intervention. On Digital UNIX systems, there is no static mapping between the VMEbus address space and system memory. The kernel maps pages of system memory to the VMEbus address spaces by using page map registers (PMRs). Each page is mapped independently by one or more PMRs. Usually, PMRs can map any physical memory page so that I/O can be directly moved into and out of user buffers. To allocate the DMA space and then set up the mapping registers for DMA transfer, call the dma_map_alloc or dma_map_load interfaces or both.

During DMA allocation, extra page map registers (PMRs) will be reserved in addition to the PMRs needed to map the system's memory buffer. If you pass DMA_GUARD_UPPER, DMA_GUARD_LOWER, or both to the flags argument of the dma_map_alloc or dma_map_load interfaces, one page map register is reserved for each flag specification. If you call both dma_map_alloc and dma_map_load, you must OR the DMA_GUARD_UPPER and DMA_GUARD_LOWER flags in the flags argument of both interfaces. (The dma_map_alloc interface reserves the PMR resources.) You specify DMA_GUARD_UPPER if the VME device is accessing mapped system memory in an ascending order. You specify DMA_GUARD_LOWER if the VME device is accessing mapped system memory in a descending order. The purpose of the guard pages is to protect system memory from a VME device that exceeds its requested boundaries. Protection is only to a page and not to the residual data from the end of the system memory buffer to the end of the alpha page.

In addition, an extra page map register is reserved for the system memory buffer. This is done for two reasons:

The dma_map_alloc interface is not passed the system memory buffer. Therefore, the extra page map register is needed to handle unaligned system memory buffers when you call the dma_map_load interface.
You can call the dma_map_load interface multiple times, passing a different system memory buffer to be mapped for the transfer. You pass the system memory buffer to the virt_addr argument of the dma_map_load interface.

Following every call to the dma_map_load interface, a VMEbus device driver must call the dma_get_curr_sgentry interface to obtain the VMEbus address and byte count mapped by dma_map_load. The VMEbus device driver then passes this information to the VME device to perform VME device-to-memory read or write transfers. The VMEbus address consists of two parts:

The first part of the VMEbus address is the result of the following calculation:

The number of the first page map register allocated multiplied by the size of the bus that the page map register represents (typically an alpha page) plus any VMEbus offsets that are needed to get to the first allocatable resource (page map register).

The second part of the VMEbus address consists of the page offset of the system memory buffer from the start of an alpha page. You should call dma_get_curr_sgentry after each call to dma_map_load. The system memory buffer may not have the same page alignment as the previous call to dma_map_load. See Writing Device Drivers: Tutorial for a description and example of the dma_get_curr_sgentry interface. Section 6.10 shows how the /dev/dmaex driver calls the dma_get_curr_sgentry interface.

The following code fragment shows the preferred method for VMEbus device drivers: calling the dma_map_load interface. See Writing Device Drivers: Tutorial for a description and example of the dma_map_alloc interface. Section 6.10 shows how the /dev/dmaex driver calls the dma_map_load interface.


.
.
.

dmaex_strategy(struct buf  *bp)
{
  register int                unit = minor(bp->b_dev);  
  register struct controller  *ctlr = dmaex_ctlr[unit]; 
  register struct dmaex_softc *sc = &dmaex_softc[unit];  
  sg_entry_t                  dmaex_sg;

.
.
.

  sc->dma_handle = (dma_handle_t) NULL; [1]

.
.
.

  if (dma_map_load ((unsigned long) bp->b_bcount, [2]
                    (vm_offset_t) bp->b_un.b_addr, [3]
                    bp->b_proc, [4]
                    ctlr, [5]
                    &sc->dma_handle, [6]
                    0, [7]
                    VME_A24_UDATA_D32 | VME_BS_NOSWAP | 
                    DMA_GUARD_UPPER | DMA_ALL) == 0) { [8]

.
.
.

--> Sets the dma_handle member of the softc structure pointer associated with this device to the value NULL. The example passes this member as the fifth argument to dma_map_load.
--> Calls the dma_map_load interface to load and set the system resources necessary to perform a direct memory access (DMA) transfer. The dma_map_load interface takes a byte_count as the first argument. This argument specifies the maximum size (in bytes) of the data to be transferred during the DMA transfer operation. The kernel uses this size to determine the resources (mapping registers, I/O channels, and other software resources) to allocate.
The maximum size passed to byte_count is the value stored in the b_bcount member of the buf structure pointer. The b_bcount member specifies the size of the requested transfer (in bytes).
--> The dma_map_load interface takes a virt_addr as the second argument. The virt_addr argument specifies the virtual address where the DMA transfer occurs. The interface uses this address with the pointer to the proc structure to obtain the physical addresses of the system memory pages to load into DMA mapping resources. In this call, the example passes the value stored in the b_un.b_addr member of the buf structure pointer. The b_un.b_addr member specifies the address at which to pull or push the data.
--> The dma_map_load interface takes a proc_p as the third argument. The proc_p argument specifies a pointer to the proc structure associated with the valid context for the virtual address specified in virt_addr. The interface uses this pointer to retrieve the pmap that is needed to translate this virtual address to a physical address. If proc_p is equal to zero (0), the address is a kernel address.
In this call, the example passes a proc structure pointer stored in the b_proc member of the buf structure pointer. The b_proc member specifies a pointer to the proc structure that represents the process performing the I/O.
--> The dma_map_load interface takes a ctlr_p as the fourth argument. The ctlr_p specifies a pointer to the controller structure associated with this controller. The dma_map_load interface uses the pointer to get the bus-specific interfaces and data structures that it needs to load and set the necessary mapping resources.
In this call, the example passes the controller structure pointer associated with this device.
--> The dma_map_load interface takes a dma_handle_p as the fifth argument. The dma_handle_p argument specifies a pointer to a handle to DMA resources associated with the mapping of an in-memory I/O buffer onto a controller's I/O bus. This handle provides the information to access bus address/byte count pairs. A bus address/byte count pair is represented by the ba and bc members of an sg_entry structure pointer. Device driver writers can view the DMA handle as the tag to the allocated system resources needed to perform a DMA operation.
Typically, the device driver passes an argument of type dma_handle_t *. This argument contains the address of the DMA handle returned by dma_map_alloc, if you previously called it. If the device driver does not make a previous call to dma_map_alloc, as is the case here, then you must set the dma_handle_p argument to the value NULL.
In this call, the example passes the DMA handle stored in the dma_handle member of the driver's softc structure associated with this device (in this case the value NULL).
--> The dma_map_load interface takes a max_byte_count argument as the sixth argument. The max_byte_count argument specifies the maximum-size byte-count value that should be stored in the bc members of the sg_entry structures. This call passes the value zero (0) to the max_byte_count argument.

--> The dma_map_load interface takes a flags argument as the seventh argument. The flags argument specifies special conditions that the device driver needs the system to perform. In this call, the example passes the following bits: VME_A32_UDATA_D32, VME_BS_NOSWAP, DMA_GUARD_UPPER, and DMA_ALL.

You can pass the bitwise inclusive OR of the following special condition bits defined in /usr/sys/include/io/common/devdriver.h:

Value	Meaning
`DMA_GUARD_UPPER`	Allocates additional resources so that contiguous data overruns are captured by the system map error functions. This bit is probably most useful during device driver development and debugging.
`DMA_GUARD_LOWER`	Allocates additional resources so that contiguous data underruns are captured by the system map error functions. This bit is probably most useful during device driver development and debugging.
`DMA_SLEEP`	Puts the process to sleep if the system cannot allocate the necessary resources to perform a data transfer of size byte_count at the time the driver calls the interface.
`DMA_ALL`	Returns a nonzero value, only if the system can satisfy a DMA transfer of size byte_count.

Note
VME device drivers must not specifiy flags DMA_IN or DMA_OUT as flag parameters to the flags argument of the dma_map_alloc or dma_map_load interfaces. These flags are reserved for use by VME software interfaces that support a hardware DMA engine on the VME adapter. See Section D.3 for a description and examples of using the VME adapter's DMA engine for performing master block transfers. Also refer to the vba_set_dma_addr and vba_dma interfaces to set up and perform master block transfers with the local DMA engine.

In addition, VMEbus device drivers must pass special condition bits defined in /usr/sys/include/io/dec/vme/vbareg.h. The following table describes some of these bits:

Bit Category	Value	Meaning
Swap mode bits	`VME_BS_NOSWAP`	Specifies no byte swapping.
	`VME_BS_BYTE`	Specifies byte swapping in bytes.
	`VME_BS_WORD`	Specifies byte swapping in words.
	`VME_BS_LWORD`	Specifies byte swapping in longwords.
	`VME_BS_QUAD`	Specifies byte swapping in quadwords.
Address space bits	`VME_A16`	Specifies a request for the 16-bit address space.
	`VME_A24`	Specifies a request for the 24-bit address space.
	`VME_A32`	Specifies a request for the 32-bit address space.
	`VME_A64`	Specifies a request for the 64-bit address space.
Transfer size bits	`VME_D08`	Specifies a request for the 8-bit data size.
	`VME_D16`	Specifies a request for the 16-bit data size.
	`VME_D32`	Specifies a request for the 32-bit data size.
Access mode bits	`VME_UDATA`	Specifies user data.
	`VME_UPROG`	Specifies a user program.
	`VME_SDATA`	Specifies supervisory data.
	`VME_SPROG`	Specifies a supervisory program.
CPU allocation space bits		Depending on VMEbus adapter implementation, this bit may not be applicable. To specify dense space on a platform for which the default CPU allocation space is sparse, you use the `VME_DENSE` bit.

5.2 Unloading System Direct Memory Access Resources

To unload system direct memory access (DMA) resources that you previously allocated in a call to dma_map_load, call the dma_map_unload interface. The dma_map_unload interface unloads (invalidates) the resources that were loaded and set up in a previous call to dma_map_load. A call to dma_map_unload does not release or deallocate the resources that were allocated in a previous call to dma_map_alloc unless the driver sets the flags argument to the DMA_DEALLOC bit. If you did not specify the DMA_DEALLOC bit, you must call dma_map_dealloc to release the resources. Use of the dma_map_unload interface makes device drivers more portable between DMA hardware-mapping implementations across different hardware platforms because it masks out any future changes in the kernel- and system-level DMA mapping data structures.

The following code fragment shows a call to the dma_map_unload interface. Because this example driver sets the flags argument to the DMA_DEALLOC bit, it does not need to call dma_map_dealloc.


.
.
.

if (dma_map_load ((unsigned long) bp->b_bcount,
                  (vm_offset_t) bp->b_un.b_addr,
                  bp->b_proc,
                  ctlr,
                  &sc->dma_handle,
                  0,
                  VME_A24_UDATA_D32 | VME_BS_NOSWAP |
                  DMA_GUARD_UPPER | DMA_ALL) == 0) {

.
.
.

dma_map_unload (DMA_DEALLOC, sc->dma_handle); [1]

.
.
.

--> Calls the dma_map_unload interface to unload system DMA resources.
The dma_map_unload interface takes two arguments:
- The first argument is a flags argument. To cause a deallocation of DMA mapping resources, you set the first argument to the special condition bit DMA_DEALLOC.
  This bit setting is analogous to setting the dma_handle_p argument to the value zero (0) for dma_map_load to allocate the DMA mapping resources.
  In this call, the example passes the value zero (0).
- The second argument specifies a handle to DMA resources associated with the mapping of an in-memory I/O buffer onto a controller's I/O bus. This handle provides the information to access bus address/byte count pairs. A bus address/byte count pair is represented by the ba and bc members of an sg_entry structure pointer. Device driver writers can view the DMA handle as the tag to the allocated system resources needed to perform a DMA operation.
  In this call, the example passes the DMA handle stored in the dma_handle member of the driver's softc structure associated with this device. This DMA handle was passed in a previous call to dma_map_load.
Upon successful completion, dma_map_unload returns the value 1. Otherwise, it returns the value zero (0).

5.3 Obtaining the VMEbus Address

There are situations when your device driver may need to know the VMEbus address that corresponds to the I/O handle that the VMEbus configuration code passed to the driver's probe interface. To retrieve this address, you call the vba_get_vmeaddr interface.

The following code fragment shows a call to vba_get_vmeaddr:


.
.
.

vme_addr_t vmeaddr;
io_handle_t addr;
register struct controller *ctlr;
addr = (io_handle_t)ctlr->addr;

.
.
.

vmeaddr = vba_get_vmeaddr (ctlr,  [1]
                           addr); [2]

.
.
.

--> Shows that the first argument specifies a pointer to a controller structure associated with this controller. The controller structure represents an instance of a controller entity, one that connects logically to a bus. A controller can control devices that are directly connected or can perform some other controlling operation, such as a network interface or terminal controller operation.
--> Specifies the I/O handle returned in a previous call to vba_map_csr. Note that the VMEbus configuration code calls the bus adapter-specific map_csr interface on the driver's behalf prior to calling the driver's probe interface. The bus configuration code converts the addresses specified for the Csr1 and Csr2 fields of the driver's VBA_Option entry in sysconfigtab, combined with information in driver structure members addr1_size, addr1_atype, addr2_size, and addr2_atype, into appropriate I/O handles.
The addr2 member of the pointer to the controller structure associated with this VMEbus device would have been used if the driver wanted the second CSR space.

5.4 Reading and Writing Data from a Device Register

To read data from a device register, call the read_io_port interface. The read_io_port interface is a generic interface that maps to a bus- and machine-specific interface that actually performs the read operation. Using this interface to read data from a device register makes the device driver more portable across different bus architectures, different CPU architectures, and different CPU types within the same CPU architecture.

The following code fragment shows a call to read_io_port:


.
.
.

char                        csr;
register struct dmaex_softc *sc = &dmaex_softc[ctlr->ctlr_num];

.
.
.

sc->csr_handle = (io_handle_t)addr; [1]

.
.
.

csr = (char) read_io_port (sc->csr_handle + DMAEX_CSR_OFF, 
                           DMAEX_CSR_SIZE, 0); [2]
if (csr & ERROR)
  return(0);

.
.
.

--> Sets the I/O handle for this DMAEX device to the I/O handle passed to the driver's xxprobe interface by the VMEbus configuration code.
--> Calls the read_io_port interface.
The read_io_port interface takes three arguments:
- The first argument specifies an I/O handle that you can use to reference a device register or memory located in bus address space (either I/O space or memory space). This I/O handle references a device register in the bus address space where the read operation originates. You can perform standard C mathematical operations (addition and subtraction only) on the I/O handle. For example, you can add an offset to or subtract an offset from the I/O handle.
  In this call, the example passes the I/O handle passed to the driver's xxprobe interface, plus the offset value DMAEX_CSR_OFF.
- The second argument specifies the width (in bytes) of the data to be read. Valid values are 1, 2, 3, 4, and 8. Not all CPU platforms or bus adapters support all of these values.
  In this call, the example passes the constant DMAEX_CSR_SIZE.
- The third argument specifies flags to indicate special processing requests. Currently, no flags are used.
Upon successful completion, read_io_port returns the requested data from the device register located in the bus address space: a byte (8 bits), a word (16 bits), a longword (32 bits), or a quadword (64 bits). This interface returns data justified to the low-order byte lane. For example, a byte (8 bits) is always returned in byte lane 0 and a word (16 bits) is always returned in byte lanes 0 and 1.

To write data to a device register, call the write_io_port interface. The write_io_port interface is a generic interface that maps to a bus- and machine-specific interface that actually performs the write operation. Using this interface to write data to a device register makes the device driver more portable across different bus architectures, different CPU architectures, and different CPU types within the same CPU architecture.

The following code fragment shows a call to write_io_port:


.
.
.

register struct dmaex_softc *sc = &dmaex_softc[ctlr->ctlr_num];

.
.
.

sc->csr_handle = (io_handle_t)addr; [1]

.
.
.

write_io_port (sc->csr_handle+DMAEX_CSR_OFF, DMAEX_CSR_SIZE, 0, RESET); [2]
mb(); /* perform a memory barrier */

.
.
.

--> Sets the I/O handle for this DMAEX devicA)to the I/O handle passed to the driver's xxprobe interface by the VMEbus configuration code.
--> Calls the write_io_port interface.
The write_io_port interface takes four arguments:
- The first argument specifies an I/O handle that you can use to reference a device register or memory located in bus address space (either I/O space or memory space). This I/O handle references a device register in the bus address space where the write operation occurs. You can perform standard C mathematical operations (addition and subtraction only) on the I/O handle. For example, you can add an offset to or subtract an offset from the I/O handle.
  In this call, the example passes the I/O handle passed to the driver's xxprobe interface, plus the offset value DMAEX_CSR_OFF.
The write_io_port interface does not return a value.

5.5 Using Events to Synchronize Driver Actions

The following kernel interfaces can be used to synchronize a driver action, such as posting an interrupt request or initiating a device DMA transfer, with a response from an interrupt handler:

event_init - Creates and initializes an event
event_clear - Clears an event after creation or use
event_wait - Waits for an event posting (signaling)
event_post - Posts (signals) an event
event_terminate - Deletes a previously created event

Note
The kernel event interfaces currently are undocumented. They will be documented in a future release of Digital UNIX.

5.5.1 Initializing an Event

The event_init kernel interface takes one argument, a pointer to an event structure of type event_t. (Header file event.h contains the typedef for the event structure.) An example of the interface's use can be found in the probe interface of the /dev/dmaex example device driver.

5.5.2 Clearing an Event

The event_clear kernel interface takes one argument, a pointer to an event structure of type event_t representing a previously created event. Examples of the interface's use can be found in the probe interface, the VME_POST_IRQ ioctl interface, and the strategy interface (DEVICE_DMA_MODE section) of the /dev/dmaex example device driver. In general, an event is cleared after every creation or completed wait.

5.5.3 Waiting For an Event

The event_wait kernel interface takes three arguments: a pointer to an event structure of type event_t representing a previously created event; a Boolean value indicating whether the wait can time out (TRUE) or not (FALSE); and a timeout value expressed as n * hz, where hz is an externally declared kernel global variable that stores clocks ticks per second. The third argument is ignored if the second argument is FALSE.

The event_wait interface also returns a value: 0 if succcessful, nonzero on an error.

Examples of the interface's use can be found in the VME_POST_IRQ ioctl interface and the strategy interface (DEVICE_DMA_MODE section) of the /dev/dmaex example device driver.

5.5.4 Posting an Event

The event_post kernel interface takes one argument, a pointer to an event structure of type event_t representing a previously created event. Examples of the interface's use can be found in the Interrupt Sections of the /dev/dmaex example device driver.

5.5.5 Terminating an Event

The event_terminate kernel interface takes one argument, a pointer to an event structure of type event_t representing a previously created event. Examples of the interface's use can be found in the controller unattach interface of the /dev/dmaex example device driver.

5.6 Passing Mechanism for the busphys_to_iohandle Interface.

The VMEbus flags associated with the atype argument of the vba_map_csr interface are passed to the busphys_to_iohandle interface through the following controller structure pointer member: ctlr->vme_flags. The vme_flags member is defined in /usr/sys/include/io/dec/vme/vbareg.h as the conn_priv[4] member of the controller structure.

The I/O handle returned by busphys_to_iohandle represents the mapped VMEbus address under the following conditions:

If a VMEbus address was previously mapped to a particular address space, a transfer size, an access mode, and a swap mode through the autoconfiguration software
If a VMEbus address was previously mapped to a particular address space, a transfer size, an access mode, and a swap mode through a device driver invoking the vba_map_csr interface

If the specified VMEbus address and the VMEbus flags associated with the atype argument of the vba_map_csr interface could not be found within the mapped VMEbus addresses or the VMEbus adapter support does not provide this functionality, busphys_to_iohandle returns a NULL I/O handle.

The following example calls busphys_to_iohandle to obtain an I/O handle associated with a previously mapped VMEbus address of 0x100, address space of A16, access mode of supervisory, and a swap mode of no swap:


.
.
.

struct controller *ctlr;
io_handle_t       ioh;

ctlr->vme_flags = (void *)(VME_A16_SUPER_ACC | VME_BS_NOSWAP);
ioh = busphys_to_iohandle(0x100,BUS_MEMORY,ctlr);

.
.
.

Note
The I/O handle returned by busphys_to_iohandle may not be a handle that this driver may have mapped through the autoconfiguration software or through a call to the vba_map_csr interface. Other device drivers may have mapped to the same VMEbus address space but used only a portion of that space for device CSR accesses. If this is the case, another device driver may invoke vba_unmap_csr, thus unmapping the I/O handle returned by the busphys_to_iohandle interface.