8    Complex Lock Routines

After you decide that the complex lock method is the appropriate method for locking specific resources, you use the complex lock routines to accomplish the locking. To use complex locks in a kernel module, perform the following tasks:

To illustrate the use of these routines, the chapter uses code from an example kernel module called if_fta, which operates on some FTA device.

8.1    Declaring a Complex Lock Data Structure

Before using a complex lock, declare a complex lock data structure for the resource you want to lock. The following code fragment shows how to declare a complex lock data structure for a specific field of the fta_kern_str structure:

#include <kern/lock.h> [1]
.
struct cmd_buf {
        u_long *req_buf;
        u_long *rsp_buf;
        short  timeout;
        struct cmd_buf *next;
}; [2]

.
.
.
struct fta_kern_str { . struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
}; [3]
.
.
.

  1. Includes the header file /usr/sys/include/kern/lock.h. The lock.h file defines the simple spin lock and complex lock structures that kernel modules use for synchronization on single-processor and multiprocessor systems. [Return to example]

  2. Defines a cmd_buf data structure. The fta_kern_str structure declares two instances of cmd_buf. This structure describes a command queue and is a candidate for locking in a symmetric multiprocessing (SMP) environment. It is necessary to protect the integrity of the data stored in the command queue from multiple writes by more than one kernel thread. [Return to example]

  3. Defines an fta_kern_str data structure. The example shows only those fields related to the discussion of complex locks.

    In this example, the fta_kern_str structure contains the following fields:

    [Return to example]

8.2    Initializing a Complex Lock

After declaring the complex lock data structure, you initialize it by calling the lock_init routine. The following code fragment shows a call to lock_init by the if_fta module's ftaattach routine. The ftaattach routine performs the tasks necessary to establish communication with the actual device. One of these tasks is to initialize any global data structures. Thus, the ftaattach routine initializes the complex lock data structure cmd_buf_q_lock.

The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, and the declaration of the complex lock.


.
.
.
#include <kern/lock.h> [1]
.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; }; [2]
.
.
.
struct fta_kern_str {
.
.
.
struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
}; [3]
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE); [4]
.
.
.
/* Perform other tasks */ }

  1. Includes the header file /usr/sys/include/kern/lock.h. The lock.h file defines the simple spin lock and complex lock structures that kernel modules use for synchronization on single-processor and multiprocessor systems. [Return to example]

  2. Defines a cmd_buf data structure. The fta_kern_str structure declares two instances of cmd_buf. This structure describes a command queue and is a candidate for locking in an SMP environment. It is necessary to protect the integrity of the data stored in the command queue from multiple writes by more than one kernel thread. [Return to example]

  3. Defines an fta_kern_str data structure. The example shows only those fields related to the discussion of complex locks.

    In this example, the fta_kern_str structure contains the following fields:

    [Return to example]

  4. Calls the lock_init routine to initialize the simple lock structure called cmd_buf_q_lock.

    The lock_init routine takes two arguments:

    [Return to example]

8.3    Performing Access Operations on a Complex Lock

After declaring and initializing the complex lock data structure, you can perform the following access operations on the complex lock:

Each of these tasks is discussed in the following sections.

8.3.1    Asserting a Complex Lock

After declaring and initializing the complex lock data structure, you can assert a complex lock with read-only access or a complex lock with write access by calling the lock_read or lock_write routine. The following sections describe how to use these routines.

8.3.1.1    Asserting a Complex Lock with Read-Only Access

The lock_read routine asserts a lock with read-only access for the resource associated with the specified lock structure pointer. The following code fragment shows a call to lock_read by the if_fta module's ftaioctl routine.

The ftaioctl routine is called as the result of an ioctl system call.

The ftaioctl routine performs the following tasks:

The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, the declaration of the complex lock structure in the fta_kern_str structure, and the initialization of the complex lock structure by the kernel module's ftaattach routine. Section 8.2 provides descriptions of these tasks.

#include <kern/lock.h>

.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; };
.
.
.
struct fta_kern_str {
.
.
.
struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
};
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE);
.
.
.
/* Perform other tasks */ }
.
.
.
ftaioctl(register struct ifnet *ifp, unsigned int cmd, caddr_t dataifp) { struct fta_kern_str *sc = &fta_kern_str[ifp->if_unit];
.
.
.
switch (cmd) { case SIOCENABLBACK: {
.
.
.
if (ifp->if_flags & IFF_RUNNING) { [1] lock_read(&sc->cmd_buf_q_lock);   /* Performs read operation on the resource */ if(sc->q_first->req_buf = (u_long*)(data);
.
.
.
}

  1. Calls the lock_read routine if the IFF_RUNNING bit flag is set in the if_flags field of the ifp structure pointer.

    The lock_read routine takes one argument: a pointer to the complex lock structure lock. This is the lock structure associated with the resource on which you want to assert a complex lock with read-only access. The ftaioctl routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer.

    Figure 8-1 shows what happens when multiple instances of the if_fta kernel module assert a read-only complex lock on the specified code block. As the figure shows, kernel threads from the if_fta kernel module executing on CPU1, CPU2, and CPU3 assert read-only complex locks on the specified code block. The lock_read routine allows multiple kernel threads to have read-only access to the resource at the same time. When a read lock is asserted, the protected resource is guaranteed not to change. In this case, the cmd_buf resource is guaranteed not to change. [Return to example]

Figure 8-1:  Three Instances of the if_fta Module Asserting a Read-Only Complex Lock

8.3.1.2    Asserting a Complex Lock with Write Access

The lock_write routine asserts a lock with exclusive write access for the resource associated with the specified lock structure pointer. This means that once a write lock is asserted, no other kernel thread can gain read or write access to the resource until it is released.

The following code fragment shows a call to lock_write by the if_fta module's ftaioctl routine.

The ftaioctl routine is called as the result of an ioctl system call.

The ftaioctl routine performs the following tasks:

The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, the declaration of the complex lock structure in the fta_kern_str structure, and the initialization of the complex lock structure by the kernel module's ftaattach routine. Section 8.2 provides descriptions of these tasks.

#include <kern/lock.h>

.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; };
.
.
.
struct fta_kern_str { . struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command /* /* request queue */
.
.
.
};
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE);
.
.
.
/* Perform other tasks */ }
.
.
.
ftaioctl(register struct ifnet *ifp, unsigned int cmd, caddr_t data) { struct fta_kern_str *sc = &fta_kern_str[ifp->if_unit];
.
.
.
switch (cmd) { case SIOCENABLBACK: {
.
.
.
if (ifp->if_flags & IFF_RUNNING) { [1] lock_write(&sc->cmd_buf_q_lock); sc->q_first->req_buf = (u_long*) (data);    

  1. Calls the lock_write routine if the IFF_RUNNING bit flag is set in the if_flags field of the ifp structure pointer.

    The lock_write routine takes one argument: a pointer to the complex lock structure lock. This is the lock structure associated with the resource on which you want to assert a complex lock with write access. The ftaioctl routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer.

    Figure 8-2 shows what happens when multiple instances of the if_fta kernel module assert a write complex lock on the specified code block. As the figure shows, kernel threads from the if_fta module executing on CPU1, CPU2, and CPU3 assert write complex locks on the specified code block. The kernel thread emanating from CPU3 asserts the write complex lock before the kernel threads emanating from CPU1 and CPU2. The kernel thread emanating from CPU3 writes to the req_buf field.

    The lock_write routine blocks (puts to sleep) the kernel threads emanating from CPU1 and CPU2 by placing the requests on a lock queue. This shows that once lock_write successfully asserts a complex write lock, no other kernel thread can gain read or write access to the resource until the resource is released. [Return to example]

Figure 8-2:  Three Instances of the if_fta Module Asserting a Write Complex Lock

8.3.2    Releasing a Previously Asserted Complex Lock

After you finish manipulating the resource associated with the complex lock, you need to release the lock. To release a complex lock that you previously asserted with a call to lock_read or lock_write, call the lock_done routine. The following code fragment shows a call to lock_done by the if_fta kernel module's ftaioctl routine.

The ftaioctl routine is called as the result of an ioctl system call.

The ftaioctl routine performs the following tasks:

The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, the declaration of the complex lock structure in the fta_kern_str structure, the initialization of the complex lock structure by the module's ftaattach routine, and the assertion of a complex write lock on the code block by the kernel module's ftaioctl routine. Section 8.2 and Section 8.3.1.2 provide descriptions of these tasks.


.
.
.
#include <kern/lock.h>
.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; };
.
.
.
struct fta_kern_str {
.
.
.
struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
};
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE);
.
.
.
/* Perform other tasks */ }
.
.
.
ftaioctl(register struct ifnet *ifp, unsigned int cmd, caddr_t data) { struct fta_kern_str *sc = &fta_kern_str[ifp->if_unit];
.
.
.
switch (cmd) { case SIOCENABLBACK: {
.
.
.
if (ifp->if_flags & IFF_RUNNING) { lock_write(&sc->cmd_buf_q_lock); sc->q_first->req_buf = (u_long*) (data);
.
.
.
lock_done(&sc->cmd_buf_q_lock); [1]
.
.
.
}

  1. Calls the lock_done routine to release the complex write lock previously asserted by lock_write.

    The lock_done routine takes one argument: a pointer to the complex lock structure lock. This is the lock structure associated with the resource on which you want to assert a complex lock with write access. The ftaioctl routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer.

    Figure 8-3 shows what happens when one instance of the if_fta module releases a previously asserted complex write lock on the code block that writes to the command buffer queue. As the figure shows, the CPU3 kernel thread releases the complex write lock on the code block that writes to the command buffer queue. The CPU1 and CPU2 kernel threads are blocked, waiting on the wait queue for the complex write lock to be freed. Because the CPU1 kernel thread is first on the wait queue, it now obtains the complex write lock. Furthermore, the figure shows that the CPU3 kernel thread makes another attempt to assert a complex write lock on the code block. This time lock_write blocks (puts to sleep) the CPU3 kernel thread by placing it on the wait queue behind the CPU2 kernel thread. [Return to example]

Figure 8-3:  One Instance of the if_fta Module Releasing a Complex Write Lock

8.3.3    Trying to Assert a Complex Lock

After declaring and initializing the complex lock data structure, you can try to assert a complex lock with read-only access or a complex lock with write access by calling the lock_try_read or lock_try_write routine. Unlike the lock_read or lock_write routines, the lock_try_read and lock_try_write routines do not block if the lock associated with the resource is owned by another kernel thread.

The following sections describe how to use these routines.

8.3.3.1    Trying to Assert a Complex Lock with Read-Only Access

To try to assert a complex lock with read-only access, call the lock_try_read routine. The lock_try_read routine tries to assert a complex lock (without blocking) with read-only access for the resource associated with the specified lock structure pointer.

The following code fragment shows a call to lock_try_read by the if_fta module's ftaioctl routine. The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, the declaration of the complex lock structure in the fta_kern_str structure, and the initialization of the complex lock structure by the module's ftaattach routine. Section 8.2 provides descriptions of these tasks. In addition, the code fragment shows a call to lock_done if the complex read-only lock is successfully asserted.


.
.
.
#include <kern/lock.h>
.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; };
.
.
.
struct fta_kern_str {
.
.
.
struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
};
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE);
.
.
.
/* Perform other tasks */ }
.
.
.
ftaioctl(register struct ifnet *ifp, unsigned int cmd, caddr_t data) { struct fta_kern_str *sc = &fta_kern_str[ifp->if_unit]; boolean_t try_ret_val; [1]
.
.
.
switch (cmd) { case SIOCENABLBACK: {
.
.
.
if (ifp->if_flags & IFF_RUNNING) { [2] try_ret_val = lock_try_read(&sc->cmd_buf_q_lock); if (try_ret_val == TRUE) { [3] if (sc->q_first->req_buf == (u_long*) (data)) {
.
.
.
lock_done(&sc->cmd_buf_q_lock); [4] } } }
.
.
.
else [5]
.
.
.
/* Code that executes when try_ret_val == FALSE */
.
.
.
}
.
.
.
}
.
.
.
}

  1. Declares a variable to store one of the following return values from the lock_try_read routine:

    TRUE

    The attempt to acquire the read-only complex lock was successful.

    FALSE

    The attempt to acquire the read-only complex lock was unsuccessful.

    [Return to example]

  2. Calls the lock_try_read routine if the IFF_RUNNING bit flag is set in the if_flags field of the ifp structure pointer.

    The lock_try_read routine takes one argument: a pointer to the complex lock structure lock. This is the lock structure associated with the resource on which you want to try to assert a complex lock with read-only access. The ftaioctl routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer. [Return to example]

  3. If the return from lock_try_read is TRUE, obtains the read-only complex lock on the code block that performs the read operation. [Return to example]

  4. After completing the read operation, releases the read-only complex lock by calling lock_done. [Return to example]

  5. If the return from lock_try_read is FALSE, did not obtain the read-only complex lock on the code block that performs the read operation. In this case, it is not necessary to call lock_done.

    Figure 8-4 shows what happens when two instances of the if_fta module attempt to assert a read-only complex lock on the code block that performs a read operation on the resource. As the figure shows, both the CPU1 and CPU2 kernel threads try to assert a read-only complex lock on the code block that performs a read operation on the command buffer queue. Because this is a read-only operation, the CPU1 and CPU2 kernel threads obtain the read-only complex lock, and as a result, lock_try_read returns the value TRUE in both cases. [Return to example]

Figure 8-4:  The if_fta Module Trying to Assert a Complex Read-Only Lock

8.3.3.2    Trying to Assert a Complex Lock with Write Access

To try to assert a complex lock with write access, call the lock_try_write routine. The lock_try_write routine tries to assert a complex lock (without blocking) with write access for the resource associated with the specified lock structure pointer.

The following code fragment shows a call to lock_try_write by the if_fta module's ftaioctl routine. The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, the declaration of the complex lock structure in the fta_kern_str structure, and the initialization of the complex lock structure by the module's ftaattach routine. Section 8.2 provides descriptions of these tasks. In addition, the code fragment shows a call to lock_done if the complex write lock is successfully asserted.


.
.
.
#include <kern/lock.h>
.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; };
.
.
.
struct fta_kern_str {
.
.
.
struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
};
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE);
.
.
.
/* Perform other tasks */ }
.
.
.
ftaioctl(register struct ifnet *ifp, unsigned int cmd, caddr_t data) { struct fta_kern_str *sc = &fta_kern_str[ifp->if_unit]; boolean_t try_ret_val; [1]
.
.
.
switch (cmd) { case SIOCENABLBACK: {
.
.
.
if (ifp->if_flags & IFF_RUNNING) { [2] try_ret_val = lock_try_write(&sc->cmd_buf_q_lock); if (try_ret_val == TRUE) { [3] sc->q_first->req_buf = (u_long*) (data);
.
.
.
lock_done(&sc->cmd_buf_q_lock); [4] }
.
.
.
else [5]
.
.
.
/* Code that executes when try_ret_val == FALSE */
.
.
.
}
.
.
.
}
.
.
.
}

  1. Declares a variable to store one of the following return values from the lock_try_write routine:

    TRUE

    The attempt to acquire the write complex lock was successful.

    FALSE

    The attempt to acquire the write complex lock was unsuccessful.

    [Return to example]

  2. Calls the lock_try_write routine if the IFF_RUNNING bit flag is set in the if_flags field of the ifp structure pointer.

    The lock_try_write routine takes one argument: a pointer to the complex lock structure lock. This is the lock structure associated with the resource on which you want to try to assert write access. The ftaioctl routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer. [Return to example]

  3. If the return from lock_try_write is TRUE, obtains the write complex lock on the code block that performs the write operation. [Return to example]

  4. After completing the write operation, releases the write complex lock by calling lock_done. [Return to example]

  5. If the return from lock_try_write is FALSE, did not obtain the write complex lock on the code block that performs the write operation. In this case, it is not necessary to call lock_done.

    Figure 8-5 shows what happens when two instances of the if_fta module attempt to assert a write complex lock on the code block that performs a write operation on the resource. As the figure shows, both the CPU1 and CPU2 kernel threads try to assert a write complex lock on the code block that performs a write operation on the command buffer queue. The CPU1 kernel thread obtains the write complex lock first and as a result lock_try_write returns the value TRUE. Because the CPU2 kernel thread was not successful in obtaining the write complex lock, lock_try_write immediately returns (does not block the kernel thread) the value FALSE. [Return to example]

Figure 8-5:  The if_fta Module Trying to Assert a Complex Write Lock

8.4    Terminating a Complex Lock

After unlocking a complex read or write lock and knowing that you are finished using the lock for this resource, you can terminate the lock by calling the lock_terminate routine. Typically, you terminate any locks in the kernel module's controller (or device) unattach routine. These routines are associated with loadable kernel modules. One task associated with a controller or device unattach routine is to terminate any locks initialized in the kernel module's attach routine.

The following code fragment shows a call to lock_terminate by the if_fta module's fta_ctlr_unattach routine. The code fragment also shows the include file associated with complex locks, definitions of the cmd_buf and fta_kern_str structures, the declaration of the complex lock structure in the fta_kern_str structure, and the initialization of the complex lock structure by the kernel module's ftaattach routine. Section 8.2 provides descriptions of these tasks. In addition, the code fragment shows calls to lock_write and lock_done.


.
.
.
#include <kern/lock.h>
.
.
.
struct cmd_buf { u_long *req_buf; u_long *rsp_buf; short timeout; struct cmd_buf *next; };
.
.
.
struct fta_kern_str {
.
.
.
struct cmd_buf *q_first; /* first in the request queue */ struct cmd_buf *q_last; /* last in the request queue */ lock_data_t cmd_buf_q_lock; /* lock for the command */ /* request queue */
.
.
.
};
.
.
.
ftaattach(struct controller *ctlr) { struct fta_kern_str *sc = &fta_kern_str[ctlr->ctlr_num];
.
.
.
/* Tasks to perform controller-specific initialization */
.
.
.
lock_init(&sc->cmd_buf_q_lock, TRUE);
.
.
.
/* Perform other tasks */ }
.
.
.
ftaioctl(register struct ifnet *ifp, unsigned int cmd, caddr_t data) { struct fta_kern_str *sc = &fta_kern_str[ifp->if_unit];
.
.
.
switch (cmd) { case SIOCENABLBACK: {
.
.
.
if (ifp->if_flags & IFF_RUNNING) { lock_write(&sc->cmd_buf_q_lock); sc->q_first->req_buf = (u_long*) (data);
.
.
.
lock_done(&sc->cmd_buf_q_lock);
.
.
.
}
.
.
.
fta_ctlr_unattach(struct bus *bus, struct controller *ctlr) { register int unit = ctlr->ctlr_num;   if ((unit > num_fta) || (unit < 0) { return(1); }   if (fta_is_dynamic == 0) { return(1); } /* Performs controller unattach tasks */
.
.
.
lock_terminate(&sc->cmd_buf_q_lock); [1]
.
.
.
}

  1. Calls the lock_terminate routine to determine that the if_fta module is permanently done using this complex lock.

    The lock_terminate routine takes one argument: a pointer to the complex lock structure lock. In this call, the fta_ctlr_unattach routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer. In calling lock_terminate, the if_fta module must not reference this complex lock again. [Return to example]