9    Kernel Threads

This chapter discusses the following topics associated with kernel threads:

• The advantages of using kernel threads

• Kernel threads execution

• Issues related to using kernel threads

• Kernel threads operations

• The thread and task data structures

In addition, this chapter discusses the routines that allow you to perform kernel thread operations. Specifically, these routines allow you to:

• Create and start a kernel thread

• Block (put to sleep) a kernel thread

• Unblock (wake up) kernel threads

• Terminate a kernel thread

• Set a timer for the current kernel thread

• Obtain the current kernel thread

9.1    Using Kernel Threads in Kernel Modules

A thread is a single, sequential flow of control within a program. Within a single thread is a single point of execution. Applications use threads to improve their performance (throughput, computational speed, and responsiveness). To start, terminate, delete, and perform other operations on threads, the application programmer calls the routines that the DECthreads product provides.

The term kernel thread distinguishes between the threads that applications use. A kernel thread is a single sequential flow of control within a kernel module or other systems-based program. The kernel module or other systems-based program uses the routines (instead of a threads library package such as DECthreads) to start, terminate, delete, and perform other kernel thread operations.

Kernel threads execute within (and share) a single address space. Therefore, kernel threads read from and write to the same memory locations.

You use kernel threads to improve the performance of a kernel module. Multiple kernel threads are useful in a multiprocessor environment, where kernel threads run concurrently on separate CPUs. However, multiple kernel threads also improve kernel module performance on single-processor systems by permitting the overlap of input, output, or other slow operations with computational operations.

Kernel threads allow kernel modules to perform other useful work while waiting for a device to produce its next event, such as the completion of a disk transfer or the receipt of a packet from the network.

Typically, you use kernel threads in kernel modules when:

• The kernel module must perform a long operation

  One example of a long operation is the reset sequences associated with a multistep device.

  One reason for creating a kernel thread to perform a long operation is to prevent the kernel module from running at a high interrupt priority level (IPL) for long periods of time.

• The resource or resources associated with that operation are not available

  This situation refers to allocating memory or accessing address space that might cause a page fault.

• The operation performed on the resource (for example, blocking) is illegal.

  One example of this operation is that access to a data item is not allowed at an elevated IPL, for example, the proc structure.

Figure 9-1 shows one example of the previously described situations. As the figure shows, a kernel module must check a number of device state changes. One of these device state changes checks for an adapter fatal error condition. If the fatal error condition occurs, the kernel module must reset the adapter. The code that resets the adapter must block to accomplish the adapter reset operation. Furthermore, the only time this error can occur is during a device interrupt. It is not legal to block in an interrupt service routine. Therefore, the figure shows that the interrupt service routine for the kernel module calls an xxstate routine that handles all of the state changes. This routine creates a kernel thread called xxerr that starts up when the adapter becomes operational. The job of this kernel thread is to reset the adapter when a fatal error occurs. Note that it is legal for this kernel thread to perform blocking operations.

Figure 9-1:  Using Kernel Threads in a Kernel Module

9.1.1    Kernel Threads Execution

You can view multiple kernel threads in a program as executing simultaneously. However, you cannot make any assumptions about the relative start or finish times of kernel threads or the sequence in which they execute. You can influence the scheduling of kernel threads by setting scheduling and policy priority.

Each kernel thread has its own unique thread identifier. This thread identifier is a pointer to the thread data structure associated with the kernel thread. The kernel threads creation routines return this thread data structure pointer to the kernel module after they successfully create and start the kernel thread. Kernel modules use this pointer as a handle to a specific kernel thread in calls to other kernel thread routines.

A kernel thread changes states during the course of its execution and is always in one of the following states:

• Waiting

  The kernel thread is not eligible to execute because it is synchronizing with another kernel thread or with an external event, such as I/O.

• Ready

  The kernel thread is eligible to be executed by a CPU.

• Running

  The kernel thread is currently being executed by a CPU.

• Terminated

  The kernel thread has completed all of its work.

9.1.2    Issues Related to Using Kernel Threads

When you design and code a kernel module that uses the kernel thread routines, consider the following issues:

• Interplay among kernel threads

  Using kernel threads can simplify the coding and designing of a kernel module. However, you need to be sure that the synchronization and interplay among kernel threads is correct. You use simple and complex locks to synchronize access to data.

• Race conditions

  A race condition is a programming error that causes unpredictable and erroneous program behavior. Specifically, the error occurs when two or more kernel threads perform an operation and the result of the operation depends on unpredictable timing factors, for example, when each kernel thread executes and waits and when each kernel thread completes the operation.

• Deadlocks

  A deadlock is a programming error that causes two or more kernel threads to be blocked indefinitely. Specifically, the error occurs when a kernel thread holds a resource while waiting for a resource held by another kernel thread and that kernel thread is also waiting for the first kernel thread's resource.

• Priority inversion

  Priority inversion occurs when the interaction among three or more kernel threads blocks the highest-priority kernel thread from executing. For example, a high-priority kernel thread waits for a resource locked by a low-priority kernel thread, and the low-priority kernel thread waits while a middle-priority kernel thread executes. The high-priority kernel thread is made to wait while a kernel thread of lower priority (the middle-priority kernel thread) executes.

  To avoid priority inversion, associate a priority with each resource that is at least as high as the highest-priority kernel thread that will use it, and force any kernel thread using that object to first raise its priority to that associated with the object.

9.1.3    Kernel Threads Operations

Table 9-1 lists the routines associated with kernel threads and describes the operations they perform.

Table 9-1:  Summary of Operations That Kernel Thread Routines Perform

9.2    Using the thread and task Data Structures

This section discusses the two data structures that kernel thread routines use: thread and task. The thread data structure contains kernel thread information. Kernel modules typically use the wait_result field (along with the current_thread routine) to check for the result of the wait. The header file /usr/sys/include/kern/thread.h shows a typedef statement that assigns the alternate name thread_t for a pointer to the thread structure. Many of the kernel thread routines operate on these pointers to thread structures.

The thread structure is an opaque data structure; that is, all of its associated fields (except for the wait_result field) are referenced and manipulated by the operating system and not by the user of kernel threads.

The task data structure contains task-related information. The header file /usr/sys/include/kern/task.h shows a typedef statement that assigns the alternate name task_t for a pointer to the task structure. Some of the kernel thread routines require that you pass a pointer to the task structure.

The task structure is an opaque data structure; that is, all of its associated fields are referenced and manipulated by the operating system and not by the user of kernel threads.

9.3    Creating and Starting a Kernel Thread

You can create and start a kernel thread with the following routines:

• kernel_thread_w_arg

  Starts a kernel thread with a calling argument passed in.

• kernel_isrthread

  Starts a fixed-priority kernel thread dedicated to interrupt service.

The following sections describe each of these routines.

9.3.1    Creating and Starting a Kernel Thread at a Specified Entry Point

To create and start a kernel thread at a specified entry point and with a specified argument, call the kernel_thread_w_arg routine. The kernel_thread_w_arg routine creates and starts a kernel thread in the specified task at the specified entry point with a specified argument. The kernel_thread_w_arg routine passes the specified argument to the newly created kernel thread. The kernel_thread_w_arg routine creates and starts a kernel thread with timeshare scheduling. A kernel thread created with timeshare scheduling means that its priority degrades if it consumes an inordinate amount of CPU resources. A kernel module should call kernel_thread_w_arg only for long-running tasks. A kernel module should always attach a kernel thread to the first task.

The kernel_thread_w_arg routine is actually a convenience wrapper for the thread_create routine (which creates the kernel thread) and the thread_start routine (which starts the newly created kernel thread).

The following code fragment shows a call to kernel_thread_w_arg by the if_fta module's fta_transition_state routine. The fta_transition_state routine changes the state of the kernel module by performing certain fixed functions for any given state.

#include <kern/thread.h> [1]

.
.
.
#define ADAP "fta"

.
.
.
extern task_t first_task; [2]

.
.
.
struct  fta_kern_str {

.
.
.
short reinit_thread_started; /* reinit thread running? */

.
.
.
}; [3]

.
.
.
struct  ifnet {

.
.
.
short if_unit; /* subunit for lower-level driver */

.
.
.
}; 

.
.
.
fta_transition_state(struct fta_kern_str *sc,
                     short unit,
                     short state)
{

.
.
.
switch(state) {

.
.
.
case PI_OPERATIONAL: {
          int s;
          NODATA_CMD *req_buff;
          thread_t thread; [4]
 
          if (sc->reinit_thread_started == FALSE) { [5]
 
              thread = kernel_thread_w_arg(first_task,
                                           fta_error_recovery,
                                           (void *)sc); [6]
              if (thread == NULL) { [7]
                  printf("%s%d: Cannot start error recovery thread.\n",
                         ADAP, ifp->if_unit);
              }
              sc->reinit_thread_started = TRUE;
          }
}

1. Includes the header file /usr/sys/include/kern/thread.h. The thread.h file defines structures that kernel thread routines use. [Return to example]

2. Declares a pointer to a task structure and calls it first_task. Every kernel thread must be part of a task. You pass this pointer to the kernel_thread_w_arg routine. [Return to example]

3. Defines an fta_kern_str data structure. The example shows only the field related to the discussion of the kernel_thread_w_arg routine. [Return to example]

4. Declares a pointer to a thread structure and calls it thread. This variable stores the thread structure pointer returned by kernel_thread_w_arg. [Return to example]

5. If the reinitialized kernel thread evaluates to FALSE (the reinit kernel thread is not running), calls the kernel_thread_w_arg routine. [Return to example]

6. Calls the kernel_thread_w_arg routine.

   The kernel_thread_w_arg routine takes three arguments:

   • The first argument specifies a pointer to a task structure. This pointer identifies the task in which the kernel_thread_w_arg routine starts the newly created kernel thread. In this call, the fta_transition_state routine passes a task structure called first_task.

   • The second argument specifies a pointer to a routine that is the entry point for the newly created kernel thread. In this call, the entry point for the newly created kernel thread is the fta_error_recovery routine. The fta_error_recovery routine is a kernel thread that starts up when the adapter becomes operational. This kernel thread is responsible for resetting the adapter in the event of a fatal error.

   • The third argument specifies an argument that kernel_thread_w_arg passes to the entry point specified in the second argument. In this call, the fta_transition_state routine passes a pointer to the fta_kern_str structure. The fta_error_recovery routine performs a variety of tasks that require the fta_kern_str structure.

   [Return to example]

7. Upon successful completion, kernel_thread_w_arg returns a pointer to the thread structure associated with the kernel thread started at the specified entry point. Kernel modules can use this pointer as a handle to a specific kernel thread in calls to other kernel thread routines.

   The fta_transition_state routine checks the return. If the return is NULL, kernel_thread_w_arg did not create the error recovery kernel thread. The fta_transition_state routine calls printf to display an appropriate message on the console terminal. If the return is not NULL, fta_transition_state sets the reinit_thread_started field to the value TRUE to indicate that the error recovery kernel thread is started. [Return to example]

To create and start a fixed-priority kernel thread dedicated to interrupt service, call the kernel_isrthread routine. The kernel_isrthread routine creates and starts a kernel thread at the specified entry point. This kernel thread handles only interrupt service requests in the specified task and at the specified priority level. A kernel module should always attach a kernel thread to the first task.

The following code fragment shows a call to kernel_isrthread by the if_fta module's ftaprobe routine. The ftaprobe routine determines if the adapter exists, fills in a variety of register values, and initializes a variety of descriptors.

.
.
.
#include <kern/thread.h> [1]

.
.
.
extern task_t first_task; [2]

.
.
.
ftaprobe(io_handle_t reg,
         struct  controller *ctlr)
{

.
.
.
thread = kernel_isrthread(first_task,
                                     fta_rec_intr,
                                     BASEPRI_SYSTEM); [3]

.
.
.
}

1. Includes the header file /usr/sys/include/kern/thread.h. The thread.h file defines structures that kernel thread routines use. [Return to example]

2. Declares a pointer to a task structure and calls it first_task. Every kernel thread must be part of a task. You pass this pointer to the kernel_isrthread routine. [Return to example]

3. Calls the kernel_isrthread routine.

   The kernel_isrthread routine takes three arguments:

   • The first argument specifies a pointer to a task structure. This pointer identifies the task in which the kernel_isrthread routine starts the newly created kernel thread dedicated to interrupt service handling. In this call, the ftaprobe routine passes a task structure called first_task.

   • The second argument specifies a pointer to a routine that is the entry point for the newly created kernel thread. In this call, the entry point for the newly created kernel thread is the fta_rec_intr routine. The fta_rec_intr routine is a kernel thread that starts up when the kernel module discovers a receive type device interrupt. This kernel thread is responsible for handling the receive type interrupt.

   • The third argument specifies the scheduling priority level for the newly created kernel thread.

     The following priority usage table describes the possible scheduling priorities. The first column shows a range of priorities. The second column shows an associated scheduling priority constant defined in <src/kernel/kern/sched.h> (if applicable). The third column describes the usage of the priority ranges. To specify a scheduling priority of 38, you pass the constant BASEPRI_SYSTEM, as shown in the example. To specify a scheduling priority of 33, you can pass the following: BASEPRI_HIGHEST + 1.

   [Return to example]

9.4    Blocking (Putting to Sleep) a Kernel Thread

The routines you use to block (put to sleep) a kernel thread depend on whether or not the block (sleep) can be interrupted. For interruptable sleep (that is, the kernel thread can take asynchronous signals), you must call the symmetric multiprocessor (SMP) sleep call, mpsleep (see Section 9.4.2).

For uninterruptable sleep, use one of the following routines:

• assert_wait_mesg

  Call this routine to assert that the current kernel thread is about to block until some specified event occurs. You use this routine with the thread_block routine, which actually blocks (puts to sleep) the current kernel thread.

• thread_block

  Call this routine to block the current kernel thread and select the next kernel thread to start.

These routines are described in the following sections.

9.4.1    Asserting That the Current Kernel Thread Is About to Block Until the Specified Event Occurs

To assert that the current kernel thread is about to block until some specified event occurs, call the assert_wait_mesg routine. To actually block (put to sleep) the current kernel thread, call thread_block.

The following code fragment shows a call to assert_wait_mesg and thread_block by the if_fta module's fta_error_recovery routine. The fta_error_recovery routine is a kernel thread that starts up when the adapter becomes operational. This kernel thread resets the adapter if a fatal error occurs. The code fragment also shows the code that contains the call to kernel_thread_w_arg, which calls fta_error_recovery.

.
.
.
#include <kern/thread.h> [1]

.
.
.
#define ADAP "fta"

.
.
.
extern task_t first_task; [2]

.
.
.
struct  fta_kern_str {

.
.
.
short reinit_thread_started; /* reinit thread running? */

.
.
.
short error_recovery_flag; /* flag to wake up a process */

.
.
.
}; [3]

.
.
.
struct  ifnet {

.
.
.
short if_unit; /* subunit for lower-level driver */

.
.
.
}; 

.
.
.
fta_transition_state(struct fta_kern_str *sc,
                     short unit,
                     short state)
{

.
.
.
switch(state) {

.
.
.
case PI_OPERATIONAL: {
          int s;
          NODATA_CMD *req_buff;
          thread_t thread; [4]
 
          if (sc->reinit_thread_started == FALSE) { [5]
 
              thread = kernel_thread_w_arg(first_task,
                                           fta_error_recovery,
                                           (void *)sc); [6]
              if (thread == NULL) { [7]
                  printf("%s%d: Cannot start error recovery thread.\n",
                         ADAP, ifp->if_unit);
              }
              sc->reinit_thread_started = TRUE;
          }

.
.
.
void fta_error_recovery(struct fta_kern_str *sc) [8]
{
    struct ifnet *ifp;
 
 
    /*
     * Collect the argument left by the kernel_thread_w_arg().
     */
    ifp = &sc->is_if;
 
    for(;;) { [9]
        assert_wait_mesg((vm_offset_t)&sc->error_recovery_flag,
                          TRUE,"ftaerr"); [10]
        thread_block(); [11]
 
/* Performs tasks to reset the adapter */

.
.
.
}

.
.
.
}

1. Includes the header file /usr/sys/include/kern/thread.h. The thread.h file defines structures that kernel thread routines use. [Return to example]

2. Declares a pointer to a task structure and calls it first_task. Every kernel thread must be part of a task. You pass this pointer to the kernel_thread_w_arg routine. [Return to example]

3. Defines an fta_kern_str data structure. The example shows only the fields related to the discussion of the kernel_thread_w_arg, assert_wait_mesg, and thread_block routines. [Return to example]

4. Declares a pointer to a thread structure and calls it thread. This variable stores the thread structure pointer returned by kernel_thread_w_arg. [Return to example]

5. If the reinitialized kernel thread evaluates to FALSE (the reinit kernel thread is not running), calls the kernel_thread_w_arg routine. [Return to example]

6. Calls the kernel_thread_w_arg routine.

   The kernel_thread_w_arg routine takes three arguments:

   • The first argument specifies a pointer to a task structure. This pointer identifies the task in which the kernel_thread_w_arg routine starts the newly created kernel thread. In this call, the fta_transition_state routine passes a task structure called first_task.

   • The second argument specifies a pointer to a routine that is the entry point for the newly created kernel thread. In this call, the entry point for the newly created kernel thread is the fta_error_recovery routine. The fta_error_recovery routine is a kernel thread that starts up when the adapter becomes operational. This kernel thread is responsible for resetting the adapter in the event of a fatal error.

   • The third argument specifies an argument that kernel_thread_w_arg passes to the entry point specified in the second argument. In this call, the fta_transition_state routine passes a pointer to the fta_kern_str structure. The fta_error_recovery routine performs a variety of tasks that require the fta_kern_str structure.

   [Return to example]

7. Upon successful completion, kernel_thread_w_arg returns a pointer to the thread structure associated with the kernel thread started at the specified entry point. Kernel modules can use this pointer as a handle to a specific kernel thread in calls to other kernel thread routines.

   The fta_transition_state routine checks the return. If the return is NULL, kernel_thread_w_arg did not create the error recovery kernel thread. The fta_transition_state routine calls printf to display an appropriate message on the console terminal. If the return is not NULL, fta_transition_state sets the reinit_thread_started field to the value TRUE to indicate that the error recovery kernel thread is started. [Return to example]

8. The fta_error_recovery routine is a kernel thread that starts up when the adapter becomes operational. This kernel thread resets the adapter if a fatal error occurs.

   A fatal error requires resetting the adapter; this error is discovered during a device interrrupt. It is necessary to block in the interrupt service routine while resetting the adapter. Because it is not legal to block in an interrupt service routine, the fta_transition_state calls this kernel thread to perform the reset operation on the adapter.

   The kernel_thread_w_arg routine passes the kern_str structure pointer to fta_error_recovery. [Return to example]

9. Sets up an infinite loop that executes when the adapter becomes operational. [Return to example]

10. Calls the assert_wait_mesg routine to assert that the current kernel thread is about to block (sleep).

    The assert_wait_mesg routine takes three arguments:

    • The first argument specifies the event associated with the current kernel thread. In this call, the event associated with the current kernel thread is stored in the error_recovery_flag field.

    • The second argument specifies a Boolean value that indicates how the kernel thread is awakened. You can pass one of the following values:

      TRUE

      The current kernel thread is interruptible. This value means that a signal can awaken the current kernel thread.

      FALSE

      The current kernel thread is not interruptible. This value means that only the specified event can awaken the current kernel thread.

      In this call, the value TRUE is passed.

    • The third argument specifies a mnemonic for the type of wait. The /bin/ps command uses this mnemonic to print out more meaningful messages about a process. In this call, the fta_error_recovery routine passes the string ftaerr.

    The assert_wait_mesg routine does not return a value. [Return to example]

11. Calls the thread_block routine. The thread_block routine blocks (puts to sleep) the current kernel thread and selects the next kernel thread to start (run). The routine schedules the next kernel thread onto this CPU.

    The thread_block routine does not return a value. [Return to example]

To block the current kernel thread, call the mpsleep routine--the symmetric multiprocessor (SMP) sleep call. The following code fragment shows a call to mpsleep by the if_fta module's fta_error_recovery routine. The fta_error_recovery routine is a kernel thread that starts up when the adapter becomes operational. This kernel thread resets the adapter if a fatal error occurs. The code fragment also shows the use of a simple lock with the mpsleep routine.

.
.
.
struct  fta_kern_str {

.
.
.
short error_recovery_flag; /* flag to wake up a process */

.
.
.
int is_state; [1]
simple_lock_data_t lk_fta_kern_str; [2] 

.
.
.
};

.
.
.
void fta_error_recovery(struct fta_kern_str *sc)
{
    struct ifnet *ifp;
 
 
    /*
     * Collect the argument left by the kernel_thread_w_arg().
     */
    ifp = &sc->is_if;
    simple_lock (&sc->lk_fta_kern_str); [3]
    while (sc->is_state == RUN_NOT) { [4]
 
    for(;;) { [5]
        mpsleep ((vm_offset_t)&sc->error_recovery_flag, PCATCH,
                 "ftaerr", 0, &sc->lk_fta_kern_str,
                 MS_LOCK_SIMPLE | MS_LOCK_ON_ERROR)) [6] 
 
/* Performs tasks to reset the adapter */

.
.
.
}
    }
}

.
.
.

1. Declares a field to hold state flags. [Return to example]

2. Declares a simple lock structure pointer as a field of the fta_kern_str structure to protect the integrity of the data stored in the fields of this structure. Assume that this simple lock was initialized in the example kernel module's attach routine. The fta_error_recovery routine passes this simple lock structure pointer to the mpsleep routine. [Return to example]

3. Calls the simple_lock routine to assert an exclusive access on the following code block. [Return to example]

4. While the is_state flag is equal to the RUN_NOT flag, execute the for loop. [Return to example]

5. Sets up an infinite loop that executes when the is_state flag is equal to the RUN_NOT flag. [Return to example]

6. Calls the mpsleep routine to block (put to sleep) the current kernel thread.

   The mpsleep routine takes six arguments:

   • A channel argument

     The channel argument specifies an address associated with the calling kernel thread to be put to sleep. In this call, the address (or event) associated with the current kernel thread is stored in the error_recovery_flag field.

   • A pri argument

     The pri argument specifies whether the sleep request is interruptible. Setting this argument to the PCATCH flag causes the process to sleep in an interruptible state (that is, the kernel thread can take asynchronous signals). Not setting the PCATCH flag causes the process to sleep in an uninterruptible state (that is, the kernel thread cannot take asynchronous signals).

     In this call, fta_error_recovery passes the value PCATCH.

   • A wmesg argument

     The wmesg argument specifies the wait message.

     In this call, fta_error_recovery passes the string ftaerr.

   • A timo argument

     The timo argument specifies the maximum amount of time the kernel thread should block (sleep). If you pass the value 0 (zero), mpsleep assumes there is no timeout.

     In this call, fta_error_recovery passes the value 0 (zero) to indicate there is no timeout.

   • A lockp argument

     The lockp argument specifies a pointer to a simple or complex lock structure. You pass a simple or complex lock structure pointer if you want to release the lock. If you do not want to release a lock, pass the value 0 (zero).

     In this call, fta_error_recovery passes the address of the simple lock.

   • A flags argument

     The flags argument specifies the lock type. You can pass the bitwise inclusive OR of the valid lock bits defined in /usr/sys/include/sys/param.h.

     In this call, fta_error_recovery passes the bitwise inclusive OR of the lock bits MS_LOCK_SIMPLE (calls mpsleep with a simple lock asserted) and MS_LOCK_ON_ERROR (forces mpsleep to relock the lock on failure). You would specify these bits only if you pass a simple or complex lock.

   The mpsleep routine blocks (puts to sleep) the current kernel thread until a wakeup is issued on the address you specified in the channel argument. The kernel thread blocks a maximum of timo divided by hz seconds. The value 0 (zero) means there is no timeout.

   If you pass the PCATCH flag to the pri argument, mpsleep checks signals before and after blocking. Otherwise, mpsleep does not check signals.

   The mpsleep routine allows you to specify a pointer to a simple or complex lock structure that is associated with some resource. This routine unlocks this resource prior to blocking. The flags argument specifies the lock type. The mpsleep routine releases the lock when the current kernel thread successfully performs an assert wait on the specified channel.

   The mpsleep routine returns the value 0 (zero) if awakened (success) and EWOULDBLOCK if the timeout specified in the timo argument expires (failure). On success, mpsleep relocks the lock if you did not set MS_LOCK_NO_RELOCK in flags. On failure, it leaves the lock unlocked. If you set the flags argument to MS_LOCK_ON_ERROR, mpsleep relocks the lock on failures. [Return to example]

You can unblock (awaken) a kernel thread with the following routines:

• thread_wakeup_one

  Call this routine to unblock the first kernel thread on the specified event.

• thread_wakeup

  Call this routine to unblock all kernel threads on the specified event.

The following code fragment compares the calls to thread_wakeup_one and thread_wakeup by the if_fta module's ftaintr routine:

ftaintr(int unit)
{

.
.
.
fta_transition_state(sc, unit, PI_OPERATIONAL); [1]

.
.
.
/*******************************************************
 * Code fragment 1: Shows call to thread_wakeup_one    *
 *******************************************************/

.
.
.
thread_wakeup_one((vm_offset_t)&sc->error_recovery_flag); [2]
}

1. This code fragment shows the call to fta_transition_state. The fta_transition_state routine changes the state of the kernel module by performing certain fixed functions for any given state.

   After fta_transition_state performs its tasks, it returns to ftaintr, which calls thread_wakeup_one. This routine takes an event as the first argument. [Return to example]

2. The code fragment shows that the first argument for each of the routines specifies the event associated with the current kernel thread. It passes the address of the value stored in the error_recovery_flag field.

   The kernel module's fta_error_recovery routine is the kernel thread created and started to perform error recovery tasks. The fta_error_recovery routine blocked on the event stored in the error_recovery_flag field. [Return to example]

ftaintr(int unit)
{

.
.
.
fta_transition_state(sc, unit, PI_OPERATIONAL); [1]

.
.
.
/*******************************************************
 * Code fragment 2: Shows call to thread_wakeup        *
 *******************************************************/

.
.
.
thread_wakeup((vm_offset_t)&sc->error_recovery_flag); [2]
}

1. This code fragments shows the call to fta_transition_state. The fta_transition_state routine changes the state of the kernel module by performing certain fixed functions for any given state.

   After fta_transition_state performs its tasks, it returns to ftaintr, which calls thread_wakeup. This routine takes an event as the first argument. [Return to example]

2. The code fragment shows that the first argument for each of the routines specifies the event associated with the current kernel thread. It passes the address of the value stored in the error_recovery_flag field.

   The kernel module's fta_error_recovery routine is the kernel thread created and started to perform error recovery tasks. The fta_error_recovery routine blocked on the event stored in the error_recovery_flag field. [Return to example]

The thread_wakeup_one routine wakes up only the first kernel thread in the hash chain waiting for the event specified in the event argument. This routine is actually a convenience wrapper for the thread_wakeup_prim routine with the one_thread argument set to TRUE (wake up only the first kernel thread) and the result argument set to THREAD_AWAKENED (wakeup is normal).

The thread_wakeup routine wakes up all kernel threads waiting for the event specified in the event argument. This routine is actually a convenience wrapper for the thread_wakeup_prim routine with the one_thread argument set to FALSE (wake up all kernel threads) and the result argument set to THREAD_AWAKENED (wakeup is normal).

9.6    Terminating a Kernel Thread

To terminate a kernel thread, call the thread_terminate routine. The thread_terminate routine prepares to stop or permanently stops execution of the specified kernel thread. You created and started this kernel thread in a previous call to the kernel_isrthread or kernel_thread_w_arg routine. These routines return a pointer to the thread structure associated with the newly created and started kernel thread. Kernel modules use this pointer as a handle to identify the specific kernel thread that thread_terminate stops executing.

Typically, a kernel thread terminates itself. However, one kernel thread can terminate another kernel thread. A kernel thread that terminates itself must call thread_halt_self immediately after the call to thread_terminate. The reason for this is that thread_terminate only prepares the self-terminating kernel thread to stop execution. The thread_halt_self routine completes the work needed to stop execution by performing the appropriate cleanup work of the self-terminating kernel thread.

You do not need to terminate every kernel thread that you create. You should not terminate a kernel thread that is waiting for some event. The basic rule is that you should terminate only those kernel threads that you do not need anymore. For example, if a dynamically configured kernel module uses kernel threads, you should terminate them in the CFG_OP_UNCONFIGURE entry point of the loadable kernel module's configure routine. The kernel threads are no longer needed after the kernel module is unconfigured.

Note that the thread_terminate routine (for kernel threads that terminate other kernel threads) not only permanently stops execution of the specified kernel thread, but it also frees any resources associated with that kernel thread; thus, this kernel thread can no longer be used.

The following code fragment shows you how the if_fta kernel module's fta_error_recovery kernel thread terminates itself by calling thread_terminate and thread_halt_self.

The fta_error_recovery routine is a kernel thread that starts up when the adapter becomes operational. This kernel thread resets the adapter if a fatal error occurs. The code fragment also shows the code that contains the call to kernel_thread_w_arg, which calls fta_error_recovery.

.
.
.
#include <kern/thread.h>

.
.
.
#define ADAP "fta"

.
.
.
extern task_t first_task;

.
.
.
struct  fta_kern_str {

.
.
.
short reinit_thread_started; /* reinit thread running? */

.
.
.
short error_recovery_flag; /* flag to wake up a process */

.
.
.
};

.
.
.
struct  ifnet {

.
.
.
short if_unit; /* subunit for lower-level driver */

.
.
.
}; 

.
.
.
fta_transition_state(struct fta_kern_str *sc,
                     short unit,
                     short state)
{

.
.
.
switch(state) {

.
.
.
case PI_OPERATIONAL: {
          int s;
          NODATA_CMD *req_buff;
          thread_t err_recov_thread;
 
          if (sc->reinit_thread_started == FALSE) {
 
              err_recov_thread = kernel_thread_w_arg(first_task,
                                           fta_error_recovery,
                                           (void *)sc);
              if (err_recov_thread == NULL) {
                  printf("%s%d: Cannot start error recovery thread.\n",
                         ADAP, ifp->if_unit);
              }
              sc->reinit_thread_started = TRUE;
          }

.
.
.
/* Perform other cases */

.
.
.
void fta_error_recovery(struct fta_kern_str *sc)
{
    struct ifnet *ifp;
    int ret_val;
 
 
    /*
     * Collect the argument left by the kernel_thread_w_arg().
     */
    ifp = &sc->is_if;
 
    for(;;) {
        assert_wait_mesg((vm_offset_t)&sc->error_recovery_flag,
                          TRUE,"ftaerr");
        thread_block();
        if (current_thread()->wait_result == THREAD_SHOULD_TERMINATE) { [1]
        ret_val = thread_terminate(err_recov_thread); [2]
        thread_halt_self();   [3]
        }
   }

.
.
.
/* Performs tasks to reset the adapter */

.
.
.
}

1. If the wait_result field of the thread structure pointer associated with the current kernel thread is set to the THREAD_SHOULD_TERMINATE constant, there is no need to keep this error recovery kernel thread. The fta_error_recovery routine uses the current_thread routine to obtain the pointer to the currently running kernel thread.

   The current_thread routine is a pointer to the currently running kernel thread. Typically, kernel modules use this routine to reference the wait_result field of the thread structure pointer associated with the currently running kernel thread. A kernel module calls current_thread after calls to assert_wait_mesg and thread_block. If the kernel module needs to set a timeout, then it calls current_thread after calls to assert_wait_mesg, thread_set_timeout, and thread_block. [Return to example]

2. Calls the thread_terminate routine to terminate the error recovery kernel thread.

   The thread_terminate routine takes a thread_to_terminate argument, which is a pointer to the thread structure associated with the kernel thread that you want to terminate. This pointer was returned in a previous call to the kernel_isrthread or kernel_thread_w_arg routine.

   The kernel_thread_w_arg routine returns this pointer to the err_recov_thread variable. This variable is passed to thread_terminate.

   Upon successfully terminating the specified kernel thread, thread_terminate returns the constant KERN_SUCCESS. If the thread structure pointer passed to the thread_to_terminate argument does not identify a valid kernel thread, thread_terminate returns the constant KERN_INVALID_ARGUMENT. On any other error, thread_terminate returns the constant KERN_FAILURE. [Return to example]

3. A kernel thread that terminates itself must call thread_halt_self immediately after the call to thread_terminate. The reason for this is that thread_terminate only prepares the self-terminating kernel thread to stop execution. The thread_halt_self routine completes the work needed to stop execution of the self-terminating kernel thread by performing the appropriate cleanup work. [Return to example]

The following code fragment shows you how the if_fta module's fta_transition_state routine terminates another kernel thread (in this example, the error recovery kernel thread) by calling only thread_terminate. The fta_transition_state routine changes the state of the kernel module by performing certain fixed tasks for a given state.

.
.
.
#include <kern/thread.h>

.
.
.
#define ADAP "fta"

.
.
.
extern task_t first_task;

.
.
.
struct  fta_kern_str {

.
.
.
short reinit_thread_started; /* reinit thread running? */

.
.
.
short error_recovery_flag; /* flag to wake up a process */

.
.
.
};

.
.
.
struct  ifnet {

.
.
.
short if_unit; /* subunit for lower-level driver */

.
.
.
}; 

.
.
.
fta_transition_state(struct fta_kern_str *sc,
                     short unit,
                     short state)
{

.
.
.
int ret_val;

.
.
.
switch(state) {

.
.
.
case PI_OPERATIONAL: {
          int s;
          NODATA_CMD *req_buff;
          thread_t err_recov_thread;
 
          if (sc->reinit_thread_started == FALSE) {
 
              err_recov_thread = kernel_thread_w_arg(first_task,
                                           fta_error_recovery,
                                           (void *)sc);
              if (err_recov_thread == NULL) {
                  printf("%s%d: Cannot start error recovery thread.\n",
                         ADAP, ifp->if_unit);
              }
              sc->reinit_thread_started = TRUE;
          }

.
.
.
/* Perform other cases */

.
.
.
/* After performing all other cases, no more need for the */
/* kernel thread */
      case PI_SHUTDOWN: { [1]
 
         ret_val = thread_terminate(err_recov_thread); [2]

.
.
.
void fta_error_recovery(sc)
    struct fta_kern_str *sc;
{
    struct ifnet *ifp;
 
 
    /*
     * Collect the argument left by the kernel_thread_w_arg().
     */
    ifp = &sc->is_if;
 
    for(;;) {
        assert_wait_mesg((vm_offset_t)&sc->error_recovery_flag,
                          TRUE,"ftaerr");
        thread_block();
 
/* Performs tasks to reset the adapter */

.
.
.
}

.
.
.
}

1. After the fta_error_recovery routine completes its work and returns to fta_transition_state, there is no need to keep this error recovery kernel thread. The fta_transition_state routine sets up a case statement to handle the termination of the error recovery kernel thread. [Return to example]

2. Calls the thread_terminate routine to terminate the error recovery kernel thread.

   The thread_terminate routine takes a thread_to_terminate argument, which is a pointer to the thread structure associated with the kernel thread that you want to terminate. This pointer was returned in a previous call to the kernel_isrthread or kernel_thread_w_arg routine.

   The kernel_thread_w_arg routine returns this pointer to the err_recov_thread variable. This variable is passed to thread_terminate.

   Upon successfully terminating the specified kernel thread, thread_terminate returns the constant KERN_SUCCESS. If the thread structure pointer passed to the thread_to_terminate argument does not identify a valid kernel thread, thread_terminate returns the constant KERN_INVALID_ARGUMENT. On any other error, thread_terminate returns the constant KERN_FAILURE. [Return to example]

To set a time delay on the current kernel thread, call the thread_set_timeout routine.

You must call the thread_set_timeout routine as follows:

1. Lock the resource.

2. Call assert_wait_mesg to assert that the current kernel thread is about to block.

3. Unlock the resource.

4. Call thread_set_timeout to set the time of delay for the current kernel thread.

5. Call thread_block to block (put to sleep) the current kernel thread.

The following code fragment shows a call to thread_set_timeout by the if_fta module's fta_cmd_req routine. This routine puts a DMA request onto the request queue of the adapter.

.
.
.
#include <kern/thread.h> [1]

.
.
.
struct  fta_kern_str {

.
.
.
struct cmd_buf *q_first; /* first in the request queue */
struct cmd_buf *q_last;  /* last in the request queue */
lock_data_t cmd_buf_q_lock; /* lock for the cmdreq queue */

.
.
.
}; [2]

.
.
.
short fta_cmd_req(cmdbuf, sc, command)
struct cmd_buf *cmdbuf;
struct fta_kern_str *sc;
short command;
{

.
.
.
lock_write(&sc->cmd_buf_q_lock); [3]

.
.
.
assert_wait_mesg((vm_offset_t)cmdbuf, TRUE, "dmareq"); [4]
    lock_done(&sc->cmd_buf_q_lock); [5]
    thread_set_timeout(hz * 2); [6]
    thread_block(); [7]

.
.
.
}

1. Includes the header file /usr/sys/include/kern/thread.h. The thread.h file defines structures that kernel thread routines use. [Return to example]

2. Defines an fta_kern_str data structure.

   In this example, the fta_kern_str structure contains the following fields:

   • q_first

     Specifies a pointer to a cmd_buf data structure. This field represents the first command queue in the linked list.

   • q_last

     Specifies a pointer to a cmd_buf data structure. This field represents the last command queue in the linked list.

   • cmd_buf_q_lock

     Declares a lock structure called cmd_buf_q_lock. The purpose of this lock is to protect the integrity of the data stored in the linked list of cmd_buf data structures. Note that the alternate name lock_data_t is used to declare the complex lock structure. Embedding the complex lock in the fta_kern_str structure protects the cmd_buf structure for any number of instances.

   [Return to example]

3. Calls the lock_write routine to lock the command request queue.

   The lock_write routine takes one argument: a pointer to the complex lock structure lock. This is the lock structure associated with the resource on which you want to assert a complex lock with write access. The fta_cmd_req routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer. [Return to example]

4. Calls the assert_wait_mesg routine to assert that the current kernel thread is about to block.

   The assert_wait_mesg routine takes three arguments:

   • The first argument specifies the event associated with the current kernel thread. In this call, the event associated with the current kernel thread is the cmdbuf structure pointer.

   • The second argument specifies a Boolean value that indicates how the kernel thread is awakened. You can pass one of the following values:

     TRUE

     The current kernel thread is interruptible. This value means that a signal can awaken the current kernel thread.

     FALSE

     The current kernel thread is not interruptible. This value means that only the specified event can awaken the current kernel thread.

     The code fragment shows that fta_cmd_req passes the value TRUE.

   • The third argument specifies a mnemonic for the type of wait. The /bin/ps command uses this mnemonic to print out more meaningful messages about a process. The code fragment shows that fta_cmd_req passes the string dmareq.

   [Return to example]

5. Calls the lock_done routine to unlock the command request queue.

   The lock_done routine takes one argument: a pointer to the complex lock structure lock. The fta_cmd_req routine passes the address of the cmd_buf_q_lock field of the fta_kern_str structure pointer. [Return to example]

6. Calls the thread_set_timeout routine to set a timer for the current kernel thread.

   The thread_set_timeout routine takes one argument: the amount of time to wait for an event. The time is used in conjunction with the assert_wait routine. The fta_cmd_req routine passes the value hz * 2.

   The time you specify to wait for the event is automatically canceled when the kernel thread awakes.

   The thread_set_timeout routine does not return a value. [Return to example]

7. Calls the thread_block routine to block (put to sleep) the current kernel thread. [Return to example]