This chapter explains how to use the Ladebug debugger to debug multithreaded programs that use DECthreads or kernel threads.
The debugger supports two levels of threads:
The debugger variable $threadlevel tells the
debugger to view (interpret) the application threads as
DECthreads threads or native threads. Depending on the setting of
$threadlevel, only the DECthreads or the native threads
are recognized by the debugger.
(For complete information on using DECthreads, see the DECthreads Reference Manual.)
By default, the $threadlevel variable is set to
decthreads if the debuggee application is multithreaded
and is using DECthreads. Otherwise, the $threadlevel
is set to native. This happens each time an application
is loaded into the debugger via the command line, the load
command, or the attach command.
For core file debugging, the threadlevel is always set
to native.
You can switch the threads debugging mode from native
to decthreads (or the reverse) by setting
$threadlevel appropriately. Use the debugger command
set $threadlevel, as follows:
set $threadlevel="decthreads" set $threadlevel="native"
The valid values of thread identifiers for native threads and DECthreads are as follows:
The thread identifier is interpreted according to the
$threadlevel.
The debugger variable $curthread contains the thread
identifier of the current thread. The $curthread value
is updated when program execution stops or completes.
The current thread context can be modified by setting
$curthread to a valid thread identifier. This is
equivalent to issuing the thread thread_identifier
command.
When there is no process or program, $curthread is set
to 0.
The debugger variable $tid is the same as
$curthread . (See Chapter 22.)
The categories of thread commands are as follows:
These commands are discussed in the following sections.
You can use the thread command to identify or set the
current thread context. The syntax is as follows:
thread [thread_identifier]
If you supply a thread identifier, the debugger sets the current context to the thread you specify. If you omit the thread identifier, the debugger displays the current thread context. See Section 19.2 for a list of valid thread identifier values.
The debugger interprets the thread identifier as a DECthreads or
kernel thread identifier depending on the value of the debugger
variable $threadlevel.
The thread context can also be modified by setting the debugger
variables $curthread or $tid. (See Section 19.2.)
These commands control the execution of one or more threads in a process. You can stop and resume execution of some or all of the threads in the application.
Use one of the following stop commands to set
breakpoints in specific threads:
l)stop [variable] [thread thread_identifier_list] [at line_number] [if expression] stop [variable] [thread thread_identifier_list] [in function] [if expression]
The thread_identifier_list parameter identifies one or more threads of the current debugging level.
A thread identifier is treated as one of the conditions on the
breakpoint. When the $threadlevel mode is changed,
this condition may no longer be true.
If you list one or more thread identifiers, the debugger sets a breakpoint only in those threads that you specify. If you omit the thread identifier specification, the breakpoint is set at the process level.
Use one of the following trace or when
command syntaxes to set tracepoints in specific threads:
trace [variable] [thread thread_identifier_list]
[at line_number] [if expression]
trace [variable] [thread thread_identifier_list]
[in function] [if expression]
when [variable] [thread thread_identifier_list]
[at line_number] [if expression] { command [;<]
{command [; . . . ]}
when [variable] [thread thread_identifier_list]
[in function] [if expression] {com mand [;<ion]
{command [; . . . ]}
If you list one or more thread identifiers, the debugger sets a tracepoint only in those threads that you specify. If you omit the thread identifier specification, the debugger sets a tracepoint at the process level.
Use one of the following commands to step while putting all other threads on hold:
step stepi next nexti
The debugger steps only the current thread.
You can use the cont command to resume execution
of the current thread that was put on hold (for example, at a
breakpoint). As the current thread resumes, all other threads
continue by default. The syntax is as follows:
cont [signal]
If you specify a signal, the program continues execution with that signal. The signal value can be either a signal number or a string name (for example, SIGSEGV). The default is 0, which allows the program to continue execution without specifying a signal.
Thread information commands let you view information available
from the debugger about the threads in your application.
The information displayed may vary depending on whether the
$threadlevel variable is set to decthreads
or native.
To obtain the maximum detail, set the $verbose
debugger variable to a value of 1. For more information
about $verbose, see Section 10.13.
You can use the show thread command to list all the
threads known to the debugger. The syntax is as follows:
show thread [thread_identifier_list]
If you specify one or more thread identifiers, the debugger displays information about those threads. If you do not specify any thread identifiers, the debugger displays information for all known threads. For example:
(ladebug) print $threadlevel
``decthreads''
(ladebug) show thread
Id State Substate Policy Prior Name
--- ------- ---------- --------- ---- -------------
1 running throughput 19 default thread
3 running throughput 19 <pthread user@0x1400005d8
* 4 stopped throughput 19 <pthread user@0x1400005e8
5 running throughput 19 <pthread user@0x1400005f8
6 running throughput 19 <pthread user@0x140000608
2 terminated exited throughput 19 <pthread user@0x1400005c8
(ladebug) set $threadlevel="native"
(ladebug) print $threadlevel
``native''
(ladebug) show thread
Id State
----------- ------
> 0xffffffff81ad6f00 running
0xffffffff81ad72c0 running
* 0xffffffff81b3f2c0 stopped at 0x12001410 main(....) : 24
0xffffffff81ad7a40 running
0xffffffff81b16f00 running
0xffffffff81b50000 running
Use the show thread with state command to list threads
in a specific state, such as threads that are currently blocked.
The possible states depend on whether you have DECthreads or native
threads.
The following state values apply to DECthreads (see the DECthreads documentation for the meaning of the states):
ready
blocked
running
terminated
detached
The following state values apply to native threads:
stopped
running
terminated
The syntax is as follows:
show thread [thread_identifier_list] with state == ready show thread [thread_identifier_list] with state == blocked show thread [thread_identifier_list] with state == running show thread [thread_identifier_list] with state == terminated show thread [thread_identifier_list] with state == detached show thread [thread_identifier_list] with state == stopped
You can use the where command to display the stack
trace of current threads. You can specify one or more threads or all
threads. The syntax is as follows:
where [number] [thread [thread_identifier_list | all |*]]
The where command displays the stack trace of
currently active functions, for the current thread.
The where thread thread_identifier_list command
displays the stack trace(s) of the specified thread(s).
The where thread all and the where thread *
commands are equivalent; they display the stack traces of
all threads.
Include the optional number argument to see a specified
number of levels from the top of the stack. (Each active function
is designated by a number that can be used as an argument to the
func command. The top level on the stack is 0; so if
you enter the command where 3 , you will see levels 0,
1, and 2.) If you do not specify the number argument,
you will see all levels.
The print command evaluates an optional expression
in the context of the current thread and displays the result. The
syntax is as follows:
print expression [ . . . ]
The call command evalutes an expression in the context
of the current thread and makes the call in the context of the
current thread. The syntax is as follows:
call expression
The printregs command prints the registers for the
current thread. The syntax is as follows:
printregs
If your $threadlevel is decthreads, you
can use the show condition command to list information
about currently available condition variables. The syntax is as
follows:
show condition [condition_identifier_list] [with state == wait]
If you supply one or more condition variable identifiers, the debugger displays information about those condition variables that you specify, provided that the list matches the identity of currently available condition variables. If you omit the condition variable identifier specification, the debugger displays information about all condition variables currently available.
Example 19-1 shows the output
from a simple show condition command.
(ladebug) show condition Condition variable thread 1 join 1 (0x14001dcf0) Condition variable thread 1 wait 2 (0x14001de40) Condition variable Last thread CV 3 (0x14001ffb8) Condition variable thread 2 join 6 (0x140020af0) Condition variable thread 2 wait 7 (0x140020c40) Condition variable thread 3 join 9 (0x140021108) Condition variable thread 3 wait 10 (0x140021258) (ladebug)
If you specify with state == wait, the debugger
displays information exclusively for the condition identifiers that
have one or more threads waiting on them.
If $verbose is set to 1, Ladebug also displays the
sequence number of the threads waiting on the condition variables.
If the debuggee application has no DECthreads or the
$theadlevel is set to native, an
appropriate message is issued.
If your $threadlevel is decthreads, you
can use the show mutex command to list information
about currently available mutexes. The syntax is as follows:
show mutex [mutex_identifier_list] [with state == locked]
If you supply one or more mutex identifiers, the debugger displays information about only those mutexes that you specify, provided that the list matches the identity of currently available mutexes. If you omit the mutex identifier specification, the debugger displays information about all mutexes currently available.
You can specify with state == locked to display
information exclusively for locked mutexes.
If $verbose is set to 1, Ladebug also displays the
sequence number of the threads locking the mutex.
Example 19-2 shows the output from a
simple show mutex command.
(ladebug) show mutex Mutex thread 1 lock 14 (0x14001dc48), type fast, unlocked Mutex thread 1 wait 15 (0x14001dd98), type fast, unlocked Mutex thread 2 lock 69 (0x140020a48), type fast, unlocked Mutex thread 2 wait 70 (0x140020b98), type fast, unlocked (ladebug)
If the debuggee application has no DECthreads or the
$theadlevel is set to native, an
appropriate message is issued.
The following example shows the use of Ladebug commands to debug a
multithreaded program. For more information on this test program,
prime_numbers, see the DECthreads Reference
Manual.
Welcome to the Ladebug Debugger Version 3.0
------------------
object file name: thread_prime_numbers
Reading symbolic information ...done
(ladebug) record io out1
(ladebug) stop at 432
[#1: stop at "thread_prime_numbers.c":43 ]
(ladebug) run3
[1] stopped at [prime_search:43 0x120001bb4]
43 while (not_done)
(ladebug)thread4
Thread 2 (running) "<pthread user@0x140000758>"
Scheduling: throughput policy at priority 19
Stack: 0x3ef80; base is 0x40000, guard area at 0x39fff
General cancelability enabled, asynch cancelability disabled
(ladebug)print my_number5
0
(ladebug)show thread with state == running6
Thread 2 (running) "<pthread user@0x140000758>"
Scheduling: throughput policy at priority 19
Stack: 0x3ef80; base is 0x40000, guard area at 0x39fff
General cancelability enabled, asynch cancelability disabled
Thread 3 (running) "<pthread user@0x140000768>"
Scheduling: throughput policy at priority 19
Stack: 0x2e000; base is 0x2e000, guard area at 0x27fff
General cancelability enabled, asynch cancelability disabled
Thread 4 (running) "<pthread user@0x140000778>"
Scheduling: throughput policy at priority 19
Stack: 0x1e000; base is 0x1e000, guard area at 0x17fff
General cancelability enabled, asynch cancelability disabled
Thread 5 (running) "<pthread user@0x140000788>"
Scheduling: throughput policy at priority 19
Stack: 0x8a000; base is 0x8a000, guard area at 0x83fff
General cancelability enabled, asynch cancelability disabled
Thread 6 (running) "<pthread user@0x140000798>"
Scheduling: throughput policy at priority 19
Stack: 0x78000; base is 0x78000, guard area at 0x71fff
General cancelability enabled, asynch cancelability disabled
(ladebug)show thread with state == blocked7
Thread 1 (blocked) "default thread"
Waiting on condition variable 4 using mutex 81
Scheduling: throughput policy at priority 19
Stack: 0x0 (default stack)
!! Thread is not on stack !!
General cancelability enabled, asynch cancelability disabled
(ladebug) where8
>0 0x120001bb4 in prime_search(arg=0x0) thread_prime_numbers.c:43
#1 0x3ff80c5cdf0 in cma__thread_base() ../../../../../src/usr/ccs/lib
/DECthreads/COMMON/cma_thread.c:1547
(ladebug) thread 49
Thread 4 (running) "<pthread user@0x140000778>"
Scheduling: throughput policy at priority 19
Stack: 0x1e000; base is 0x1e000, guard area at 0x17fff
General cancelability enabled, asynch cancelability disabled
(ladebug)where10
>0 0x3ff80c5cd10 in cma__thread_base() ../../../../../src/usr/ccs/lib
/DECthreads/COMMON/cma_thread.c:1498
(ladebug) thread 111
Thread 1 (blocked) "default thread"
Waiting on condition variable 4 using mutex 81
Scheduling: throughput policy at priority 19
Stack: 0x0 (default stack)
!! Thread is not on stack !!
General cancelability enabled, asynch cancelability disabled
(ladebug) where12
>0 0x3ff808edf14 in msg_receive_trap() /usr/build/osf1/goldos.bld/export
/alpha/usr/include/mach/syscall_sw.h:74
#1 0x3ff808e4764 in msg_receive() ../../../../../src/usr/ccs/lib/libmach
/msg.c:95
#2 0x3ff80c63d60 in cma__vp_sleep() ../../../../../src/usr/ccs/lib/DECthreads
/COMMON/cma_vp.c:1569
#3 0x3ff80c4c5ec in cma__dispatch() ../../../../../src/usr/ccs/lib/DECthreads
/COMMON/cma_dispatch.c:994
#4 0x3ff80c43e00 in cma__int_wait() ../../../../../src/usr/ccs/lib/DECthreads
/COMMON/cma_condition.c:2531
#5 0x3ff80c5c41c in cma_thread_join() ../../../../../src/usr/ccs/lib
/DECthreads/COMMON/cma_thread.c:926
#6 0x3ff80c5485c in pthread_join() ../../../../../src/usr/ccs/lib/DECthreads
/COMMON/cma_pthread.c:2294
#7 0x120002104 in main() thread_prime_numbers.c:121
(ladebug) thread 213
Thread 2 (running) "<pthread user@0x140000758>"
Scheduling: throughput policy at priority 19
Stack: 0x3ef80; base is 0x40000, guard area at 0x39fff
General cancelability enabled, asynch cancelability disabled
(ladebug) step14
stopped at [prime_search:45 0x120001bbc]
45 pthread_testcancel();
(ladebug) cont15
[1] stopped at [prime_search:43 0x120001bb4]
43 while (not_done)
(ladebug) thread16
Thread 3 (running) "<pthread user@0x140000768>"
Scheduling: throughput policy at priority 19
Stack: 0x2cf80; base is 0x2e000, guard area at 0x27fff
General cancelability enabled, asynch cancelability disabled
(ladebug) where17
>0 0x120001bb4 in prime_search(arg=0x1) thread_prime_numbers.c:43
#1 0x3ff80c5cdf0 in cma__thread_base() ../../../../../src/usr/ccs/lib
/DECthreads/COMMON/cma_thread.c:1547
(ladebug) show mutex (1, 2, 3)18
Mutex 1 (fast) "default attrs mutex" is not locked
Mutex 2 (fast) "known attr list" is not locked
Mutex 3 (fast) "known mutex list" is not locked
(ladebug) show condition (5, 12, 15)19
Condition variable 5, "thread 2 wait" has no waiters
Condition variable 12, "thread 6 join" has no waiters
(ladebug) delete 120
(ladebug) cont21
Thread 0 terminated
Thread 1 terminated normally
Thread 2 terminated
Thread 3 terminated
Thread 4 terminated
The list of 110 primes follows:
1, 3, 5, 7, 11, 13, 17, 19, 23, 29,
31, 37, 41, 43, 47, 53, 59, 61, 67,
71, 73, 79, 83, 89, 97, 101, 103, 107,
109, 113, 127, 131, 137, 139, 149, 151, 157,
163, 167, 173, 179, 181, 191, 193, 197, 199,
211, 223, 227, 229, 233, 239, 241, 251, 257,
263, 269, 271, 277, 281, 283, 293, 307, 311,
313, 317, 331, 337, 347, 349, 353, 359, 367,
373, 379, 383, 389, 397, 401, 409, 419, 421,
431, 433, 439, 443, 449, 457, 461, 463, 467,
479, 487, 491, 499, 503, 509, 521, 523, 541,
547, 557, 563, 569, 571, 577, 587, 593, 599,
601
Thread has finished executing
(ladebug) quit
record io command saves both debugger
input and output to a file named out in this
example.
stop at command suspends program
execution at the specified line (43 in the example) for any
thread.
run command starts program execution.
thread command identifies or sets the
current thread context. Because the thread identification is
omitted, the debugger displays the current thread context.
print command displays the value of the
object my_number for the thread that stopped.
show thread with state == running
command displays threads that are running.
where command displays the stack trace
of currently active functions for the current thread (2).
thread command sets the current thread
context to thread 4.
thread command sets the current thread
context to 1.
thread command sets the current thread
context to 2.
step command executes one line of source
code for thread 2 only.
cont command resumes program execution
until a breakpoint, signal, error, or end of the program is
encountered.
thread command identifies or sets the
current thread context. Because the thread identification is
omitted, the debugger determines that the current thread is 3.
Note that it reached line 43 for thread 3.
where command displays the stack trace
of currently active functions in thread 3.
show mutex (1, 2, 3) command shows that
none of the specified mutexes is locked.
show condition (5, 12, 15) command shows
that variables 5 and 12 have no threads waiting. Condition 15
does not exist; therefore, no information is shown.
delete 1 command removes the specified
breakpoint or trace.
cont command resumes program execution.
When the program finishes execution, the threads are terminated
and the result is displayed.