Previous Next Contents Generated Index Home


Chapter 6

Advanced Data Model Realization Techniques




This chapter includes the following sections:


What are Filters

The filter file defines data filters implemented using Tcl/TOE procedures. These filters are used to extract the pertinent information from the raw results of data acquisition commands. This enables the agent to use raw extensions or system commands (such as df for Solaris software) to acquire data, with the processing/parsing of the output being performed within the agent, not through external utilities such as awk or sed.


Note - For more information on Tcl/TOE, refer to the Appendix A.

Standard Extensions for File Name

<module><-subspec>-d.flt

This file is optional for a module, and only exists if the module is using filter functions. Only Tcl/TOE procedures can be defined in this file.


Examples of Filters


CPU Data Filter

The UNIX command vmstat 10 2 returns four lines of data in the format:

procs     memory            page            disk          faults      cpu
r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 s2   in   sy   cs us sy id
0 0 0  24120  7040   0  63  6  2  4  0  1  0  4  2  1  215  347  167  5 11 83
0 0 0 280020  1172   0   1  0  8 18  0  2  0  1  0  0   39   52   46  1  1 98 

This data is passed as the datalist argument in the cpuFilter procedure. The procedure parses the percent idle, system and user fields from the fourth line of data, computes the percent busy, and returns the results. The code is written in Tcl.

proc cpuFilter { datalist } {
    set line [ lindex $datalist 3 ]
    set user [ string range $line 69 71 ]
    set system [ string range $line 72 74 ]
    set idle [ string range $line 75 77 ]
    set busy [ expr 100 - $idle ]
    return "$idle $busy $system $user"
}


User Data Filter

The UNIX command who returns data in the following format:

tom        console      Oct  2 09:54
tom        pts/1        Oct  4 23:43    (superior)
tom        pts/0        Oct  2 09:55
tom        pts/2        Oct  2 09:55
tom        pts/5        Oct  2 09:55
tom        pts/4        Oct  3 09:29
tom        pts/3        Oct  2 09:55

This data is passed to the userFilter procedure as the datalist argument. The procedure loops through each line to determine the console user and count the number of unique users and sessions.

proc userFilter { datalist } {
    set console none
    set ucount 0
    set scount 0
    foreach line $datalist {
        set name [ lindex $line 0 ]
        set source [ lrange $line 5 end ]
        if { [ lindex $line 1 ] == "console" } { set console $name }
        if { [ catch { set users($name) } ] } {
            set users($name) ""
            incr ucount
        }
        if { [ catch { set sessions($name:$source) } ] } {
            set session($name:$source) ""
            incr scount
        }        
    }
    return [ list $console $ucount $scount ]
}


Load Data Filter

The UNIX command uptime returns data in the following format:

 11:45pm  up 4 day(s), 16:29,  2 users,  load average: 0.00, 0.00, 0.01 

This data is passed to the loadFilter procedure where the 1-, 5-, and 15-minute load averages are picked out using a regular expression pattern.

proc loadFilter { datalist } {
    regexp {^.*up *(.*), +[0-9]+ +users*,.+: *([^,]+), *([^,]+), *([^,]+)} 
    [ lindex $datalist 0 ] dummy a b c d
    return [ list $b $c $d $a ]
} 


File System Data Filter

The UNIX command df -kF ufs returns data in the following format:

Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t0d0s0      22847   11912   8655    58%    /
/dev/dsk/c0t0d0s6     246167  185193  36364    84%    /usr
/dev/dsk/c0t0d0s3     105943    3183  92170     4%    /var
/dev/dsk/c0t0d0s7     793382       9 714043     1%    /export1
/dev/dsk/c0t0d0s5     288855   43131 216844    17%    /opt 

This data is passed to the fileFilter procedure where the desired data fields are picked out from each line.

proc fileFilter { datalist } {
    #
    # pick out appropriate fields for each line
    #
    set result ""
    set continuation ""
    foreach line [ lrange $datalist 1 end ] {
        set line $continuation$line
        if { [ llength $line ] < 6 } {
            set continuation "$line "
            continue
        } else {
            set continuation ""
        }
        regsub {%} $line "" line
        lextract $line 1 kbytes 3 avail 4 capacity 5 mount                    
        lappend result $mount $kbytes $avail $capacity
    }
    return $result
}                                                                              


Adding Filters to Data Model Realization


Example Data Model File

The following code example lists the Solaris Example Model file,
solaris-example-models-d.x. It has three independent manged objects; CPU, system, and file system.

CODE  EXAMPLE  6-1 The solaris-example-models-d.x File 
#
# Solaris Managed Object and Property Models
#
type = reference
 
#
# Cpu Managed Object
#
cpu = { [ use MANAGED-OBJECT ]

    mediumDesc             = CPU Properties
    consoleHint:mediumDesc = base.modules.solaris-example:cpu
 
    idle = { [ use PERCENT MANAGED-PROPERTY ]
        shortDesc       = Idle
        mediumDesc      = CPU Idle Time
        fullDesc        = Percentage of time the CPU is in the 
idle state units = % } busy = { [ use PERCENT MANAGED-PROPERTY ] shortDesc = Busy mediumDesc = CPU Busy Time fullDesc = Percentage of time the CPU is in the
busy state units = % } system = { [ use PERCENT MANAGED-PROPERTY ] shortDesc = System mediumDesc = CPU System Time fullDesc = Percentage of time the CPU is running
in system mode units = % } user = { [ use PERCENT MANAGED-PROPERTY ] shortDesc = User mediumDesc = CPU User Time fullDesc = Percentage of time the CPU is running
in user mode units = % } } # # System Managed Object # system = { [ use MANAGED-OBJECT ] mediumDesc = System Information consoleHint:mediumDesc = base.modules.solaris-example:system userstats = { [ use MANAGED-PROPERTY-CLASS ] mediumDesc = User Statistics consoleHint:mediumDesc = \
base.modules.solaris-example:system.userstats consoleUser = { [ use STRING MANAGED-PROPERTY ] shortDesc = User mediumDesc = Console User fullDesc = User currently logged in on the console consoleHint:mediumDesc = \
base.modules.solaris-example:system.userstats.consoleUser } numUsers = { [ use INT MANAGED-PROPERTY ] shortDesc = #Users mediumDesc = Number of Users fullDesc = Number of unique users currently
logged in consoleHint:mediumDesc = \
base.modules.solaris-example:system.userstats.numUsers } numSessions = { [ use INT MANAGED-PROPERTY ] shortDesc = Sessions mediumDesc = Number of User Sessions fullDesc = Number of currently active user
sessions } primaryUser = { [ use STRING MANAGED-PROPERTY ] shortDesc = User mediumDesc = Primary System User fullDesc = The login name of the primary user } } load = { [ use MANAGED-PROPERTY-CLASS ] mediumDesc = Load Average consoleHint:mediumDesc = \
base.modules.solaris-example:system.load one = { [ use FLOAT MANAGED-PROPERTY ] shortDesc = 1min mediumDesc = 1 Min Load Avg fullDesc = The one minute load average consoleHint:mediumDesc = \
base.modules.solaris-example:system.load.one } five = { [ use FLOAT MANAGED-PROPERTY ] shortDesc = 5min mediumDesc = 5 Min Load Avg fullDesc = The five minute load average consoleHint:mediumDesc = \
base.modules.solaris-example:system.load.five } } } # # Filesystem Table # filesystems = { [ use MANAGED-OBJECT ] mediumDesc = Filesystems fileTable = { [ use MANAGED-OBJECT-TABLE ] mediumDesc = Filesystem Property Table fileEntry = { [ use MANAGED-OBJECT-TABLE-ENTRY ] mediumDesc = Filesystem index = mount mount = { [ use STRING MANAGED-PROPERTY ] shortDesc = Mount Pt mediumDesc = Filesys Mount Point fullDesc = The mount point for the filesystem } size = { [ use INT MANAGED-PROPERTY ] shortDesc = Filesys Sz mediumDesc = Filesystem Size fullDesc = Total filesystem size in KBytes units = KB } avail = { [ use INT MANAGED-PROPERTY ] shortDesc = FilesysAvl mediumDesc = Filesystem Space fullDesc = Available filesys diskspace in KB units = KB } pctUsed = { [ use PERCENT MANAGED-PROPERTY ] shortDesc = Disk Used mediumDesc = Filesystem Capacity fullDesc = Percentage of Disk Space Used units = % } } } }

The solaris-example.properties file is shown below. This file also contains the module parameter internationalization key and strings:

CODE  EXAMPLE  6-2 The solaris-example.properties File 
#
# Module Parameters
#
moduleName=Solaris Example
moduleType=operatingSystem
moduleDesc=This is an example module monitoring cpu, load, 
and filesystem statistics. # # Node Descriptions # cpu=CPU Properties system=System Information system.userstats=User Statistics system.userstats.consoleUser=Console User system.userstats.numUsers=Number of Users system.load=Load Average system.load.one=1 Min Load Avg system.load.five=5 Min Load Avg


Example Data Model Realization File Using Tcl Filters

The following code example lists the Solaris example model realization file using Tcl filters developed in the section, "Examples of Filters".

CODE  EXAMPLE  6-3 Solaris Example Model Realization File  
[ use MANAGED-MODULE ]
[ requires template solaris-example-models-d ]

#
# Load Module Parameters
#
[ load solaris-example-m.x ]

#
# Define services required by this module
#
_services = { [ use SERVICE ]
    #
    # Standard Bourne Shell
    #
    sh = {
        command = "pipe://localhost//bin/sh;transport=shell"
        max     = 2
    }
}

#
# Load filters required by this module
#
_filters = { [ use PROC ]
    [ source solaris-example-d.flt ]
}

#
# Cpu Information uses the cpuFilter which is already discussed
# in the previous section
#
cpu = { [ use templates.solaris-example-models-d.cpu _filters ]
    type                = active
    refreshService      = _services.sh
    refreshCommand      = vmstat 10 2
    refreshFilter       = cpuFilter
    refreshInterval     = 60
}

#
# System User and Load Information uses the userFilter 
# which is already discussed
# in the previous section
#
system = { [ use templates.solaris-example-models-d.system ]
    userstats = { [ use _filters ]
        type                = active
        refreshService      = _services.sh
        refreshCommand      = who
        refreshFilter       = userFilter
        refreshInterval     = 120
  
        primaryUser = {
            type            = active
            refreshService  = _services.sh
            refreshCommand  = solaris-example-primary-user-d.sh
            refreshInterval = 86400
        }
    }

    load = { 
        one = {
            type            = active
            refreshService  = _services.sh
            refreshCommand  = echo 10.2
            refreshInterval     = 60
        }
        five = {
            type            = active
            refreshService  = _services.sh
            refreshCommand  = echo 10.2
            refreshInterval     = 60
        }
    }

}

#
# Filesystem Information uses the fileFilter 
# which is already discussed
# in the previous section
#
filesystems = { [use templates.solaris-example-models-
d.filesystems 
                  _filters]
    type                = active
    refreshService      = _services.sh
    refreshCommand      = df -kF ufs
    refreshFilter       = fileFilter
    refreshInterval     = 120
}

The Solaris Example Primary user-d.sh file is shown below:

CODE  EXAMPLE  6-4 The solaris-example-primary-user-d.sh File 
#!/bin/sh

echo "I am a primary user (from Sh)"


Note - All the filters used, for example, CPUfilter, and so on, in the example above, are defined in the file solaris-example-d.flt.

Loading the DAQ Services


Tcl Filters

If the DAQ was implemented using Tcl filters, the Filters file must be loaded into a container object. Nodes that want to call a procedure defined in the Filters file must inherit this object.

The _services.<shell> object can then be used by other objects for Tcl shell DAQ services.


RefreshQualifier for Filters


refreshFilter

Refresh filters can be specified in active and derived nodes:

refreshFilter = <Tcl command or procedure>

The refreshFilter qualifier specifies a Tcl command or procedure that is used to process the data acquired by the refresh command. The Tcl procedure must take a single argument that is the result returned by the refresh command. The result of the refresh filter is cascaded into the managed properties. Recall that if the refresh command is implemented as a UNIX command or shell script, any data returned on stderr constitutes a data acquisition error. As a result, no data is passed to the refresh filter, regardless if data was returned on stdout too.


Solaris Example--Loading the Filter File

In the following example, the Solaris Example Filter file is loaded into the filters object. This object can be inherited by other objects that want to use the procedures defined in the Filter file. One such object is the CPU managed object.

The following is a code example of loading the filter file:

CODE  EXAMPLE  6-5 Loading the Filter File
_filters = { [ use PROC ]
    [ source solaris-example-d.flt ]
}
cpu = { [ use templates.solaris-example-models-d.cpu _filters ] 
... }


Advanced Data Acquisition Mechanisms

These are other techniques and are discussed in detail later on in this chapter.


Tcl/TOE Code

Standard Tcl/TOE commands, such as file to get file statistics, can be used to acquire data. In addition, Tcl procedures can be written to filter raw results returned by UNIX programs. Tcl provides many useful commands for parsing strings (regexp, regsub, and so forth.).

In general, Tcl procedures are preferred over using shell scripts when filtering data as they are typically easier to implement and more efficient. When returning data to the agent from Tcl procedures or commands, the data elements must be elements in a Tcl list.


C Code Libraries and Tcl/TOE Command Extensions

C-code libraries and Tcl/TOE command extensions can also be written to perform DAQ. This is accomplished by packaging the DAQ functions as shared object libraries that can be dynamically loaded by the Sun Management Center agent.

When returning more than one item of data from Tcl commands, the data elements must be elements in a Tcl list.

The next few sections introduce concepts necessary to understand advanced data model realization techniques.


Other Node Types based on their Operational Behavior


Passive Nodes

Nodes that do not actively collect data but instead have data cascaded into them are known as passive nodes. By default, all nodes are passive, unless otherwise specified using the type qualifier.

A node can be explicitly specified to be passive using the following specification:

type = passive

Passive managed property nodes can specify an update filter to process that data being cascade into it.

updateFilter = <Tcl procedure or command>

If no update filter or other qualifier is required, it is unnecessary to explicitly declare a passive node in the agent file at all. For such nodes, it is sufficient to model them in the model file only. For more details on updateFilter, refer to the section, "updateFilter".


Derived Nodes

A node is specified to be a derived node using the following specification:

type = derived

Derived nodes establish dependency relationships with the nodes on which they rely through the use of the refresh triggers specification. Nodes can be triggered by the change in value or status of another node, and refresh automatically when such an event occurs. Derived nodes can also refresh at an interval, although this is usually unnecessary if the triggers are specified properly.

A derived node can use other MIB nodes as the service(s) for its refresh. In other words, its value is often a function of the values or qualifiers of one or more other managed properties. Through the use of derived variables, it is possible to create nodes whose value represents averages, rates of change, specific digital filters (for example: high pass, low pass, or band pass or other useful calculated information).

All derived nodes must specify the following refresh parameters:

refreshService = <service object>
refreshCommand = <command to run in the context of refreshService>
refreshTrigger = <node name>[:<event>] [<node name>[:<event>] ...]

The refreshCommand could also be Tcl commands and procedures.


Note - The refresh interval is optional for a derived node. Derived nodes may be forced to update at periodic intervals, although this is usually unnecessary if the refresh triggers are speficied properly. For more information on refreshTrigger, refer to the next section.


refreshQualifiers & Other Qualifiers

The following refresh qualifiers can also be specified in active and derived nodes.


timeoutInterval

The timeout interval can be specified for active and derived nodes.

timeoutInterval = <timex specification>

If the refresh command does not complete within the specified timeoutInterval, the command is aborted. In that case, the alarm state of the node is marked indeterminate (unknown value). The default is no timeout. This can be used in conjunction with refreshMode = sync (described later) to ensure that the agent does not hang while collecting data.


refreshTrigger

Refresh triggers must be specified in derived nodes.

refreshTrigger = <node name>[:<event>] [<node name>[:<event>] ...]

Derived nodes establish dependency relationships with one or more nodes on which they rely through the use of the refresh triggers specification. Nodes can be triggered off the change in value or status of another node, and refreshes automatically when such an event occurs.

The refreshTrigger specifies the name of the node that the derived node depends on. The possible events are:

status  

event generated upon status change of the node  

refresh  

event generated when an active node is to refresh its value  

update  

event generated in managed properties when the data values are updated  

set  

event generated in the node when a SNMP set is made  

If no event is specified, the occurrence of any of the above events in the specified node will trigger the execution of the refresh operation. If an event is specified, only the occurrence of the specified event on the specified node triggers the execution of the refresh operation.


Specifying Node Name

<node name> can be specified two ways. If the triggering node is a direct child of the current node, or is the direct child of a superior of the current node, the object name can be used directly. For example, in the Solaris Example Module, the CPU managed object has several child managed properties, including: idle, user, system, busy, and average (derived). Any of these refresh triggers are valid for the derived average node:

refreshTrigger   busy:update  
refreshTrigger   idle:refresh  
refreshTrigger   system:status  

If the triggering node does not meet the above criteria, the full name of the triggering node must be specified. Wildcards are allowed. It is important to take care that the name is uniquely specified. Otherwise, the first node matching the name becomes the trigger.

The average node could trigger off the 15 minutes load average as follows:

refreshTrigger = *solaris*load.fifteen

Wildcards can be used as placeholders only for full node names.You cannot use wildcards to partially specify a node name. For example, the following is not valid, because the node name solaris is not fully specified:

refreshTrigger = *sol*load.fifteen


Specifying RefreshTriggers from a Node in Another Module

The triggering node must not reside in a different module from the current node. Otherwise, if the other module is unloaded, the triggering relationship is lost, and both modules must be reloaded to restore the relationship.


Note - This cannot be used for active nodes.

refreshParams

Refresh params can be specified in active and derived nodes:

refreshParams = <params>

The refreshParams qualifier is used to specify arguments to be passed to the refresh command. If the refresh mode is set to multi, multilist, or multiecm, and the refresh params specifies a space separated list of arguments, the refresh command is executed once for each argument. The next section describes refreshMode.


refreshMode

The refresh mode can be specified in active and derived nodes:

refreshMode = async | sync 

The refreshMode qualifier specifies the execution mode of the refresh command.


async

By default, active nodes have a refresh mode of async. This specification implies that the refresh operation is asynchronous. That is, the agent is allowed to process other events during the execution of the refresh command.


sync

If the refresh command must return immediately with the result, setting the refreshMode to sync reduces the overhead associated with an asynchronous command.


initInterval

The initialization interval can be specified in active and derived nodes.

initInterval = <timex specification>

The initInterval specifies, the time window within which the node must run the refreshCommand for the first time after the module initializes.


Note - initInterval does not specify an exact time at which the node will first issue its refresh command. Rather, this interval specifies a time range. Sometime within that time range, the first refresh command is issued. The exact time is randomized by the agent to spread out load.

If initInterval is specified as -1, the first execution of the refreshCommand is executed as specified by the refreshInterval.

The initialization interval enables module designers to control the rate at which the module nodes begin their data acquisition operations. This might be done to prioritize the order in which nodes initialize, or to avoid large spikes in data acquisition activity during module start up.


initHoldoff

The initialization hold off can be specified in active and derived nodes:

initHoldoff = <timex specification>

The initHoldoff qualifier specifies the time, in Timex specification, to wait before running the refresh command the first time.

The initialization hold off is typically used to delay the execution of a specific node that depends on the value of another node that must execute first. Effectively, the initHoldoff time is added to the initInterval time, thereby delaying initialization. initHoldoff must never be set to less than 2 seconds.

Here are some examples of how initHoldoff and initInterval interact:

initInterval = 10
initHoldoff = 2

The node will initialize sometime between 2 and 12 seconds.

initInterval = 10
initHoldoff = 30

The node will initialize sometime between 30 and 40 seconds.


Check Qualifiers

checkCommand, checkService and checkInterval

The checkCommand, checkService, and checkInterval can be specified for active and derived nodes to perform check operations.

checkService = <service specification>
checkCommand = <command to execute in service context> 
checkInterval = <timex specification> 

The check operation provides a mechanism for triggering refresh operations based on some criteria tested by the check operation. Typically, the check operation is a lighter weight operation and is performed at a higher frequency relative to the refresh operation. This mechanism enables managed objects to be monitored in a more timely fashion without the performance penalty of executing the refresh operation at a higher frequency.

The check operation triggers the execution of the refresh operation only when the value returned by the check command differs from the value from the previous check.

For example, when monitoring the contents of a file, the refresh operation can involve reading the contents of the file at every refresh interval. To monitor the file more effectively, the last modification date of the file can be checked at a higher frequency to determine if the file has changed. If the check detects that the file has changed, the refresh operation is triggered. Hence, the check mechanism provides an efficient way to monitor the file since checking the last modification date of a file (using stat system call) is much more lightweight than having to open, read, and close the file.

checkService specifies the service used to run the checkCommand. If this qualifier is not specified, the checkCommand shall use the refreshService service.
checkInterval specifies the interval, in timex specification, at which to run the checkCommand. This checkInterval must be specified if the checkCommand is specified.
checkInterval and refreshInterval operate completely independently of each other. refreshInterval specifies the interval at which refreshes will definitely occur. checkInterval specifies the interval at which refreshes can occur, depending on the result of the check condition.

For example:

checkInterval = 10
refreshInterval = 300

This means that every 10 seconds checkCommand is executed. If the check passes, the refresh command is invoked. But regardless of those checks, every 5 minutes the refresh command is invoked.


updateFilter

Passive managed property nodes can specify an update filter to process the data being cascaded into it.

updateFilter = <Tcl command or procedure>

The update filter specifies a Tcl command or procedure to process the data being cascaded into the passive node. Like the refreshFilter, the Tcl procedure specified by the updateFilter takes a single argument that is the data that is cascaded into the passive node.

To execute a user-defined Tcl procedure, the procedure must be available in the current node's context. To execute a user-defined Tcl command extension, the appropriate Tcl package must be loaded as described in the earlier section Loading the DAQ Services.


refreshService

Besides the refreshService command discussed in the previous chapter, the other refresh services are:


SNMP Service

refreshService = .services.snmp

Specify the SNMP service when the refresh command is an SNMP get request for acquiring data from another SNMP agent. The SNMP service object is created by all Sun Management Center agents for general SNMP communications.


Internal Service

refreshService = _internal

Specify the internal service when the refresh command is a Tcl/TOE command or procedure to be executed in the current node's context.

To execute user-defined Tcl procedures, the procedures must be available in the current node's context.

To execute user-defined Tcl command extensions, the appropriate Tcl package must be loaded as described in the earlier section"Loading the DAQ Services".


Superior Service

refreshService = _superior

Specify the superior service when the refresh command is a Tcl/TOE command or procedure to be executed in the context of the current node's superior in the tree hierarchy.

For example, in the Solaris Example module, the CPU managed object node contains a managed property node called "busy." In that example, the CPU node is the superior of the busy node.

To execute user-defined Tcl procedures, the procedures must be available in the superior's context.

To execute user-defined Tcl command extensions, the appropriate Tcl package must be loaded as described in the earlier section Loading the DAQ Services.


MIB Node Service

refreshService = <node name>

Use this type of service when the refresh command is to be executed in the context of another MIB node. The refresh command specified must be available in the context of the specified MIB node.

This specification is often used for derived nodes that specify the MIB node whose value the derived node depends on.

For example, in the Solaris Example Module, the CPU managed object has several child managed properties, including: idle, user, system, busy, and average (derived). So, the refresh service for the derived average node is set to one of these other children, such as:

refreshService = busy

For information on what are valid <node name> specifications, refer to the description of refreshTrigger in the section on "refreshTrigger".


Note - The specified node could reside in an entirely different module. However, this type of interdependency between modules is not suggested, since the modules can be loaded or unloaded independently.


Data Model Realization Specifications with Tcl procedures as DAQ


Example Data Model File

The following code example lists the additional specifications needed in the Solaris Example Model file:

CODE  EXAMPLE  6-6 Solaris Example Model File  
# Additional managed property average needs to be managed and 
# is to be added to the managed object cpu
# in the models file of solaris-example from the previous section o
# on the Example Data Model File.
cpu = { [ use MANAGED-OBJECT ]
       .....
       ....
    average = { [ use PERCENTHI MANAGED-PROPERTY ]
        shortDesc       = AvgCPU
        mediumDesc      = Average CPU Usage
        fullDesc        = Average percentage of time CPU is busy
        units           = %
 
    }

}

The following is an example of the Data Model Realization file using Tcl procedures as DAQ:

CODE  EXAMPLE  6-7 Solaris Example Model Realization File  
[ use MANAGED-MODULE ]
[ requires template solaris-example-models-d ]

#
# Load Module Parameters
#
[ load solaris-example-m.x ]

#
# Define services required by this module
#
_services = { [ use SERVICE ]
    #
    # Standard Bourne Shell
    #
    sh = {
        command = "pipe://localhost//bin/sh;transport=shell"
        max     = 2
    }
}

#
# Load filters required by this module
#
_filters = { [ use PROC ]
    [ source solaris-example-d.flt ]
}

_procedures = {
        [ use PROC ]
        [ source solaris-example-system.prc ]
        [ source solars-example-average-d.prc]

}
 
#
# Cpu Information
#
cpu = { [ use templates.solaris-example-models-d.cpu _filters ]
    type                = active
    refreshService      = _services.sh
    refreshCommand      = vmstat 10 2
    refreshFilter       = cpuFilter
    refreshInterval     = 60

    average = { [ use _procedures ]
        type            = derived
        refreshService  = _internal
        refreshTrigger  = busy:update
        refreshCommand  = getAverage 

    }
}

#
# System User and Load Information
#
system = { [ use templates.solaris-example-models-d.system }
    userstats = { [ use _filters ]
        type                = active
        refreshService      = _services.sh
        refreshCommand      = who
        refreshFilter       = userFilter
        refreshInterval     = 120

        numUsers{ [ use _procedures ]
        refreshService = _internal
        updateFilter = numUsersFilter


        primaryUser = {  [ use _procedures]
            type            = active
            refreshService  = _internal
            refreshCommand  = getPrimaryUser
            initInterval    = 10
            refreshInterval = 86400
        }
    }

    load = { 
        one = {
            type            = active
            refreshService  = _services.sh
            refreshMode = sync
            refreshCommand  = echo 10.2
            initInterval = 20
            refreshInterval     = 60

        }
        five = {  [ use _procedures ]
            type            = active
            refreshService  = _services.sh
            refreshCommand  = echo 10.2
            initInterval = 60
            refreshInterval     = 60
            checkService = _internal
            checkCommand = testFive
            checkInterval = 10
        }
    }


}

#
# Filesystem Information
#
filesystems = { [use templates.solaris-example-models\
d.filesystems _filters]
    type                = active
    refreshService      = _services.sh
    refreshCommand      = df -kF ufs
    refreshFilter       = fileFilter
    refreshInterval     = 120
}

The following is an example of the corresponding solaris-example-system.prc File:

CODE  EXAMPLE  6-8 The solaris-example-system.prc File  
# This is called by the primaryUser node
proc getPrimaryUser{}{
 set primaryUser "Hello"
 return $primaryUser
}

# This is the updateFilter for numUsers
proc numUsersFilter{ index value } {
    if { $value > 0 } {
        return $value
    } else {
        return 1
    }

}

# This is the checkCommand
proc testFive{}{
   set result 10.2
   return $result
}

The following is an example of the corresponding solaris-example-average-d.flt File:

CODE  EXAMPLE  6-9 The solaris-example-average-d.flt File  
# The corresponding solaris-example-average-d.flt file 
# finds the value of the other nodes and finds their average.  

proc getAverage{} {
   set busy [ toe_send [ locate busy ] getValue ]
   set idle [ toe_send [ locate idle ] getValue ]
   set user [ toe_send [ locate user] getValue ]
   set system [ toe_send [ locate system] getValue ]
   set average [ expr  ( $busy + $idle + $user + $system )/4 ]
}


Standard Extension for File Name

<module><-subspec>-d.prc

Objects in the MIB tree can perform special data acquisition functions or alarm status actions. These specialized functions can be specified as Tcl procedures and placed in a module-specific procedures file. This provides a simple mechanism to override or extend the functionality of the core MIB object primitives.

This file is optional for a module, and only exists if the module is using procedures. Only Tcl/TOE procedures can be defined in this file.


Loading the DAQ Services

The following section describes the Tcl procedures and node types based on the operational behavior to be used and refresh qualifiers.


Tcl Procedures

If the DAQ was implemented using Tcl procedures, the Procedures file must be loaded into a container object. Nodes that need to call a procedure defined in the Procedure file must then inherit this object.

_procedures = { [ use PROC ] 
     [ source <modules><-subspec>-d.prc ]
}
<node> = { [ use <PRIMITIVE> _procedures ] ... }


Node Type Based on Operational Behavior

Node types can be active, derived or passive.


Refresh Qualifiers

All the refresh qualifiers discussed earlier are applicable. Also, the refreshService will be _internal.


Data Model Realization Specifications with C libraries and Tcl/TOE Command Extensions as DAQ


Solaris Example Data Model Realization File

This is an example of the migration of DAQ functionality to Tcl command extensions. The discussion of this example follows in the next few sections.

CODE  EXAMPLE  6-10 Agent File Modifications 
[ requires package ssi ]
[ requires template solaris-example-models-d ]
[ use MANAGED-MODULE ]
[ load solaris-example-m.x ]

_services = { [ use SERVICE ]
    sh = {
        command = "pipe://localhost//bin/sh;transport=shell"
        max             = 2
    }
}
#
# default the refresh service for the entire module
#
refreshService      = _internal

cpu = { [ use templates.solaris-example-models-d.cpu ]
    type                = active
    refreshMode         = sync
    refreshCommand      = ssinfo cpu
    refreshInterval     = 60

    average = {
        type            = derived
        refreshTrigger  = busy:update
        refreshCommand  = digitalFilter [ valueOf busy.0 ]
        refreshParams   = 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
    }
}
system = { [ use templates.solaris-example-models-d.system ]
    userstats = { [ use _filters ]
        type                        = active
        refreshCommand              = ssinfo user
        refreshInterval             = 120
  
        primaryUser = {
            type            = active
            refreshService  = _services.sh
            refreshCommand  = solaris-example-primary-user-d.sh
            initInterval    = 10
            refreshInterval = 86400
        }
    }
 load = { [ use _filters ]
        type                        = active
        refreshCommand              = ssinfo load
        refreshInterval             = 120
    }
}

filesystems = { [use templates.solaris-example-models-
d.filesystem ]
    type                = active
    refreshMode         = sync
    refreshCommand      = ssinfo file
    refreshInterval     = 120
}

[ load solaris-example-d.def ]

Shown below are code fragments from the ssi package files.

CODE  EXAMPLE  6-11 Code Fragments From ssi Package File 
packages.h
.
.
.
#define PKG_SSI  "ssi"
extern int Ssi_Init();
.
.
.
pkgssi.c
.
.
.
int
Ssi_Init(interp)
    Tcl_Interp *interp;
{
    int code;
 
    code = Tcl_PkgProvide(interp, PKG_SSI, "1.0");
    if (code != Tcl_OK) {
        return code;
    }
    /* --- create "ssinfo" command --- */
    Tcl_CreateCommand(interp, "ssinfo", cmdSsinfo, (ClientData) 
"ssinfo",
                      (Tcl_CmdDeleteProc *) NULL);
    return(Tcl_OK);
}

int
cmdSsinfo(dummy, interp, argc, argv)
    ClientData dummy;                   /* Not used. */
    Tcl_Interp *interp;                 /* Current interpreter. */
    int argc;                           /* Number of arguments. */
    char **argv;                        /* Argument strings. */
{
.
.
.
switch(*argv[1]) {
        case "l":
        case "L":
            if(!strcasecmp(argv[1], "load")) {
                float one, five, fifteen;
        	     code = ssiGetLoadAverage(&one,&five,&fifteen);
        	       if (code == 0) {
 sprintf(buf, "%3.2f %3.2f %3.2f", one, five, fifteen);
                    Tcl_AppendResult(interp, buf, (char *)NULL);
                }
                else {
                    ssi_error = 1;
                }
                   found_option = 1;
            }
            break;
.
.
.
    return(status);
}

Shown below are code fragments of the DAQ C library code:

CODE  EXAMPLE  6-12 DAQ C code 
siSolaris.c
A code fragment from the ssi code is shown below. 
.
.
.
int ssiGetLoadAverage(float *pfOneMin, float *pfFiveMin, float 
*pfFifteenMin)
{
    long    laAveNRun[ 3 ];
    int             iSize;

    if ( !bSSIInit )
    {
        return ( ssiAPI_not_init );
    }

    if ( !( NlistArray[ X_AVENRUN ].n_value ) )
    {
         int iEntry = X_AVENRUN;
         if ( initKvmEntries ( &iEntry , 1 ) != 0 )
         {  
             return ( ssiAPI_kvm_nlist_failed );
         }
    }

    iSize = sizeof ( laAveNRun );
    if ( kvm_read(pKD, NlistArray[ X_AVENRUN ].n_value, 
                   (char *)(laAveNRun), iSize) != iSize )
    {
            return ( ssiAPI_loadavg_failed );
    }

    *pfOneMin  = (float)loaddouble ( laAveNRun[ 0 ] );
    *pfFiveMin = (float)loaddouble ( laAveNRun[ 1 ] );
    *pfFifteenMin  = (float)loaddouble ( laAveNRun[ 2 ] );

    return ( 0 );
} /* end ssiGetLoadAverage () */
.
.
.


Solaris Example - Tcl Command Extension

The migration of functionality from scripts to C involves the following steps:


Note - The initialization procedure always returns an integer with the value TCL_OK. See CODE EXAMPLE 6-11 for more information.

Writing a C Library

Although the C-code data acquisition functions can be put directly in the Tcl extension, placing the functions in a generic library enables other C programs to use the same functions, which improves code coverage and increases the reliability of the code. For example, the following is a C subroutine used to retrieve the system load average:

CODE  EXAMPLE  6-13 Code Fragment Used to Retrieve System Load Average  
int ssiGetLoadAverage(float *pfOneMin, float *pfFiveMin, float *pfFifteenMin)
{
    long    laAveNRun[ 3];
    int             iSize;

    if ( !bSSIInit )
    {
        return ( ssiAPI_not_init );
    }

    if ( !( NlistArray[ X_AVENRUN ].n_value ) )
    {
         int iEntry = X_AVENRUN;
         if ( initKvmEntries ( &iEntry , 1 ) != 0 )
         {  
             return ( ssiAPI_kvm_nlist_failed );
         }
    }
iSize = sizeof ( laAveNRun );
    if ( kvm_read(pKD, NlistArray[ X_AVENRUN ].n_value, 
                   (char *)(laAveNRun), iSize) != iSize )
    {
            return ( ssiAPI_loadavg_failed );
    }

    *pfOneMin  = (float)loaddouble ( laAveNRun[ 0 ] );
    *pfFiveMin = (float)loaddouble ( laAveNRun[ 1 ] );
    *pfFifteenMin  = (float)loaddouble ( laAveNRun[ 2 ] );
    return ( 0 );
} /* end ssiGetLoadAverage () */

This function can then be combined with other functions that determine system information to create a library of functions that access the kernel and determine system specific information.


Writing a Tcl Extension

To use C library functions in Tcl, Tcl extension files, referred to as packages, must be created. These packages define an initialization procedure that enables Tcl commands to run C code. When creating packages, consider the following issues:

These topics are described in the following sections. Refer to the Tcl documentation for information about Tcl functions (Tcl_*).


Package Naming

To enable a single Tcl application to incorporate many different packages without experiencing naming conflicts, a naming convention must be followed. This convention specifies a short, unique prefix for each package. For example, a package that uses the system- specific C library functions may use a prefix of 'ssi' (system specific interface). This prefix is then used to name the initialization function (described in the next section) and the package shared object file (pkgssi.so).


Init Function

Each package must include an initialization function. This function is called when the package is loaded. The name of the initialization procedure must contain the package prefix with the first letter capitalized followed by _Init. For example the initialization function for the ssi package would be named Ssi_Init.


Package Registration

The initialization function is used for package and command registration.

Package registration ensures that no other versions of the same package is being used currently or will be loaded later. Package registration is done using the Tcl_PkgProvide command. For example, the registration of the ssi package version 1.0 is:

code = Tcl_PkgProvide(interp, "ssi", "1.0");
if (code != Tcl_OK) {
   return code;
}


Command Registration

Commands provided by the package are registered in the Tcl interpreter using the Tcl_CreateCommand function. There must be one call to Tcl_CreateCommand for each Tcl command created. For example:

Tcl_CreateCommand(interp, "ssinfo", cmdSsinfo, (ClientData) 
"ssinfo",
                      (Tcl_CmdDeleteProc *) NULL);

In this example, a single Tcl command (ssinfo) is created to access all the ssi library functions. The Tcl_CreateCommand function creates a link between the Tcl ssinfo command and the C cmdSsinfo function. From the cmdSsinfo function, the appropriate ssi function is called based on the ssinfo command arguments. In this example, a design decision was made to create a single Tcl command to access all the ssi library functions. The other possibility is to create a Tcl command for each possible ssi library function. It is up to the developer to decide which is better.


Returning Data into Tcl

After the C library function has been called to acquire data, the data must be returned to Tcl interpreter. This is done by using the following Tcl commands:

For example to return the data from the ssiGetLoadAverage function call, the function Tcl_AppendResult is used:

code = ssiGetLoadAverage(&one,&five,&fifteen);
if (code == 0) {
    sprintf(buf, "%3.2f %3.2f %3.2f", one, five, fifteen);
    Tcl_AppendResult(interp, buf, (char *)NULL);
}


Note - Data must be returned to the Tcl interpreter as a string.

Loading the DAQ Services


Tcl Command Extension Packages

If DAQ is implement using a Tcl command extension package, the package shared object must be loaded using the following directive:

[ requires packages <package name> ]

This loads the package so that the Tcl commands provided by the package is available in the agent context. For example, to load a package named pkgssi.so, the command is:

[ requires packages ssi ]


Node Type Based on Operational Behavior

These can be active, derived or passive.


Refresh Qualifiers

To use the Tcl commands from the package, the refreshCommand must be modified to call the appropriate package function. In addition, the refreshService must be set to _internal to denote that the refreshCommand is to be run in the agent context. Finally, the refreshMode can be set to sync to optimize the function call. Other refreshQualifiers will remain the same.


Another DAQ Service


Tcl Shell Service

  To enable the agent to execute Tcl command extensions in a separate subshell process:
  1. Create binary extensions in the form of a Tcl package.
  This is described in the "C Code Libraries and Tcl/TOE Command Extensions".
  2. Create a simple Tcl wrapper script to load the packages required by the subshell.
  By convention, Tcl wrapper scripts are named with a .Tcl extension and use the pkgload command to load the required Tcl packages.
  3. Add a Tcl shell service object in the agent file.
  The Tcl shell service object is an object maintaining a pipe to one or more Tcl subshell processes where commands can be directed and the results returned asynchronously.

The module configuration file specification for a Tcl shell service object is:

 _services = { [ use SERVICE ]
        <shell> = {
command = "pipe://localhost//<Tcl shell name>;transport=shell"
            max = <max shells>
        }
    }

where:

<shell> is the name of the Tcl shell.
<Tcl shell name> is the name of the Tcl script containing the pkgload commands.
<max shells> specifies the maximum number of shell subprocess to spawn. This is typically set to 1 or 2.

The _services.solarisShell object can be used by other objects for Tcl shell DAQ services.


Solaris Example--Tcl Shell

The _services.solarisShell can be used to collect data asynchronously in a separate sub-shell.


Performance Considerations

Some data acquisition mechanisms are more efficient than others. In general, C-code libraries and Tcl command extensions are much more efficient than UNIX commands and shell scripts.

For example, CPU usage statistics can be computed using a shell script that executes the UNIX command (vmstat) and parses the result using (awk). Similarly, these CPU statistics can also be computed by running vmstat directly and parsing the results using a Tcl procedure. Using a Tcl procedure to parse data is slightly more efficient than using UNIX filter commands like sed and awk.

Alternatively, this information can be much more efficiently computed using a Tcl command extension and C system calls that do not include the overhead of creating processes and parsing data in a shell script or in Tcl.


Note - Because the Sun Management Center agent is single threaded when running Tcl commands, it is assumed that all Tcl commands return their results with little delay. If it is expected that the Tcl command will take a significant amount of time to return its result, a Tcl shell that loads the appropriate Tcl package can be spawned as a subprocess of the agent. The Tcl command can then be directed to the subprocess and the result can be returned asynchronously to ensure that the agent is not blocked by the execution of the command.




Previous Next Contents Generated Index Home

Copyright © 2000 Sun Microsystems, Inc. All Rights Reserved.