Title and Copyright Information
 
About This Manual
Audience
New and Changed Features
Organization
Related Documents
Reader's Comments
Conventions
 
1    Introduction to Performance and Availability
1.1    Performance Terminology and Concepts
1.2    High-Performance Configurations
1.2.1    CPU Resources
1.2.2    Memory Resources
1.2.2.1    Paging and Swapping
1.2.2.2    Caching Data
1.2.3    Disk Storage
1.2.3.1    RAID Technology
1.2.3.2    SCSI Concepts
1.2.3.2.1    Data Paths
1.2.3.2.2    SCSI Bus Speeds
1.2.3.2.3    Transmission Methods
1.2.3.2.4    Extending UltraSCSI Bus Segments
1.2.3.2.5    SCSI Bus Length and Termination
1.2.4    Network Subsystem
1.3    High-Availability Configurations
 
2    Planning a High-Performance and High-Availability Configuration
2.1    Identifying a Resource Model for Your Workload
2.2    Identifying Performance and Availability Goals
2.3    Choosing System Hardware
2.3.1    CPU Configuration
2.3.2    Memory and Swap Space Configuration
2.3.2.1    Determining Your Physical Memory Requirements
2.3.2.2    Choosing a Swap Space Allocation Mode
2.3.2.3    Determining Swap Space Requirements
2.3.3    I/O Bus Slot Capacity
2.3.4    Support for High-Performance Disk Storage
2.3.5    Support for High-Performance Network
2.4    Choosing Disk Storage Hardware
2.4.1    Fast Disks
2.4.2    Solid-State Disks
2.4.3    Devices with Wide Data Paths
2.4.4    High-Performance Host Bus Adapters
2.4.5    DMA Host Bus Adapters
2.4.6    RAID Controllers
2.4.7    Fibre Channel
2.5    Choosing How to Manage Disks
2.5.1    Using a Shared Pool of Storage for Flexible Management
2.5.2    Striping Data or Disks to Distribute I/O
2.5.3    Using Parity RAID to Improve Disk Performance
2.6    Choosing a High-Availability Configuration
2.6.1    Using a Cluster for System Availability
2.6.2    Using RAID for Disk Data Availability
2.6.3    Using Redundant Networks
2.6.4    Using Redundant Power Supplies and Systems
 
3    Monitoring Systems and Diagnosing Performance Problems
3.1    Obtaining Information About System Events
3.1.1    Using Event Manager
3.1.2    Using DECevent
3.1.3    Using Compaq Analyze
3.2    Using System Accounting and Disk Quotas
3.3    Continuously Monitoring Performance
3.3.1    Using Performance Manager
3.3.2    Using Performance Visualizer
3.4    Gathering Performance Information
3.5    Profiling and Debugging Kernels
3.6    Accessing and Modifying Kernel Subsystems
3.6.1    Displaying the Subsystems Configured in the Kernel
3.6.2    Displaying Current Subsystem Attribute Values
3.6.3    Displaying Minimum and Maximum Attribute Values
3.6.4    Modifying Attribute Values at Run Time
3.6.5    Modifying Attribute Values at Boot Time
3.6.6    Permanently Modifying Attribute Values
3.6.7    Displaying and Modifying Kernel Variables by Using the dbx Debugger
 
4    Improving System Performance
4.1    Steps for Configuring and Tuning Systems
4.2    Tuning Special Configurations
4.2.1    Tuning Internet Servers
4.2.2    Tuning Large-Memory Systems
4.2.3    Tuning NFS Servers
4.3    Checking the Configuration by Using the sys_check Utility
4.4    Solving Common Performance Problems
4.4.1    Application Completes Slowly
4.4.2    Insufficient Memory or Excessive Paging
4.4.3    Insufficient Swap Space
4.4.4    Processes Swapped Out
4.4.5    Insufficient CPU Cycles
4.4.6    Disk Bottleneck
4.4.7    Poor Disk I/O Performance
4.4.8    Poor AdvFS Performance
4.4.9    Poor UFS Performance
4.4.10    Poor NFS Performance
4.4.11    Poor Network Performance
4.5    Using the Advanced Tuning Guidelines
 
5    Tuning System Resource Allocation
5.1    Tuning Process Limits
5.1.1    Increasing System Tables and Data Structures
5.1.2    Increasing the Maximum Number of Processes
5.1.3    Increasing the Maximum Number of Threads
5.2    Tuning Program Size Limits
5.2.1    Increasing the Size of a User Process Stack
5.2.2    Increasing the Size of a User Process Data Segment
5.3    Tuning Address Space Limits
5.4    Tuning Interprocess Communication Limits
5.4.1    Increasing the Maximum Size of a System V Message
5.4.2    Increasing the Maximum Size of a System V Message Queue
5.4.3    Increasing the Maximum Number of Messages on a System V Queue
5.4.4    Increasing the Maximum Size of a System V Shared Memory Region
5.4.5    Increasing the Maximum Number of Shared Memory Regions Attached to a Process
5.4.6    Modifying Shared Page Table Sharing
5.5    Tuning the Open File Limits
5.5.1    Increasing the Maximum Number of Open Files
5.5.2    Increasing the Maximum Number of Open File Descriptors
 
6    Managing Memory Performance
6.1    Virtual Memory Operation
6.1.1    Physical Page Tracking
6.1.2    File System Buffer Cache Memory Allocation
6.1.2.1    Metadata Buffer Cache Memory Allocation
6.1.2.2    Unified Buffer Cache Memory Allocation
6.1.3    Process Memory Allocation
6.1.3.1    Process Virtual Address Space Allocation
6.1.3.2    Virtual Address to Physical Address Translation
6.1.3.3    Page Faults
6.1.4    Page Reclamation
6.1.4.1    Modified Page Prewriting
6.1.4.2    Reclaiming Memory by Paging
6.1.4.3    Reclaiming Memory by Swapping
6.2    Configuring Swap Space for High Performance
6.3    Displaying Memory Information
6.3.1    Displaying Memory by Using the vmstat Command
6.3.2    Displaying Memory by Using the ps Command
6.3.3    Displaying Swap Space Usage by Using the swapon Command
6.3.4    Displaying the UBC by Using the dbx Debugger
6.4    Tuning to Provide More Memory to Processes
6.4.1    Reducing the Number of Processes Running Simultaneously
6.4.2    Reducing the Static Size of the Kernel
6.4.3    Increasing the Memory Reserved for Kernel malloc Allocations
6.5    Modifying Paging and Swapping Operation
6.5.1    Increasing the Paging Threshold
6.5.2    Managing the Rate of Swapping
6.5.3    Enabling Aggressive Task Swapping
6.5.4    Limiting the Resident Set Size to Avoid Swapping
6.5.5    Managing Modified Page Prewriting
6.5.6    Managing Page-In and Page-Out Clusters Sizes
6.5.7    Managing I/O Requests on the Swap Partition
6.6    Reserving Physical Memory for Shared Memory
6.6.1    Tuning the Kernel to Use Granularity Hints
6.6.2    Modifying Applications to Use Granularity Hints
 
7    Managing CPU Performance
7.1    Gathering CPU Performance Information
7.1.1    Monitoring CPU Usage by Using the ps Command
7.1.2    Monitoring CPU Statistics by Using the vmstat Command
7.1.3    Monitoring the Load Average by Using the uptime Command
7.1.4    Checking CPU Usage by Using the kdbx Debugger
7.1.5    Checking Lock Usage by Using the kdbx Debugger
7.2    Improving CPU Performance
7.2.1    Adding Processors
7.2.2    Using the Class Scheduler
7.2.3    Prioritizing Jobs
7.2.4    Scheduling Jobs at Offpeak Hours
7.2.5    Stopping the advfsd Daemon
7.2.6    Using Hardware RAID to Relieve the CPU of I/O Overhead
 
8    Managing Disk Storage Performance
8.1    Guidelines for Distributing the Disk I/O Load
8.2    Monitoring the Distribution of Disk I/O
8.3    Displaying Disk Usage by Using the iostat Command
8.4    Managing LSM Performance
8.4.1    LSM Features
8.4.2    Basic LSM Disk, Disk Group, and Volume Guidelines
8.4.2.1    Initializing LSM Disks as Sliced Disks
8.4.2.2    Sizing the rootdg Disk Group
8.4.2.3    Sizing Private Regions
8.4.2.4    Making Private Regions in a Disk Group the Same Size
8.4.2.5    Organizing Disk Groups
8.4.2.6    Mirroring the Root File System
8.4.2.7    Mirroring Swap Devices
8.4.2.8    Saving the LSM Configuration
8.4.3    LSM Mirrored Volume Configuration Guidelines
8.4.3.1    Placing Mirrored Plexes on Different Disks and Buses
8.4.3.2    Using Multiple Plexes in a Mirrored Volume
8.4.3.3    Choosing a Read Policy for a Mirrored Volume
8.4.3.4    Using a Symmetrical Plex Configuration
8.4.3.5    Using Hot Sparing for Mirrored Volumes
8.4.4    Dirty-Region Logging Configuration Guidelines
8.4.4.1    Configuring Log Plexes
8.4.4.2    Using the Correct Log Size
8.4.4.3    Placing Logging Subdisks on Infrequently Used Disks
8.4.4.4    Using Solid-State Disks for DRL Subdisks
8.4.4.5    Using a Nonvolatile Write-Back Cache for DRL
8.4.5    LSM Striped Volume Configuration Guidelines
8.4.5.1    Increasing the Number of Disks in a Striped Volume
8.4.5.2    Distributing Striped Volume Disks Across Different Buses
8.4.5.3    Choosing the Correct LSM Stripe Width
8.4.6    LSM RAID 5 Configuration Guidelines
8.4.6.1    Using RAID 5 Logging
8.4.6.2    Using the Appropriate Strip Width
8.4.6.3    Using Hot Sparing for RAID 5 Volumes
8.4.7    Gathering LSM Information
8.4.7.1    Displaying Configuration Information by Using the volprint Utility
8.4.7.2    Monitoring Performance Statistics by Using the volstat Utility
8.4.7.3    Tracking Operations by Using the voltrace Utility
8.4.7.4    Monitoring Events by Using the volwatch Script
8.4.7.5    Monitoring Events by Using the volnotify Utility
8.5    Managing Hardware RAID Subsystem Performance
8.5.1    Hardware RAID Features
8.5.2    Hardware RAID Products
8.5.3    Hardware RAID Configuration Guidelines
8.5.3.1    Distributing Storage Set Disks Across Buses
8.5.3.2    Using Disks with the Same Data Capacity
8.5.3.3    Choosing the Correct Hardware RAID Stripe Size
8.5.3.4    Mirroring Striped Sets
8.5.3.5    Using a Write-Back Cache
8.5.3.6    Using Dual-Redundant Controllers
8.5.3.7    Using Spare Disks to Replace Failed Disks
8.6    Managing CAM Performance
 
9    Managing File System Performance
9.1    Tuning Caches
9.1.1    Tuning the namei Cache
9.1.2    Tuning the UBC
9.1.3    Tuning the Metadata Buffer Cache
9.1.4    Tuning AdvFS Access Structures
9.2    Tuning AdvFS
9.2.1    Tuning AdvFS Queues
9.2.2    AdvFS Configuration Guidelines
9.2.2.1    Storing Data Using RAID 1 or RAID 5
9.2.2.2    Forcing a Synchronous Write Request or Enabling Atomic Write Data Logging
9.2.2.3    Enabling Direct I/O
9.2.2.4    Using AdvFS to Distribute Files
9.2.2.5    Striping Data
9.2.2.6    Defragmenting a File Domain
9.2.2.7    Decreasing the I/O Transfer Size
9.2.2.8    Moving the Transaction Log
9.2.3    Displaying AdvFS Information
9.2.3.1    Displaying AdvFS Performance Statistics
9.2.3.2    Displaying Disks in an AdvFS File Domain
9.2.3.3    Displaying AdvFS File Domains
9.2.3.4    Displaying AdvFS File Information
9.2.3.5    Displaying the AdvFS Filesets in a File Domain
9.2.3.6    Displaying the Bitmap Metadata Table
9.3    Tuning UFS
9.3.1    UFS Configuration Guidelines
9.3.1.1    Modifying the File System Fragment and Block Sizes
9.3.1.2    Reducing the Density of inodes
9.3.1.3    Set Rotational Delay
9.3.1.4    Increasing the Number of Blocks Combined for a Cluster
9.3.1.5    Using MFS
9.3.1.6    Using UFS Disk Quotas
9.3.1.7    Increasing the Number of UFS and MFS Mounts
9.3.2    Displaying UFS Information
9.3.2.1    Displaying UFS Information
9.3.2.2    Monitoring UFS Clustering
9.3.2.3    Displaying the Metadata Buffer Cache
9.3.3    Tuning UFS for Performance
9.3.3.1    Adjusting UFS Smooth Sync and I/O Throttling
9.3.3.2    Delaying UFS Cluster Writing
9.3.3.3    Increasing the Number of Blocks in a Cluster
9.3.3.4    Defragmenting a File System
9.4    Tuning NFS
9.4.1    Displaying NFS Information
9.4.1.1    Displaying Network and NFS Statistics
9.4.1.2    Displaying Idle Thread Information
9.4.2    Improving NFS Performance
9.4.2.1    Configuring Server Threads
9.4.2.2    Configuring Client Threads
9.4.2.3    Modifying Cache Timeout Limits
9.4.2.4    Decreasing Network Timeouts
 
10    Managing Network Performance
10.1    Gathering Network Information
10.1.1    Monitoring Network Statistics by Using the netstat Command
10.1.2    Checking Socket Listen Queue Statistics by Using the sysconfig Command
10.2    Tuning the Network Subsystem
10.2.1    Improving the Lookup Rate for TCP Control Blocks
10.2.2    Increasing the Number of TCP Hash Tables
10.2.3    Tuning the TCP Socket Listen Queue Limits
10.2.4    Increasing the Number of Outgoing Connection Ports
10.2.5    Modifying the Range of Outgoing Connection Ports
10.2.6    Disabling PMTU Discovery
10.2.7    Increasing the Number of IP Input Queues
10.2.8    Enabling mbuf Cluster Compression
10.2.9    Enabling TCP Keepalive Functionality
10.2.10    Improving the Lookup Rate for IP Addresses
10.2.11    Decreasing the TCP Partial-Connection Timeout Limit
10.2.12    Decreasing the TCP Connection Context Timeout Limit
10.2.13    Decreasing the TCP Retransmission Rate
10.2.14    Disabling Delaying the Acknowledgment of TCP Data
10.2.15    Increasing the Maximum TCP Segment Size
10.2.16    Increasing the Transmit and Receive Buffers for a TCP Socket
10.2.17    Increasing the Transmit and Receive Buffers for a UDP Socket
10.2.18    Increasing the Maximum Size of a Socket Buffer
10.2.19    Preventing Dropped Input Packets
 
11    Managing Application Performance
11.1    Gathering Profiling and Debugging Information
11.2    Improving Application Performance
11.2.1    Using the Latest Operating System Patches
11.2.2    Using the Latest Version of the Compiler
11.2.3    Using Parallelism
11.2.4    Optimizing Applications
11.2.5    Using Shared Libraries
11.2.6    Reducing Application Memory Requirements
11.2.7    Controlling Memory Locking
 
Glossary
 
Figures
1-1    Moving Instructions and Data Through the Memory Hardware
1-2    Physical Memory Usage
1-3    Configuration with Potential Points of Failure
1-4    Fully Redundant Cluster Configuration
6-1    UBC Memory Allocation
6-2    Memory Allocation During High File System Activity and No Paging Activity
6-3    Memory Allocation During Low File System Activity and High Paging Activity
6-4    Virtual Address Space Usage
6-5    Virtual-to-Physical Address Translation
6-6    Paging and Swapping Attributes
6-7    Paging Operation
9-1    AdvFS I/O Queues
9-2    Striping Data
 
Tables
1-1    Memory Management Hardware Resources
1-2    RAID Level Performance and Availability Comparison
1-3    SCSI Bus Speeds
1-4    SCSI Bus and Segment Lengths
2-1    Resource Models and Possible Configuration Solutions
2-2    High-Performance System Hardware Options
2-3    High-Performance Disk Storage Hardware Options
2-4    High-Performance Disk Storage Configuration Solutions
2-5    High-Availability Configurations
3-1    Tools for Continuous Performance Monitoring
3-2    Kernel Profiling and Debugging Tools
4-1    Internet Server Tuning Guidelines
4-2    Large-Memory System Tuning Guidelines
4-3    NFS Server Tuning Guidelines
4-4    Advanced Tuning Guidelines
5-1    Default Values for the maxusers Attribute
5-2    IPC Limits Tuning Guidelines
6-1    Default Values for vm_page_free_target Attribute
6-2    Tools to Display Virtual Memory and UBC
6-3    Memory Resource Tuning Guidelines
7-1    CPU Monitoring Tools
7-2    Primary CPU Performance Improvement Guidelines
8-1    Disk I/O Distribution Monitoring Tools
8-2    LSM Disk, Disk Group, and Volume Configuration Guidelines
8-3    LSM Mirrored Volume Guidelines
8-4    Dirty-Region Logging Guidelines
8-5    LSM Striped Volume Guidelines
8-6    LSM RAID 5 Volume Guidelines
8-7    LSM Monitoring Tools
8-8    Hardware RAID Subsystem Configuration Guidelines
9-1    AdvFS Configuration Guidelines
9-2    Commands to Display AdvFS Information
9-3    UFS Configuration Guidelines
9-4    Commands to Display UFS Information
9-5    UFS Tuning Guidelines
9-6    Commands to Display NFS Information
9-7    NFS Tuning Guidelines
10-1    Network Monitoring Tools
10-2    Network Tuning Guidelines
11-1    Application Profiling and Debugging Tools
11-2    Application Performance Improvement Guidelines
 
Index