| About This Manual |
| Audience |
| Organization |
| Related Documents |
| Reader's Comments |
| Conventions |
| 1 | Introduction to High-Performance and High-Availability Systems |
| 1.1 | Terminology and Concepts |
| 1.1.1 | System Configuration |
| 1.1.2 | System Performance |
| 1.1.3 | Disk Performance |
| 1.1.4 | High Availability |
| 1.2 | Understanding High Availability |
| 1.2.1 | Eliminating Points of Failure |
| 1.2.2 | Increasing System Availability |
| 1.2.3 | Increasing Data Availability |
| 1.2.4 | Achieving High Availability and High Performance |
| 1.3 | Understanding High Performance |
| 1.4 | Planning Your Configuration |
| 1.4.1 | Understanding Your Workload |
| 1.4.2 | Determining Performance and Availability Goals |
| 1.4.3 | Choosing an Appropriate Configuration |
| 1.5 | Primary Configuration and Tuning Recommendations |
| 1.6 | Steps to Configure and Tune Systems |
| 2 | Diagnosing Performance Problems |
| 2.1 | Checking System Performance |
| 2.2 | Obtaining Performance Information |
| 2.2.1 | Configuring Event Logging |
| 2.2.2 | Setting up System Accounting and Disk Quotas |
| 2.2.3 | Choosing How to Monitor System Events |
| 2.2.3.1 | Using DECevent |
| 2.2.3.2 | Using Performance Manager |
| 2.2.3.3 | Using Performance Visualizer |
| 2.2.4 | Gathering Performance Statistics |
| 2.3 | Performance Monitoring Tools Overview |
| 2.4 | Gathering CPU and Virtual Memory Information |
| 2.4.1 | Using ps to Display CPU and Memory Usage |
| 2.4.2 | Using vmstat to Display Virtual Memory and CPU Statistics |
| 2.4.3 | Using uptime to Display the Load Average |
| 2.4.4 | Using swapon to Display Swap Space Usage |
| 2.4.5 | Checking CPU Usage With kdbx cpustat |
| 2.4.6 | Checking Lock Usage With kdbx lockstats |
| 2.4.7 | Checking Virtual Memory With dbx vm_perfsum |
| 2.5 | Gathering General Disk Information |
| 2.5.1 | Using iostat to Display Disk Usage |
| 2.5.2 | Checking the namei Cache With dbx nchstats |
| 2.5.3 | Checking the UBC With dbx vm_perfsum |
| 2.5.4 | Monitoring CAM Data Structures With dbx |
| 2.6 | Gathering UFS Information |
| 2.6.1 | Using dumpfs to Display UFS Information |
| 2.6.2 | Checking UFS Clustering With dbx ufs_clusterstats |
| 2.6.3 | Checking the Metadata Buffer Cache With dbx bio_stats |
| 2.7 | Gathering AdvFS Information |
| 2.7.1 | Using advfsstat to Display AdvFS Performance Information |
| 2.7.2 | Using advscan to Identify Disks in an AdvFS File Domain |
| 2.7.3 | Using showfdmn to Display AdvFS File Domain Information |
| 2.7.4 | Using showfile to Display AdvFS File Information |
| 2.7.5 | Using showfsets to Display AdvFS Filesets in a File Domain |
| 2.8 | Gathering LSM Information |
| 2.8.1 | Using volprint to Display LSM Configuration Information |
| 2.8.2 | Using volstat to Display LSM Performance Information |
| 2.8.3 | Using voltrace to Display LSM I/O Operation Information |
| 2.8.4 | Using volwatch to Monitor LSM Failures |
| 2.8.5 | Using dxlsm to Display LSM Configuration Information |
| 2.9 | Gathering Network Information |
| 2.9.1 | Using netstat to Display Network Information |
| 2.9.2 | Using nfsstat to Display Network and NFS Information |
| 2.9.3 | Checking Socket Listen Queue Statistics With sysconfig |
| 2.9.4 | Using ps to Display Idle Thread Information |
| 2.10 | Gathering Profiling and Debugging Information |
| 2.11 | Modifying the Kernel |
| 2.11.1 | Using dbx to Display and Modify Run-Time Kernel Variables |
| 2.11.2 | Using the Kernel Tuner to Display and Modify Attributes |
| 2.11.3 | Using the sysconfig Command to Display and Modify Run-Time Attributes |
| 2.11.4 | Using the sysconfigdb Command to Modify Attributes |
| 2.11.5 | Modifying Parameters in the System Configuration File |
| 3 | Optimizing Applications and CPU Performance |
| 3.1 | Configuring CPU Resources |
| 3.2 | Identifying CPU Bottlenecks |
| 3.3 | Optimizing CPU Resources |
| 3.4 | Identifying Application Bottlenecks |
| 3.5 | Improving Application Performance |
| 3.6 | Interprocess Communications Facilities |
| 4 | Configuring and Tuning Memory |
| 4.1 | Understanding Memory Management |
| 4.2 | Understanding Memory Hardware |
| 4.3 | Understanding Virtual Memory |
| 4.3.1 | Allocating Virtual Address Space to Processes |
| 4.3.2 | Translating Virtual Addresses to Physical Addresses |
| 4.3.3 | Page Faulting |
| 4.3.4 | Managing and Tracking Pages |
| 4.3.5 | Prewriting Modified Pages |
| 4.3.6 | Using Attributes to Control Paging and Swapping |
| 4.3.7 | Using Attributes to Control UBC Memory Allocation |
| 4.3.8 | Paging Operation |
| 4.3.9 | Swapping Operation |
| 4.3.10 | Using Swap Buffers |
| 4.4 | Understanding the Unified Buffer Cache |
| 4.5 | Understanding the Metadata Buffer Cache |
| 4.6 | Configuring Memory and Swap Space |
| 4.6.1 | Determining Your Physical Memory Requirements |
| 4.6.2 | Configuring Swap Space |
| 4.6.3 | Choosing a Swap Space Allocation Mode |
| 4.7 | Tuning Virtual Memory |
| 4.7.1 | Reducing the Number of Processes Running Simultaneously |
| 4.7.2 | Reducing the Static Size of the Kernel |
| 4.7.3 | Increasing the Available Address Space |
| 4.7.4 | Increasing the Available System Resources |
| 4.7.5 | Increasing the Number of Memory-Mapped Files |
| 4.7.6 | Increasing the Number of Pages With Individual Protections |
| 4.7.7 | Increasing the Size of a System V Message and Queue |
| 4.7.8 | Increasing the Size of a System V Shared Memory Region |
| 4.7.9 | Increasing the Minimum Size of a System V Shared Memory Segment |
| 4.7.10 | Reducing Application Memory Requirements |
| 4.7.11 | Reducing the Memory Available to the UBC |
| 4.7.12 | Changing the Rate of Swapping |
| 4.7.13 | Controlling Dirty Page Prewriting |
| 4.7.14 | Modifying the Size of the Page-In and Page-Out Clusters |
| 4.7.15 | Modifying the Swap I/O Queue Depth for Pageins and Swapouts |
| 4.7.16 | Modifying the Swap I/O Queue Depth for Pageouts |
| 4.7.17 | Modifying the UBC Write Device Queue Depth |
| 4.7.18 | Controlling Large File Caching |
| 4.7.19 | Increasing the Paging Threshold |
| 4.7.20 | Enabling Aggressive Task Swapping |
| 4.7.21 | Decreasing the Size of the Metadata Buffer Cache |
| 4.7.22 | Decreasing the Size of the namei Cache |
| 4.7.23 | Decreasing the Size of the AdvFS Buffer Cache |
| 4.7.24 | Reserving Physical Memory for Shared Memory |
| 4.7.24.1 | Tuning the Kernel to Use Granularity Hints |
| 4.7.24.2 | Modifying Applications to Use Granularity Hints |
| 4.8 | Tuning the UBC |
| 4.8.1 | Increasing the Maximum Size of the UBC |
| 4.8.2 | Decreasing the Amount of Borrowed Memory |
| 4.8.3 | Increasing the Minimum Size of the UBC |
| 4.8.4 | Using mmap in Your Applications |
| 4.9 | Tuning the Metadata Buffer Cache |
| 4.9.1 | Increasing the Size of the Metadata Buffer Cache |
| 4.9.2 | Increasing the Size of the Hash Chain Table |
| 5 | Configuring and Tuning Storage Subsystems |
| 5.1 | Understanding Storage Subsystems |
| 5.2 | Choosing How to Manage Disks and Files |
| 5.2.1 | Understanding RAID Levels and Products |
| 5.2.1.1 | Hardware RAID Subsystem Features |
| 5.2.1.2 | LSM Features |
| 5.2.2 | Understanding AdvFS |
| 5.3 | General Disk Storage Guidelines |
| 5.3.1 | High-Performance Hardware Guidelines |
| 5.3.1.1 | Using Fast Disks |
| 5.3.1.2 | Using Disks with Small Platters |
| 5.3.1.3 | Using Disks with Wide Data Paths |
| 5.3.1.4 | Using Solid-State Disks |
| 5.3.1.5 | Using High-Performance Host Bus Adapters |
| 5.3.1.6 | Using DMA Host Bus Adapters |
| 5.3.1.7 | Using Prestoserve |
| 5.3.1.8 | Using Write-Back Caches |
| 5.3.2 | Distributing the Disk I/O Load Guidelines |
| 5.3.2.1 | Distributing Swap Space Across Disks and Buses |
| 5.3.2.2 | Distributing I/O Across Disks and Buses |
| 5.3.2.3 | Distributing File Systems Across Disks |
| 5.3.2.4 | Placing Data at the Beginning of ZBR Disks |
| 5.3.3 | General File System Tuning Guidelines |
| 5.3.3.1 | Increasing the Maximum Number of Open Files |
| 5.3.3.2 | Increasing the Size of the namei Cache |
| 5.3.3.3 | Increasing the Size of the Hash Chain Table for the namei Cache |
| 5.3.3.4 | Allocating More Memory for the UBC |
| 5.3.3.5 | Using Prestoserve to Cache Only File System Metadata |
| 5.3.3.6 | Caching More Free vnodes |
| 5.3.3.7 | Increasing the Time vnodes Remain on the Free List |
| 5.3.3.8 | Delaying the Deallocation of vnodes |
| 5.3.3.9 | Accelerating the Deallocation of vnodes |
| 5.3.3.10 | Disabling vnode Deallocation |
| 5.3.3.11 | Modifying the Maximum Number of Open File Descriptors |
| 5.3.3.12 | Disabling Clearing of the DMA Scatter/Gather Map Registers |
| 5.4 | Using the Logical Storage Manager |
| 5.4.1 | Basic LSM Configuration Guidelines |
| 5.4.1.1 | Initializing LSM Disks as Sliced Disks |
| 5.4.1.2 | Increasing the Maximum Number of LSM Volumes |
| 5.4.1.3 | Sizing the rootdg Disk Group |
| 5.4.1.4 | Sizing Private Regions |
| 5.4.1.5 | Making Private Regions in a Disk Group the Same Size |
| 5.4.1.6 | Grouping Disks in Disk Groups |
| 5.4.1.7 | Choosing the Number and Size of the Database and Log Copies |
| 5.4.1.8 | Distributing the Database and Log Copies Across Buses |
| 5.4.2 | LSM Mirrored Volume Configuration Guidelines |
| 5.4.2.1 | Mirroring Volumes Across Different Buses |
| 5.4.2.2 | Choosing a Read Policy for a Mirrored Volume |
| 5.4.2.3 | Using Multiple Plexes in a Mirrored Volume |
| 5.4.2.4 | Using a Symmetrical Configuration |
| 5.4.2.5 | Using Multiple BCL Subdisks |
| 5.4.2.6 | Using a Write-Back Cache with LSM |
| 5.4.2.7 | Sizing BCL Subdisks |
| 5.4.2.8 | Placing BCL Logging Subdisks on Infrequently Used Disks |
| 5.4.2.9 | Using Solid-State Disks for BCL Subdisks |
| 5.4.3 | LSM Striped Volume Configuration Guidelines |
| 5.4.3.1 | Increasing the Number of Disks in a Striped Volume |
| 5.4.3.2 | Distributing Striped Volume Subdisks Across Different Buses |
| 5.4.3.3 | Choosing the Correct Stripe Width |
| 5.4.4 | LSM Tuning Guidelines |
| 5.5 | Hardware RAID Subsystem Configuration Guidelines |
| 5.5.1 | Distributing Storage Set Disks Across Buses |
| 5.5.2 | Using Disks with the Same Data Capacity |
| 5.5.3 | Choosing the Correct Chunk Size |
| 5.5.4 | Striping Mirrored Sets |
| 5.5.5 | Using a Write-Back Cache |
| 5.5.6 | Using Dual-Redundant Controllers |
| 5.5.7 | Using Spare Disks |
| 5.6 | Using the Advanced File System |
| 5.6.1 | AdvFS Configuration Guidelines |
| 5.6.1.1 | Using Multiple-Volume File Domains |
| 5.6.1.2 | Improving the Transaction Log Performance |
| 5.6.1.3 | Improving Bitmap Metadata Table Performance |
| 5.6.1.4 | Striping Files |
| 5.6.1.5 | Using AdvFS Quotas |
| 5.6.2 | AdvFS Tuning Guidelines |
| 5.6.2.1 | Modifying the Size of the AdvFS Buffer Cache |
| 5.6.2.2 | Defragmenting a File Domain |
| 5.6.2.3 | Increasing the Dirty Data Caching Threshold for a Volume |
| 5.6.2.4 | Decreasing the I/O Transfer Read-Ahead Size |
| 5.6.2.5 | Disabling the Flushing of Dirty mmapped Pages |
| 5.6.2.6 | Modifying the AdvFS Device Queue Limit |
| 5.6.2.7 | Consolidating I/O Transfers |
| 5.6.2.8 | Forcing Synchronous Writes |
| 5.6.2.9 | Moving the Transaction Log |
| 5.6.2.10 | Balancing a Multivolume File Domain |
| 5.6.2.11 | Migrating Files Within a File Domain |
| 5.7 | Using the UNIX File System |
| 5.7.1 | UFS Configuration Guidelines |
| 5.7.1.1 | Modifying the File System Fragment Size |
| 5.7.1.2 | Reducing the Density of inodes |
| 5.7.1.3 | Allocating Blocks Contiguously |
| 5.7.1.4 | Increasing the Number of Blocks Combined for a Read |
| 5.7.1.5 | Using a Memory File System |
| 5.7.1.6 | Using UFS Disk Quotas |
| 5.7.2 | UFS Tuning Guidelines |
| 5.7.2.1 | Defragmenting a File System |
| 5.7.2.2 | Delaying Full Write Buffer Flushing |
| 5.7.2.3 | Increasing the Number of Blocks Combined for Read Ahead |
| 5.7.2.4 | Increasing the Number of Blocks Combined for a Write |
| 5.7.2.5 | Increasing the Number of UFS or MFS Mounts |
| 5.8 | Tuning CAM |
| 6 | Tuning the Network Subsystem |
| 6.1 | Tuning Networks |
| 6.1.1 | Improving the Lookup Rate for TCP Control Blocks |
| 6.1.2 | Tuning the Socket Listen Queue Limits |
| 6.1.3 | Increasing the Maximum Number of Concurrent Nonreserved Dynamically Allocated Ports |
| 6.1.4 | Enabling TCP keepalive Functionality |
| 6.1.5 | Improving the Lookup Rate for IP Addresses |
| 6.1.6 | Decreasing the Partial TCP Connection Timeout Limit |
| 6.1.7 | Decreasing the TCP Connection Context Timeout Limit |
| 6.1.8 | Decreasing the TCP Retransmission Rate |
| 6.1.9 | Disabling Delaying the Acknowledgment of TCP Data |
| 6.1.10 | Increasing the Maximum TCP Segment Size |
| 6.1.11 | Increasing the Transmit and Receive Buffers for a TCP Socket |
| 6.1.12 | Increasing the Transmit and Receive Buffers for a UDP Socket |
| 6.1.13 | Allocating Sufficient Memory to the UBC |
| 6.1.14 | Disabling Use of a PMTU |
| 6.2 | Tuning the Network File System |
| 6.2.1 | Using Prestoserve to Improve Server Performance |
| 6.2.2 | Using the Appropriate Number of nfsd Daemons |
| 6.2.3 | Using the Appropriate Number of nfsiod Daemons |
| 6.2.4 | Increasing the Number of Threads |
| 6.2.5 | Modifying Cache Timeout Limits |
| 6.2.6 | Decreasing Network Timeouts |
| 6.2.7 | Using NFS Protocol Version 3.0 |
| A | Tuning Special Configurations |
| A.1 | Tuning Internet Servers |
| A.2 | Tuning a Low-Memory Workstation |
| A.2.1 | Attribute Settings for Low-Memory Workstations |
| A.2.2 | Swap Space and Memory Tuning on Low-Memory Systems |
| A.2.3 | X Window System Considerations for Low-Memory Workstations |
| B | Configuration Attribute Definitions |
| B.1 | AdvFS Subsystem Attributes |
| B.2 | TTY Subsystem Attribute |
| B.3 | Configuration Manager Subsystem Attribute |
| B.4 | DLI Subsystem Attributes |
| B.5 | Generic Kernel Subsystem Attributes |
| B.6 | Internet Subsystem Attributes |
| B.7 | I/O Subsystem Attributes |
| B.8 | IPC Subsystem Attributes |
| B.9 | LSM Subsystem Attributes |
| B.10 | Network Subsystem Attributes |
| B.11 | Prestoserve Subsystem Attribute |
| B.12 | Process Subsystem Attributes |
| B.13 | Pseudoterminal Subsystem Attribute |
| B.14 | Real-Time Subsystem Attributes |
| B.15 | Security Subsystem Attributes |
| B.16 | SNMP Information Subsystem Attribute |
| B.17 | Socket Subsystem |
| B.18 | STREAMS Subsystem Attributes |
| B.19 | UFS Subsystem Attributes |
| B.20 | VFS Subsystem Attributes |
| B.21 | Virtual Memory Subsystem Attributes |
| B.22 | XPR Subsystem Attributes |
| Glossary |
| Figures |
| 1-1 | Configuration With Potential Points of Failure |
| 1-2 | Fully Redundant Cluster Configuration |
| 1-3 | Configuration and Tuning Process |
| 4-1 | Physical Memory Usage |
| 4-2 | Moving Instructions and Data Through the Memory Hardware |
| 4-3 | Time Consumed to Access Storage Locations |
| 4-4 | Virtual Address Space Usage |
| 4-5 | Virtual-to-Physical Address Translation |
| 4-6 | Paging and Swapping Attributes - Default Values |
| 4-7 | UBC Memory Allocation |
| 4-8 | Paging Operation |
| Tables |
| 1-1 | Increasing System Availability |
| 1-2 | Increasing Data Availability |
| 1-3 | Impact of High Availability on System Performance |
| 1-4 | Application Characteristics |
| 1-5 | User Characteristics |
| 1-6 | System Characteristics |
| 1-7 | System Requirements |
| 2-1 | CPU and Memory Monitoring Tools |
| 2-2 | General Disk Monitoring Tools |
| 2-3 | UFS Monitoring Tools |
| 2-4 | AdvFS Monitoring Tools |
| 2-5 | LSM I/O Performance and Event Monitoring Tools |
| 2-6 | Network Monitoring Tools |
| 2-7 | Profiling and Debugging Tools |
| 4-1 | Memory Management Hardware Resources |
| 4-2 | Primary Virtual Memory Tuning Guidelines |
| 4-3 | Advanced Virtual Memory Tuning Guidelines |
| 4-4 | Guidelines for Tuning the UBC |
| 4-5 | Guidelines for Tuning the Metadata Buffer Cache |
| 5-1 | RAID Level Performance and Availability Features |
| 5-2 | Guidelines for High-Performance Hardware Configurations |
| 5-3 | Guidelines for Distributing the Disk I/O Load |
| 5-4 | Guidelines for General File System Tuning |
| 5-5 | Guidelines for LSM Disks, Disk Groups, and Databases |
| 5-6 | Configuration Database and Kernel Change Log Guidelines |
| 5-7 | Guidelines for LSM Mirrored Volumes |
| 5-8 | Guidelines for LSM Block-Change Logging |
| 5-9 | Guidelines for LSM Striped Volumes |
| 5-10 | Guidelines for Configuring Hardware RAID Subsystems |
| 5-11 | AdvFS Configuration Guidelines |
| 5-12 | BMT Sizing Guidelines |
| 5-13 | AdvFS Tuning Guidelines |
| 5-14 | UFS Configuration Guidelines |
| 5-15 | UFS Tuning Guidelines |
| 6-1 | Network Tuning Guidelines |
| 6-2 | Guidelines for NFS Tuning |
| Index |