1    Introducing AdvFS

AdvFS is a log-based file system that provides flexibility, compatibility, data availability, and high performance. AdvFS takes advantage of the 64-bit computing environment and is designed to handle files and filesets as large as almost 16 terabytes. For hardware and software requirements, see the Technical Overview.

The configuration of AdvFS differs from the traditional UNIX file system. In AdvFS, the physical storage layer is managed independently of the directory layer. System administrators can add and remove storage without unmounting the file system or halting the operating system. As a result, configuration planning is less complicated and more flexible.

From a user's perspective, AdvFS behaves like any other UNIX file system. End users can use the mkdir command to create new directories, the cd command to change directories, and the ls command to list directory contents. AdvFS logical structures, quota controls, and backup capabilities are based on traditional file system design. AdvFS replaces or eliminates several standard commands, such as newfs, dump, restore, and fsck. AdvFS commands and utilities are described in Appendix A.

Without taking an AdvFS system off line, system administrators can perform backups, file system reconfiguration, and file system tuning. End users can retrieve their own unintentionally deleted files from predefined trashcan directories or from clone filesets without assistance from system administrators.

AdvFS supports multivolume file systems, which enables file-level striping (spreading data to more than one volume) to improve file transfer rates for I/O-intensive applications. Logical Storage Manager (LSM), which allows volume-level striping, can be incorporated into AdvFS configurations.

AdvFS Utilities, which is licensed separately from the Tru64 UNIX operating system, provides additional file management capabilities and a graphical user interface (GUI) to simplify system administration. The AdvFS GUI, which runs under the Common Desktop Environment (CDE), features menus, graphical displays, and comprehensive online help that make it easy to perform AdvFS operations. In addition, the GUI displays summarized system status information.

1.1    Advanced File System Features

Table 1-1 lists the main AdvFS features and their benefits.

Table 1-1:  AdvFS Features and Benefits

* This feature requires the optional AdvFS Utilities license.

1.2    What's New in Tru64 UNIX Version 5.0A

The following features have been added to AdvFS:

• Improved disk structure that increases the number of files the file domain can track (see Section 2.3.3)

• A disk salvage utility that can recover information at the block level from disks that have been damaged (see Section 7.5.8)

• An improved directory structure that increases the speed of file creation and access (see Section 2.3.3)

• Enhanced vdump and vrestore command capability (see Chapter 4)

• Remote device support for backup and restore (see Chapter 4)

• Increased quota limits (see Chapter 3)

• Direct I/O to allow unbuffered, synchronous I/O (see Section 5.1.4)

• Smooth sync option to promote continuous I/O (see Section 5.2.6)

• New utilities (such as vdf, which displays disk usage for file domains and filesets) (see Section 3.3.4.2)

• Metadata display utilities (see Section 7.7.2)

1.3    File System Design

AdvFS consists of two distinct layers: the directory hierarchy layer and the physical storage layer. The directory hierarchy layer implements the file-naming scheme and POSIX-compliant functions such as creating and opening files, or reading and writing to files. The physical storage layer implements write-ahead logging, caching, file allocation, and physical disk I/O functions.

This decoupled file system structure enables you to manage the physical storage layer apart from the directory hierarchy layer. This means that you can move files between a defined group of disk volumes without changing path names for your files. Because the path names remain the same, the action is completely transparent to end users.

AdvFS can incorporate Logical Storage Manager (LSM) volumes into the file system structure. AdvFS configured with LSM improves file system reliability and availability because AdvFS can take advantage of LSM features (see Section 2.8).

1.3.1    Filesets, File Domains and Volumes

AdvFS implements two unique file system concepts: filesets and file domains. Filesets and file domains enable a two-layer file system structure that decouples the physical storage layer of the file system from its directory hierarchy. With this unique architecture, you can manage the logical structure of your file systems independently of the storage volumes that contain them. Figure 1-1 is a representation of this structure.

Figure 1-1:  AdvFS File System Design

A fileset follows the logical structure of a traditional UNIX file system. It is a hierarchy of directory names and file names, and it is what you mount. AdvFS goes beyond the traditional file system by allowing you to create multiple filesets that share a common pool of storage called a file domain. See Section 2.4 for more information about filesets.

A file domain is a pool of storage that is managed separately from the directory structure. You can add or remove volumes without affecting the directory structure. See Section 2.3 for more information.

A volume is any mechanism that behaves like a UNIX block device: an entire disk, a disk partition, or an aggregate volume provided by a logical storage manager (see Section 2.2). When first created, all file domains consist of a single volume. If you have the optional AdvFS Utilities (see Section 1.4), you can transform a single-volume file domain into a multivolume file domain by adding one or more volumes.

File domain names can be up to 255 characters and filesets can have names of up to 31 characters. You cannot use slash (/), pound (#), colon (:), asterisk (*), question mark (?), tab, newline, formfeed, return, vertical tab, and space characters. Filesets that are not in the same file domain can have the same name and file names must be unique in the directory.

1.3.2    Transaction Log

AdvFS is a log-based file system that employs write-ahead logging to ensure the integrity of the file system. Write-ahead logging means that modifications to the metadata (the file structure information) are completely written to a transaction log before the actual changes are written to disk. The log is implemented as a circular file buffer so that the contents of the transaction log are written to disk at regular intervals. By committing only completed transactions to disk, the file system is not left in an inconsistent state after an unexpected system crash.

When a file domain is created, AdvFS creates a transaction log for it. The first time the domain is activated, 4 megabytes of storage are allocated for the log. It is separate from the user data but can be stored on the same device.

During crash recovery, AdvFS consults the transaction log to confirm file system transactions. All completed transactions are committed to disk and uncompleted transactions are undone. The number of uncommitted records in the log, not the amount of data in the file system, dictates the speed of recovery. This means that recovery usually takes only a few seconds. Traditional UNIX file systems rely on the fsck utility to recover from a system failure. The fsck utility can take hours to check and repair a large file system.

By default, only file structures are logged, but you can choose to log file data (see Section 2.7). A file that has data logging turned on will remain internally consistent in the event of a system crash. However, enabling data logging can slow system performance.

It is possible to move the transaction log to a faster volume if the one it is on is causing a bottleneck (see Section 5.1.3).

Note that the version of the transaction log on one version of the operating system may not be compatible with the version on another. After a crash it is important to recover your files on the same operating system on which it crashed (see Section 7.7.4).

1.3.3    File Storage Allocation

Files are not static; their space requirements change over time. To maintain contiguous file placement without overallocating space on the disk, AdvFS uses a unique file storage allocation scheme.

Key features of storage allocation are:

• Extents

  An extent is a contiguous area of disk space that AdvFS allocates to a file. Extents are composed of sets of one or more 8-kilobyte pages. When storage is added to a file, it is grouped in extents. There is an extent map for each volume on which the file system resides.

  File I/O is most efficient when there are few extents. If a file consists of many small extents, it takes more I/O processing to read or write that file.

  Given the dynamic nature of a file system, the file-storage allocation cannot always guarantee contiguous placement of pages. The following factors affect placement:

  • Excessive disk fragmentation

    When a disk is fragmented, there are many small free spaces, so AdvFS writes data to isolated physical pages, based on availability, instead of writing to contiguous pages. This may result in files with many extents.

  • Multiple users

    When there are many users on a system, requests for space increase, decreasing the likelihood of contiguous file allocation.

  If you have a file domain with many extents, you can decrease the number by running the defragment utility (see Section 5.2.2).

• Preallocation

  Each time a file is appended, AdvFS adds pages to the file by preallocating one-fourth of the file size, up to 16 pages. Excess preallocated space is truncated when the file is closed.

  For multivolume file domains, new files are allocated sequentially across volumes. Volumes that are more than 86% full (allocated) are not used for new file allocation unless all volumes are more than 86% full. When data is appended to existing files, storage is allocated on the volume on which the file was initially allocated until the volume is full.

• Fragments

  AdvFS writes files to disk in sets of 8-kilobyte pages. In files where holding the last bytes in an 8-kilobyte page would represent a waste of more than 5% of the allocated space, a file fragment is created. The fragment, which ranges in size from 1 kilobyte to 7 kilobytes, is allocated from the frag file, a special file used for storing the file fragments. This technique considerably reduces the amount of unused, wasted disk space.

• Sparse files

  A sparse file is one that does not have disk storage reserved for some of its pages. Reading a sparse file at a page that does not have storage reserved returns zeroes. Writing to a page that does not have storage reserved allocates a page of disk storage.

  Core files are sparse files. They have large areas with no information and don't use up disk blocks for locations where no data exists. Quota files are sparse because they are indexed by user ID. If there are gaps in the user IDs, there may be sections of the file with no data.

  In contrast, database files generally reserve storage for the entire file even when data is not available. Database applications generally write zeroes in the pages that have no useful data so that storage is allocated. In addition, by writing the data sequentially, a database file is created with large sets of contiguous pages and a small number of extents.

  To examine the length of a sparse file, including the pages that don't have disk storage, use the ls command with the -l option. The ls command with the -s option shows the amount of storage actually used by the file. The du command gives the same information as the ls -s command.

1.4    License Registration

AdvFS is a file system option on the operating system. AdvFS Utilities is a separately licensed product that provides additional processing capabilities. Before you can use the file system utilities, you must register a license product authorization key (PAK) for AdvFS Utilities. Contact your software support organization for additional information.

1.5    Related Information

Other useful documentation includes:

• System Administration

• Installation Guide

• Logical Storage Manager

• Technical Overview

• Software License Management

• System Configuration and Tuning