Recent Changes

The following is the complete list of changes made in the last year:

Historic Changes

The following changes may be of interest to early adopters of Collect:

August - October, 1998 (Collect 1.09 - 1.10)

• Bug in -T option fixed (produced "can't read from kernel" error message).
• Fixed bug with -PP <pidlist> on playback not working correctly.
• Added call to plock() to lock program in memory (can't be swapped out anymore). Collect is now compiled statically so that all the necessary pages get locked.
• Added call to renice() to give collect highest priority by default. There is a flag (-on) to disable.
• Added ability to play back and convert multiple data files.
• Added ability to read and write compressed datafiles. Compression is enabled by default for writing. Compressed input files are recognized automatically.
• Added switch (-R) to set a duration for Collect to run, after which it will automatically stop.
• Collect can now read from stdin using the -p option and write to stdout using the -f option.

collgui

• Added companion ability to select multiple data files.

June - August, 1998 (Collect 1.08d - 1.09)

• Support for Tru64 UNIX.
• Bugs fixed that caused unusual disk configurations to produce a core dump (when scsi floppy drive present, for example).
• Added PPID (parent PID) to data collected for each process (for determining creation hierarchy of processes in collected data). Also added the ability to select on PPID during playback.
• Added fork/vfork statistics to cpu subsys.
• Changed LSM output such that average service time is displayed separately for read and write operations.
• LSM not tested under Tru64 UNIX Version 5.0.
• Fixed bug when selecting processes during playback by username.
• Added new disk statistics (only available in Tru64 UNIX Version 5.0):
  • AVW - Average wait in milliseconds
  • WTQ - Number of requests in wait queue
  • %WT - Percent of requests that must spend time in wait queue
• Changed the system & user time stats in the process subsystem to display by default a normalized delta -- that is, the accumulated cpu time since the last sample normalized to 1 second. Therefore, the units are CPU-seconds per second. The cumulative time can still be displayed with a switch (see collect -h).

collgui

• Added -vga switch for VGA resolution screens (640x480).
• Added -size <size> to allow font size adjustment.
• Hard-coded text foreground to black and text background to beige -- should eliminate problems with white-on-white text under CDE.

Miscellaneous

• Added entries to /etc/magic so that file collect.output works.

June/July, 1997 (Collect 1.08b - 1.08c)

• Fixed bug in NET subsystem of BW being shown in increments of 3%.
• Added TTY subsystem.
• Added CDROMs to disk subsystem.
• Improved the 'Add' listboxes -- multiple objects can now be selected.
• Sped up filtering when processes are being selected. Now Collect is only asked to extract data for the PIDs requested, so cfilt doesn't have to wade through so much data.
• Division can now be used in cfilt with impunity. Division-by-zero errors are trapped, and a zero is returned as result.
• An optional arithmetic expression can now be evaluated after normalization has been done in cfilt.
• On systems for which there is no pre-build binary, a custom version will be built and installed.
• Incompatible Change: The normalize directive is now a hash (#) sign instead of a percent (%) sign.

May, 1997 (1.07 - 1.08)

• Added a Tape subsystem for statistics about SCSI tape drives.
• Added message queue subsystem.
• Fixed an oversight by which Collect wasn't able to seek to the last record when the records were > 32 kilobytes. Records can be up to 256 kilobytes before this causes problems.
• RAID disks can now also be specified using -DreX,reY,...
• Total flag (-T) now causes totals for disks and tapes to be displayed.
• Added -PP<pid-list>, -PC<command-list>, -PU<uid-list> options to allow collection for processes having specified parent-PIDS or belonging to specified process-group (-PP), having specified string in command (-PC), or owned by specified UID (-PU). Fixed bug when using the -n switch in playback mode.
• Added support for regular expressions for the -L and -D flags
• Added ability to convert version 7 (collect V1.07) datafiles to version 8 (collect V1.08), and simultaneously extract records for a particular interval and select subsystems.
• Changed absolute memory values from pages to megabytes.
• Disk and tape counting (for data structure allocation) and discovery has been made consistent. A side-effect of this is that a SCSI floppy drive will no longer cause Collect to dump core. (It was not being counted, but was being discovered, so Collect tried to write data into non-existent data structures.)

cfilt

• Fixed bug in cfilt that caused problems with uppercase letters in names (such as LSM volumes).

collgui

• Added new selection mechanism for lists of objects longer than can be displayed in a popup menu.
• Added a new selection mechanism for processes.
• Added a file browser for opening files.
• Added image file output (jpeg, ppm, pbm).

December, 1996 (Collect 1.06 - 1.07)

• The binary format has changed. Version 1.07 will not be able to read pre-1.07 data files. If this affects you, rename the 1.06 executable to collect1.06 (or something) and move it out of /usr/opt/COL106/bin if you used the setld kit.
• Fixed bug in collgui when two or more objects are selected and normalization is enabled. Normalization wasn't being done. This example: lsm:name=vol01,vol02:rkb/s+wkb/s% didn't work properly.
• Fixed bug in collect that affected the selection of disks with non-null LUNs (mostly on HSZs), such as rza33, rzb33, etc.
• New reference pages for cfilt and collgui.
• Collect now flushes stdout when writing to it, and cfilt will immediately process data if no normalization is being done, so it is now possible to do the following:

  collect -i1 -F -sp | cfilt proc+:user=smith:rss

  to sum RSS for user smith for all processes owned by smith.
• Fixed bug in load average that caused all load averages over 2.55 to be misrepresented.
• Superficially tested RAID (SWXCR) support.
• In collgui, changed the Memory Used expression in the memory subsystem to use the cfilt expression Active+Inactive+Wired because Active already contains the UBC pages.

October, 1996 (Collect 1.05 - 1.06)

• Added to process info input blocks, output blocks, major faults and minor faults. This caused the line to be longer than 80 columns, requiring the next fix.
• Added the -F option. Normally, not all fields are printed, and some things, such as RSS and VSZ are converted to compact format using K, M, and G tags. The -F option causes all information to print, in expanded form.
• Added to memory info processes swapped per second, UBC hits/sec, UBC pages pushed (written)/sec, and UBC pages allocated/sec.
• Improved the -C (chop out a timeslice) flag. The format is somewhat different: either the start or end time can be in the format [+]Year:Month:Day:Hour:Minute:Second. If there is a plus sign at the beginning, then the time specified is relative to the beginning of the collection period. If there is no plus sign, then the time is absolute. Any of Year, Month, Day, Hour, or Minute can be left off, in which case the values from the beginning of the collection period are taken. Therefore, if you collected from 10:23:34 to 10:44:15 on October 16th, you could use -C24:00,25:00 to extract samples from 10:24:00 to 10:25:00. You could also use: -C10:24:00,10:25:00, or even -C16:10:24:00,16:10:25:00, etc. If no start time (-C,10:25:00) or no end time (-C10:24:00,) is given, then the collection start and end times are used, respectively.
• Added (-l) option to seek to and print the last valid record. This aids collgui, reducing the time to open a large binary data file.
• Adjusted collgui to use the new time format of Collect.
• New reference page for collgui.
• New reference page for cfilt.
• Added call to task_set_notify_port (suggested by Eric Schott) so that kernel doesn't store non-deliverable mach messages and eat memory.

September, 1996 (Collect 1.04 - 1.05)

• Improved cfilt to understand almost any arbitrary expression, such as

   (100-rkb/s+wkb/s)/100" or "log(idle)

  .
• Added a graphical front-end for collect, cfilt, and gnuplot called collgui.
• Integrated functionality of cavg into cfilt with the -a num switch.
• Added the -p flag to cfilt to select only those samples that contain process data.
• Saved 8 bytes/process record (but these will soon be gobbled up by new process-record elements).

September 4, 1996 (Collect 1.03 - 1.04)

• Tested and fixed LSM support under DIGITAL UNIX Version 3.x and Version 4.0.
• Added -L option for specifying LSM volumes to collect or playback.
• Cleaned up on_disk.h.
• Added the %busy calculation for disks. This is based on:

  		(time spent doing I/Os)
  		----------------------- x 100
  		 (real time interval)
  

  Generally, a disk will be 100% busy during a tranfer, so it's just a question of for how much of the measurment interval was there an outstanding request. This has nothing to say about number of transfers, kilobits or kilobits/second transferred, etc.
• Added -DLSM to makefile to explicitly build collector with LSM support.
• Added reference page for Collect.
• Added -v flag to print program and datafile version.

August 26, 1996 (Collect 1.02 - 1.03)

• Changed -u (use) to -s (select). Sorting is now -S (capital).
• There is a new output field for disks, %BSY - percent busy. This only works under DIGITAL UNIX Version 4.0x. It is simply the service time (the time it took for the disk to complete all requests) of the disk over the interval divided by the interval.
• Added the magic number to the main header as well, so when a file is given for playback that was not written by Collect, a reasonable error message is printed.

August 8, 1996 (Collect 1.01 - 1.02)

• Fixed sorting bug: -s -nX now works properly.
• -D disklist, -P proclist, -s -nX options now work in playback mode.
• New option, -C [starttime,endtime], added to extract samples from a particular time-range.
• Collector now waits one interval before writing first sample. This solves the problem of the first sample having bogus values due to false delta values.
• Fixed bug with non-printable characters in header after CPU-TYPE.
• Added Perl script Scripts/filter that is suitable for extracting values to be read into Microsoft® Excel or gnuplot.

July 14, 1996 (1.00 - 1.01)

• A list of disks for which statistics are to be gathered can now be given to the collector in the form -DrzXX,rzXY,... (only works in collect mode, not playback).
• When a non-privileged user runs Collect and Collect is setuid-root, if a binary-output file is opened, the open is performed as the non-privleged user. This should make the program completely safe for setuid-root.
  If the binary-output file already exists, the user is prompted for confirmation before overwriting.