WASD Scripts for CSWS 1.3 and SWS 2.0 (or later)
Alpha (AXP) and Itanium (IA64)
 May 2005
WASD VMS Hypertext Services, Copyright © 1996-2005 Mark G. Daniel.
This package (all associated programs), comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it under the conditions
of the GNU GENERAL PUBLIC LICENSE, version 2.

Calendar Simple script to produce calendars.
CGILIB Collection of C language functions to ease script authoring across the WASD, OSU, Apache, Purveyor, and vanilla CGI environments (used by these scripts for portability).
CGIutl A CLI utility to assist at the DCL level with generating HTTP responses, returning content and with the handling of POSTed requests. Intended to be used within DCL procedures to manipulate the CGI environment.
Conan Access VMS Help and text libraries.
HyperReader
HyperShelf
Bookreader emulator (BNU and Bookreader format).
Hypershelf is used to navigate BNU or Bookreader shelves.
HyperSPI System performance monitor (of sorts!)
Query
Extract
Plain and HTML text file search utility.
Processes hits from the query script.
QDLogStats Quick & Dirty access LOG STATisticS


Installation


Unbundled from the WASD VMS Hypertext Services package for use by VMS Apache (ported using CSWS V1.3/Apache 1.3.26).  Requires logical APACHE$CGI_MODE to be set to 0 (default) or 1.  Logical APACHE$PREFIX_DCL_CGI_SYMBOLS_WWW can be optionally defined or not.

Installation steps:

  1. compile+link executables ... $ @BUILD   or (after UNZIPing optional object modules)
    link-only executables ... $ @BUILD LINK
  2. copy executables and support procedures ... $ @INSTALL APACHE
  3. required server configuration examples for the various components may be found in the wasd_scripts_csws.conf (CSWS 1.2 and 1.3) or wasd_scripts_sws.conf (SWS 2.0) files
  4. HyperSPI has additional requirements (see HYPERSPIREADME.TXT in this same directory)


Script Overviews


Required server configuration examples for the various components may be found in the apache.conf file.

Calendar Simple script to produce calendars.

Access this year's calendar with

  /cgi-bin/calendar?0

and next year's with

  /cgi-bin/calendar?+1
CGIutl No usage examples are provided with this kit.
Demonstrations of form decoding and file upload are available on the WASD demonstration site.
  http://wasd.vsm.com.au/ht_root/src/misc/
A comprehensive description of CGIutl functionality is available in the source code.
  cgiutl.c
Conan Access VMS help with:
  /cgi-bin/conan

Other demonstrations are available:

  runtime/conanhelp.html
HyperReader
HyperShelf
Mapping entries will be required for ALL paths to books, shelves, etc. The following two rules are useful for BNU-style ODL CDs (V7.n, thanks to Jeff Goodwin of Fairchild Semiconductor):
  AliasMatch ^/disk\$axpdoc(.+) /disk$axpdoc$1
  AliasMatch ^/disk\$vaxdoc(.+) /disk$vaxdoc$1

To access the default DECW$BOOKSHELF library:

  /cgi-bin/hypershelf

To access a BNU library specify it's location:

  /cgi-bin/hypershelf/sys$common/decw$book/library.decw$bookshelf
  /cgi-bin/hypershelf/sys$common/decw$defaults/user/library.odl

To directly access a book:

  /cgi-bin/hypershelf/sys$common/decw$book/bookreader.decw$book
HyperSPI See the HYPERSPIREADME.TXT in this same directory.

The [.RUNTIME]HYPERSPIEXAMPLE.GIF image gives some idea of what the script-processed data looks like via a browser (before you go to all the trouble of setting it up ;^)

Query
Extract
Both plain-text and HTML-text files can be searched, the latter only has the content of the file searched, not any tag-related text! "Text" files are recognised by the file extension.  A useful collection of these are predefined by the scripts.  To change the contents of this list use the /TEXT= and /HTML= qualifiers as described in the script prologues.  Additional plain-text files also must be specified to the EXTRACT script.

Examples, the first for generating a simple form-based search interface, the second for initiating a simple search:

  /cgi-bin/query/wasd/runtime/*.*
  /cgi-bin/query/wasd/runtime/*.*?wasd

It also supports form-initiated searches.  See the QUERY.C prologue.  Depending on server support it may be possible to make this script the default query (ISINDEX-style) engine.

QDLogStats Quick & Dirty LOG STATisticS provides very elementary statistics from Web server common/combined format log files.  A number of filters allow subsets of the log contents to be selected.  These filters can use the familiar asterisk wildcard or comprise simple or complex regular expressions.  This utility has a command-line and a CGI interface.
  /cgi-bin/qdlogstats

The CGI interface requires authorization to be active before it can be used.  Further setup and usage information is provided in the code description.

  qdlogstats.c



Mark G. Daniel
Mark.Daniel@vsm.com.au
Mark.Daniel@dsto.defence.gov.au

Demonstration & Download
http://wasd.vsm.com.au/
http://wasd.vsm.com.au/wasd/
ftp://ftp.vsm.com.au/wasd/index.html sary. The following example DCL executed during server startup would ensure such access. $ INSTALL ADD /PRIVILEGE=READALL CGI-BIN:[000000]QDLOGSTATS.EXE For WASD 8.1 and later access control should not need to be changed. For non-WASD and WASD earlier than 8.1 the following should be applied. If this is provided it would also be prudent to control access to the executable from arbitrary accounts at the CLI, and restricting it to the scripting account using an ACL. For example $ SET SECURITY CGI-BIN:[000000]QDLOGSTATS.EXE - /ACL=((IDENT=HTTP$SERVER,ACCESS=R+E),(IDENT=*,ACCESS=NONE) The CGI version REQUIRES the following system-level logical name to locate the log files for it. If this name does not exist the CGI interface will not work. All log files must be available via this logical. The logical name is defined /SYSTEM to limit a possibly READALL privilege being used against arbitrary paths. $ DEFINE /SYSTEM QDLOGSTATS_LOGS "HT_LOGS:" QUALIFIERS ---------- /ALL compare all records to all filters /AUTHUSER= pattern (any authenticated username) /BEFORE= files (not records!) modified before VMS time /CLIENT= pattern (client host name or IP address) /DATETIME= pattern ("11/Jun/1999:14:08:49 +0930") /DBUG turns on all "if (Debug)" statements /DECODE[=keyword] URL-decode PATH, QUERY and/or REFERER before filtering /IP= 4 or 6 to select either IPv4 or IPv6 records /LOG show the name of the log file the records come from /LOOKUP[=] lookup host name from literal address /METHOD= HTTP method ("GET", "POST", "HEAD", etc.) /OUTPUT= file specification /PATH= pattern (URL path component only) /PROGRESS[=integer] show progress during processing /PROTOCOL= HTTP protocol ("HTTP/1.0" or "HTTP/1.1") as a string /QUERY= pattern (URL query component only) /REFERER= pattern (HTTP "Referer:" field, COMBINED only) /REMOTEID= pattern (anything in the RFC819 remote ident field) /RESPONSE= HTTP response status code pattern /SINCE= files (not records!) modified since VMS time /SIZE=([MIN=|MAX=]) minimum and/or maximum response size /SOFTWAREID (and /VERSION) display QDLOGSTATS and CGILIB versions /USERAGENT= pattern (HTTP "User-Agent:", COMBINED only) /VIEW[=type] display matching records, where can be ALL, MATCH (default) or NOMATCH BUILD DETAILS ------------- See BUILD_QDLOGSTATS.COM procedure. COPYRIGHT --------- Copyright (C) 2000-2005 Mark G.Daniel This program, comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under the conditions of the GNU GENERAL PUBLIC LICENSE, version 2. Copyright (C) 1993 Free Software Foundation, Inc. Extended regular expression matching and search library, version 0.12. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. VERSION HISTORY (update SOFTWAREVN as well!) --------------- 13-APR-2005 MGD v1.10.1, bugfix; missing assignments in GetParameters() 11-MAY-2004 MGD v1.10.0, IPv6 modifications /IP= to select either IPv4 or IPv6 records 23-DEC-2003 MGD v1.9.2, minor conditional mods to support IA64 02-DEC-2003 MGD v1.9.1, 206 partial content 12-AUG-2003 MGD v1.9.0, HTTP protocol filter 17-MAY-2003 MGD v1.8.0, use regular expression matching (GNU RX1.5 source) (this fundamentally changes some previous behaviours) allow for protocol in URL ("method path protocol") 02-JAN-2003 MGD v1.7.0, filtering on HTTP response status and response size 24-NOV-2002 MGD v1.6.0, URL-decoding path/query/referer before filtering, check for account SYSPRV before CLI activating 05-NOV-2002 MGD v1.5.1, extend CGI log search to check for QDLOGSTATS_LOGS then to fallback to HTTPD$LOG and HT_LOGS logicals 26-SEP-2002 MGD v1.5.0, make CGI log search dependent on the QDLOGSTATS_LOGS logical name being defined and log files within that 16-AUG-2002 MGD v1.4.0, add log file last modification before/since, some accomodations for CSWS 1.2, bugfix; silly boy - what about HEAD method! 30-MAR-2002 MGD v1.3.0, accumulate request methods 01-JUL-2001 MGD v1.2.2, add 'SkipParameters' for direct OSU support 30-JAN-2001 MGD v1.2.1, small improvement to lookup cache for noting addresses that never resolve (and do it slowly) 10-JAN-2001 MGD v1.2.0, lookup client host name 23-DEC-2000 MGD v1.1.0, CGI interface (still pretty q&d) 14-NOV-2000 MGD v1.0.0, initial development */ #endif /* COMMENTS_WITH_COMMENTS */ /*****************************************************************************/ #define SOFTWAREVN "1.10.1" #define SOFTWARENM "QDLOGSTATS" #ifdef __ALPHA # define SOFTWAREID SOFTWARENM " AXP-" SOFTWAREVN #endif #ifdef __ia64 # define SOFTWAREID SOFTWARENM " IA64-" SOFTWAREVN #endif #ifdef __VAX # define SOFTWAREID SOFTWARENM " VAX-" SOFTWAREVN #endif /* standard C header files */ #include #include #include #include #include #include #include #include #include #include #include /* VMS-related header files */ #include #include #include #include #include #include #include #include #ifdef __VAX /* "Compaq C V6.4-005 on OpenVMS VAX V7.3" seems to need this */ # define SYS$GETMSG sys$getmsg int sys$getmsg (__unknown_params); #endif #include #include /* TCP/IP-related header files */ #include #include #include /* application related header files */ #include #include /* regular expression processing */ #define REGEX_CHAR '^' /* compile flags, case-insensitive extended GREP */ #define REGEX_C_FLAGS (REG_EXTENDED | REG_ICASE) /* execution flags */ #define REGEX_E_FLAGS (0) /* one for each of the possible unique FilterThisOut()s */ #define REGEX_PATTERN_MAX 15 #define BOOL int #define true 1 #define false 0 typedef struct _IO_SB { unsigned short Status; unsigned short Count; unsigned long Unused; } IO_SB; #define VMSok(x) ((x) & STS$M_SUCCESS) #define VMSnok(x) (!((x) & STS$M_SUCCESS)) #define FI_LI "QDLOGSTATS",__LINE__ /* used in modulus for every so many records progress */ #define PROGRESS_RECORDS 1000 /* wrap every so many characters */ #define PROGRESS_WRAP_CLI 80 #define PROGRESS_WRAP_CGI 60 /* maximum log line/record length */ #define MAX_LINE_LENGTH 2048 #define DEFAULT_LOGS_LOGICAL "QDLOGSTATS_LOGS" #define DEFAULT_LOGS2_LOGICAL "HTTPD$LOG" #define DEFAULT_LOGS3_LOGICAL "HT_LOGS" #define DEFAULT_LOGS_FILESPEC "*.LOG*;" #define DEFAULT_LOGS_DIR_APACHE "APACHE$SPECIFIC:[LOGS]" #define DEFAULT_LOGS_APACHE "*ACCESS_LOG*.;" #define DEFAULT_LOGS_DIR_OSU "WWW_ROOT:[SYSTEM]" #define DEFAULT_LOGS_OSU "ACCESS*.LOG;" #define DEFAULT_LOGS_DIR_WASD "HT_LOGS:" #define DEFAULT_LOGS_WASD "*ACCESS*.LOG*;" #define VIEW_NONE 0 #define VIEW_LOG 1 #define VIEW_ALL 2 #define VIEW_MATCH 3 #define VIEW_NOMATCH 4 char CopyrightInfo [] = "Copyr