Perl is a powerful language and should be used to greater advantage. Unfortunately in a CGI context the overheads of starting Perl (particularly under VMS) can severly impact request response time and system overhead.
WASD's persistant scripting (CGIplus) and persistant Run-Time Environment
(RTE) both improve script response time and reduce system impact by allowing
scripting resources to be loaded into a process just once before processing
multiple requests. For general information on Perl and WASD, and for detailed
information on CGIplus and RTE see
Other general information on WASD and Perl may also be found in the above document.
Developed and tested against (and probably only suitable for) Perl v5.8.0 and v5.6.1.
This kit is designed for the WASD 8.1 (or later) environment and so
requires that the example scripts be installed into the server's scripting file
space before use. The procedure INSTALL.COM can
assist with this.
CGI, CGIplus, RTE ... which to choose?
With multiple WASD scripting technologies available choosing which is best suited to a given application can somtimes seem daunting. This table attempts to summarize the issues.
Technology Description CGI Standard CGI is just that, the lowest common denominator between server environments. Nothing persists (except perhaps the process). The script is activated with each request, receiving request parameters via CGI environment variables. Processing occurs, with response output. The script runs down and the process becomes quiescent or exits. Perl engine activation, script source parsing (and any required modules), script activation, execution and rundown are all fairly expensive undertakings. Eliminating one or more of these improves latency and reduces system resource consumption. CGIplus CGIplus allows the script itself to persist. This means Perl engine activation, script source parsing and script resource instantiation occurs only once for any number of successive requests. This is particularly significant with highly latent resources, such as databases, network connections, etc. CGIplus scripts must be designed and implemented to take persistence into account. Care must be taken to ensure that there is no leakage of data between requests and that resources used and no longer required are released - script rundown can no longer be used to perform these tasks! It is generally a simple undertaking to make a script able to behave in a CGI or CGIplus manner depending on how it is activated. The Perl module CGIplus.pm is available to perform the few extra steps required for CGIplus. RTE A Run-Time Environment allows the scripting engine to persist. For interpreters such as Perl this can significantly reduce engine activation latency and resource consumption. The script itself does not persist and so all state is lost between activations. In the case of the Perl RTE engine the parsed code for that script is cached and will be used again when the script is next activated. Eliminating this step on subsequent activations provides a significant saving in latency and system resources. Generally any plain old CGI script can be used with the RTE, no specific changes are required for this environment. It can be a little fussy about namespace. All variables must have an explicit package reference or be "my"ed.
More information on these technologies is available in the WASD
Scripting Overview.
Hints and Kinks
A few suggestions for when using Perl with WASD.
CGI Variables
The Perl (and PerlRTE) interpreter needs to be informed about the source of it's environment variables. In WASD's case the CGI variables are stored as DCL symbols. This requires the definition of a logical name, which can be on a system-wide basis (which of course affects all Perl scripting).
$ DEFINE /SYSTEM PERL_ENV_TABLES CLISYM_GLOBAL,LNM$PROCESS
Alternatively, it may be applied only to the scripting environment through the use of a Perl "wrapper" procedure. The example PERL.COM shows how this can be done.
Note that the next comment only applies to standard Perl CGI. It does not apply when using the PerlRTE or the CGIplus.pm module described below. With these internal processing performs all such munging transparently.
Many, perhaps most, open-source Perl CGI scripts will be using CGI variable names without the common VMS (and WASD-default) prefix of "WWW_". These will not function correctly using $ENV{} to retrieve values with the standard Perl interpreter. To remove the "WWW_" prefix add an appropriate rule to HTTPD$MAP.
set /cgi*-bin/*.pl CGIprefix=
POSTed Requests
CGI.pm and Perl v5.6.0 could not read a POSTed multipart stream satisfactorily producing the error "CGI.pm: Server closed socket during multipart read (client aborted?)". This is apparently a known problem fixed by migrating to Perl 5.6.1 (or later) and it's CGI.pm.
When not using the PerlRTE engine some redirection is necessary to get the correct stream to connect to Perl's <STDIN>. It controls the script process via SYS$INPUT and supplies any POSTed body via the separate stream HTTP$INPUT requiring redirection before Perl engine activation.
$ define /user perl_env_tables clisym_global,lnm$process $ define /user sys$input http$input $ perl device:[directory]script.pl
This approach can be seen in use with PerlRTEexample4.pl and PerlRTEexample4.com below. Although PerlRTE does not actually require this because the two scripts can be used in both environments (to illustrate differences in latency) it is used in both.
There are also often issues in getting POSTed content to the script without
VMS' record boundary munging interfering. This generally requires the
<STDIN> stream being placed into binary mode, sometimes not trivial in
standard VMS Perl. The PerlRTE environment can be used as if it was the
standard Perl verb eminiating this requirement. See the comments in the
PERLRTE.C prologue. This is done with the standard CGI example of
PerlRTE_example5 (look at PerlRTE_example5.com).
Binary Streams
By default VMS I/O streams are record-oriented and RMS has a tendency to want to adjust carriage-control on these. This is often counter-productive when needing to return a binary (non-textual) response.
The VMS::stdio Perl extension allows I/O streams to be very finely
controlled using low-level C-RTL and RMS functionality. See the appropriate
Perl documentation page. The CGIplus.pm module described below uses this
extensively.
Perl RTE
The Perl Run-Time Environment provides a persistent Perl engine, caching both the running Perl interpreter and any one or more Perl scripts activated and parsed by that interpreter.
The Perl Run-Time Environment should execute any standard Perl script using any collection of modules. This approach uses standard techniques and code described in the "perlembed" document to load and keep cached multiple script and module sources. How isolated are these scripts in reality? The document indicates effectively enough! Each is treated as an autonomous package and so storage restrictions etc. need to be observed. However apart from that it would seem as if any old (perhaps slightly tweaked) CGI script could be used within this environment.
The embedding code maintains the last modification time of each script cached and checks this against the last modification time of the script file before each activiation. If there is a difference in the two times (i.e. the file has changed in some way) the cache is overwritten with a fresh evaluation of the script. There is no need to explicitly flush this cache in any way.
The CGI environment variables are available via the ENV associative array.
There are a number of ways to customize the startup of this environment. See
the PerlRTE.c and PerlRTeng.c
sources for detailed information.
Demonstrations
These examples and demonstrations rely on the following configuration.
# HTTPD$CONFIG [DclScriptRunTime] .PL PERL # or perhaps (depending on the local system) # .PL $PERL_ROOT:[000000]PERL.EXE # HTTPD$MAP exec+ /plrte/* (CGI-BIN:[000000]PERLRTE.EXE)/cgi-bin/*
Note the differences in latency between standard CGI and the RTE!
Script Sources: |
PerlRTE_example1.pl
PerlRTE_example2.pl PerlRTE_example3.pl (loads the CGI.pm module for extra latency) PerlRTE_example4.pl (it's standard CGI POST wrapper PerlRTE_example4.com) PerlRTE_example5.pl (it's standard CGI POST wrapper PerlRTE_example5.com) |
---|---|
* Standard CGI: |
/cgi-bin/PerlRTE_example2 /cgi-bin/PerlRTE_example3 |
* If the CGI variables appear to be empty make adjustments in line with CGI Variables. | |
Persistent RTE: |
/plrte/PerlRTE_example2 [restart RTE] /plrte/PerlRTE_example3 [restart RTE] |
20x Performance
Benchmarking the PerlRTE_example3.pl script (which requires CGI.PM)
using ApacheBench on the author's
development system indicated 1.7 and 32 requests/second for the CGI and RTE
usages respectively. Expressed another way, the RTE usage responded
20 times faster!! Remember this is the same script - no changes
apart from invocation path.
Scripting Engine
The Perl RTE is suitable for use as the main Perl scripting engine. That is, it can activate standard CGI, CGIplus and RTE scripts. To enable this for the execution of all CGI Perl scripts by the server add or change the following HTTPD$MAP rules and reload.
if (!script-name:/plrte/*) map /cgi-bin/*.pl* /plrte/*.pl* exec+ /plrte/* (CGI-BIN:[000000]PERLRTE.EXE)/cgi-bin/*
Also see mapping caution in CGIplus.pm.
Building PerlRTE
As there is some minor variation in socket support between Perl 5.8 and 5.6 the build procedure needs to be directed which environment to support. The default is 5.8.
Compile*+ Link for Perl 5.8
$ SET DEFAULT HT_ROOT:[SRC.PERL] $ @BUILD_PERLRTE BUILD 5.8Link-only for Perl 5.8
$ SET DEFAULT HT_ROOT:[SRC.PERL] $ @BUILD_PERLRTE LINK 5.8Compile*+ Link for Perl 5.6
$ SET DEFAULT HT_ROOT:[SRC.PERL] $ @BUILD_PERLRTE BUILD 5.6Link-only for Perl 5.6
$ SET DEFAULT HT_ROOT:[SRC.PERL] $ @BUILD_PERLRTE LINK 5.6* To compile the RTE sources the Perl distribution header files must available from PERL_ROOT:[000000].
As this release is designed for the post WASD 8.1 environment it is then necessary to install the Perl RTE interpreter into the server scripting file space.
$ @INSTALL
CGIplus Perl
CGIplus Perl is an alternative, perhaps complimentary method to Perl RTE for increasing efficiency and reducing request latency, again using the idea of persistence. This time it is up to the script to maintain the state.
These examples also demonstrate the use of the VMS::DCLsym and VMS::Stdio extensions. The principles may be more generally applied to other scripts.
Script Source: CGIplusPM_example1.pl Standard CGI: /cgi-bin/CGIplusPM_example1/ht_root/src/perl/?this+is+a+query+string CGIplus: /cgiplus-bin/CGIplusPM_example1/ht_root/src/perl/?this+is+a+query+string [restart]
30x Performance
Benchmarking this using
ApacheBench on the author's development
system indicated 0.8 and 25 requests/second for the CGI and CGIplus usages
respectively. Expressed another way, the CGIplus usage responded
approximately 30 times faster! Remember this is the same script - no
changes apart from invocation path.
CGIplus.pm
The CGIplus.pm Perl module is intended to assist authors write scripts that may be used transparently in both vanilla CGI environments (including non-WASD) as well as under WASD CGIplus. Perl implementation refinements courtesy of Dick Munroe (munroe@csworks.com).
A script using CGIplus.pm (or any other autonomous CGIplus behaviour) should never be activated using an RTE path; that is, one using the following mapping syntax.
exec+ ($cgi-bin:[000000]perlrte.exe)/plrte/* /cgi-bin/*
When an RTE becomes quiescent the server will give it another script. With the CGIplus.pm request processing loop is active servicing the one script. This unintended and probably incorrect script will become active. Always activate CGIplus.pm enabled scripts via a CGIplus path.
exec+ /plplus/* /cgi-bin/*
CGIplus.pm will detect such a mapping mistake and die!
Binary Responses
VMS' RMS complicates output streams under Perl. This is a particular issue with CGIplus end-of-file sentinals, which must be output as a single record. CGIplus.pm attempts to provide a simple mechanism for providing binary streams if necessary, while still ensuring it's own records are not interfered with. This uses Charles Bailey's VMS::Stdio extension module built into most versions of VMS Perl.
This script demonstrates how the module's stream functions can be directly used.
Script Source: CGIplusPM_example2.pl Standard CGI: /cgi-bin/CGIplusPM_example2 CGIplus: /cgiplus-bin/CGIplusPM_example2 [restart]
This script demonstrates how simply to return a binary file as a response.
Script Source: CGIplusPM_example3.pl Standard CGI: /cgi-bin/CGIplusPM_example3 CGIplus: /cgiplus-bin/CGIplusPM_example3 [restart]
Any accessable image location may be added to these scripts following the script part. Again, because these are being accessed via a CGIplus script, notice the difference in latency between the initial and subsequent requests.
Notes:
There have been a number of changes implemented in the CGIplus.pm supplied with PerlRTE 1.2. These should be backward compatible. The original module is supplied in the source directory if required.
Useful Example Script(s)
# Modularian provides a general interface for browsing the contents of # OpenVMS libraries of all types. It does not replace CGIs like # Conan and helpgate which serve different purposes.
modularian.pl (some excellent implementation notes in the script, module documentation)
Note that there are some dependencies not supplied with the script.
# It also depends on HTML::Entities and HTML::Table which are # available for download from www.cpan.org. Everything else should # be in your OpenVMS perl distribution.
Courtesy of Dick Munroe (munroe@csworks.com)
Check for more recent releases at
http://www.csworks.com/download/modularian.html