Copyright © 2007,2008 Mark G. Daniel
This program, comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it under the
conditions of the GNU GENERAL PUBLIC LICENSE, version 3, or any later version.
http://www.gnu.org/licenses/gpl.txt
PyRTE is an interface to a Python interpreter engine and environment for the WASD VMS Web server. It is designed to be able to be used in standard CGI and CGIplus/RTE persistent scripting modes. The persistent modes (default) provide a ~35x (yes, an approximate thirty-five times) improvement in script activation times (compared to CGI/command-line) and comparably reduced load on both server and system. Note that this package does not contain the Python interpreter or kit, that has to be obtained and installed separately (a simple matter as it is provided via a PCSI package).
PyRTE is linked against the Python interpreter shareable image and so shares a set of Python capabilities in common with command-line OpenVMS Python. Alpha (AXP) and Itanium object modules are provided for the VMS platforms available for Python.
PyRTE has been developed and tested on VMS V8.3 using the WASD v9.2.0 and
VMS Python kit JFP AXPVMS PYTHON251 available from and documented at http://vmspython.dyndns.org/. JFP's
Python kits are kept very up-to-date as Python updates or implementation
problems arise.
Installation
http://vmspython.dyndns.org/and install according to the instructions available.
$ RENAME HT_ROOT:[SRC]PYTHON.DIR HT_ROOT:[SRC]PYTHON_nnn.DIR
http://wasd.vsm.com.au/wasd/
$ SET DEFAULT HT_ROOT:[000000] $ UNZIP "-V" location:PYRTEnnn.ZIP
$ SET DEFAULT HT_ROOT:[SRC.PYTHON] $ @BUILD_PYRTE LINK
$ COPY HT_EXE:PYRTE.EXE CGI_EXE:
Although there are experimentation modes built into PyRTE basically it will be used as a WASD Run-Time Environment providing persisten Python interpreter(s) and environment(s) for Python scripts and applications. Benchmarking indicates the CGIplus/RTE use reduces activation time to 3% of CGI (yes, that's correct, by a factor 35 - a python interpreter is quite expensive to instantiate). There are subtle differences in the way CGIplus and RTE parse and provide the PATH_INFO data (see the "WASD Scripting Overview" for detail).
# HTTPD$CONFIG [DclScriptRunTime] .PY $CGI-BIN:[000000]PYRTE [AddType] .PY text/plain Python source .PYC application/octet-stream Python byte-code .PYO application/octet-stream Python optimised byte-code
# HTTPD$MAP
exec /py-bin/* (cgi_exe:pyrte)/cgi-bin/* \
script=syntax=unix script=query=none map=once ods=5
# HTTPD$MAP for automatic RTE usage
map /cgi-bin/*.py* /py-bin/*.py*
exec /py-bin/* (cgi_exe:pyrte)/cgi-bin/* \
script=syntax=unix script=query=none map=once ods=5
# HTTPD$MAP
exec /web/**.py* /web/*.py* \
script=syntax=unix script=query=none map=once ods=5
# HTTPD$MAP
exec /appx/**.py* /appx_root/*.py* \
script=syntax=unix script=query=none map=once ods=5 \
script=as=USERX
# HTTPD$MAP
exec /py-bin/* (cgi_exe:pyrte)/ht_root/src/python/scripts/* \
script=syntax=unix script=query=none map=once ods=5
Note that the location of the above example scripts has changed to [.SCRIPTS]
since the v1.0 release.
Remember to
$ HTTPD /DO=MAPafter changing mapping rules.
After configuration (as described immediately above) the following scripts may be used to confirm the environment is functioning.
Some of these examples also support CGIplus mode for comparison purposes.
Of course the Python code in the script may be inspected from the source directory:
/ht_root/src/python/scripts/
PyRTE contains a Web Server Gateway Interface v1.0 (WSGI). This is a Python standard specification for web servers and application servers to communicate with web applications (though it can also be used for more than that).
The WSGI code has been developed under the sponsorship of SysGroup
http://www.sysgroup.fr/and generously made available to the wider WASD community.
The following is an example of a simple (and classic) WSGI application being activated using the PyRTE wasd.wsgi_run() function.
def hello_world (environ, start_response):
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!\n']
import wasd
wasd.wsgi_run(hello_world)
PEP 333 specifies that WSGI should not buffer output and pyRTE complies but some tools (like Mercurial) don't buffer internally themselves and end up sending many very small records. This is obviously inefficient. If buffered the throughput boost for Mercurial is about x4. Well worth having for specific Web applications.
This forced buffering may be enabled on a per-application basis using a mapping rule, for example
set /mercurial/* script=param=PYRTE=/WSGI=BUFFERor by defining the logical name
$ DEFINE /JOB PYRTE_WSGI_FORCE_BUFFERING TRUEin a wrapper procedure.
The following scripts may be used to confirm the environment is functioning.
The Python code in the script may be inspected from the source directory:
/ht_root/src/python/scripts/
There are also equivalent wsgiref (the Python reference
implementation) scripts available for comparison. Just add ref to the
wsgi in the script name.
Leveraging PyRTE
Python scripts activated using the default persistent engine generally have low-latency, low-system-impact characteristics and seem to perform reliably and efficiently. The PyRTE generally handles all the WASD server interaction requirements. However there may be occasions where the application itself benefits from being persistent (many Python Web application are written with this in mind) or where more control needs to be exercised by the application designer. The WASD PyRTE provides a WASD Python module that provides an API for some of these aspects. Rather than clutter this document with such arcane detail the description may be found in the prologue to the PyRTE source code:
/ht_root/src/python/pyrte.c
An example of where an application benefits from an explicitly CGIplus activation is MoinMoin:
MoinMoin is generally run independently to a system's primary Web server using the embedded Web server available for Python. Reverse proxy is often then use to provide and control access through to the persistent MoinMoin server.
Using the same principle of keeping MoinMoin persistent (for the obvious latency and efficiency benefits) it can be run and controlled directly by the WASD Web server using a CGIplus wrapper DCL procedure and a little Python glue (directly derived from persistence code already present in the MoinMoin package).
/ht_root/src/python/moincgiplus.com
The same principles may be applied to any application environment (third-party or in-house) with similar characteristics. Python applications running as CGIplus scripts using the PyRTE must be activated via DCL wrappers as illustrated above (in part due to mapping requirements).
With CGIplus persistent applications configuration or code changes will in all probability require a 'restart' of the application using one or other of:
$ HTTPD/DO=PURGE $ HTTPD/DO=DELETE
Unfortunately the author of the PyRTE interface is such a Python novice he is not in any position to answer queries about Python "programming" or usage. There is the VMS Python Wiki
http://vmspython.dyndns.org/
and support forum
http://vmspython.dyndns.org/piforum/index.py
for VMS Python issues. If there's an obvious behavioural problem with
PyRTE then contact the author or use the info-WASD mailing list.
Acknowlegements
Many thanks to Jean-François Piéronne for his initial port of Python to VMS
and his energetic, ongoing maintenance of the product.