The HTTPd executes permanently on the server host, listening for client connection requests on TCP/IP port 80 (by default). It provides concurrent services for a (technically) unlimitted number of clients (constrained only by the server resources). When a client connects the server performs the following tasks:
For I/O intensive activities like file transfer and directory listing, the
AST-driven code provides an efficient, multi-threaded environment for the
concurrent serving of multiple clients.
2.2 - Multi-Threaded
The WASD HTTPd is written to exploit VMS operating system characteristics allowing the straight-forward implementation of event-driven, multi-threaded code. Asynchronous System Traps (ASTs), or software interrupts, at the conclusion of an I/O (or other) event allow functions to be activated to post-process the event. The event traps are automatically queued on a FIFO basis, allowing a series of events to be sequentially processed. When not responding to an event the process is quiescent, or otherwise occupied, effectively interleaving I/O and processing, and allowing a sophisticated client multi-threading.
Multi-threaded code is inherently more complex than single-threaded code, and there are issues involved in the synchronization of some activities in such an environment. Fortunately VMS handles many of these issues internally. After connection acceptance, all of the processing done within the server is at USER mode AST delivery level, and for all intents and purposes the processing done therein is atomic, implicitly handling its own synchronization issues.
The HTTPd is written to make longer duration activities, such as the transfer of a file's contents, event-driven. Other, shorter duration activites, such as accepting a client connection request, are handled synchronously.
It is worth noting that with asynchronous, and AST-driven output, the data
being written must be guaranteed to exist without modification for the
duration of the write (indicated by completion AST delivery). This means data
written must be static or in buffers that persist with the thread.
Function-local (automatic) storage cannot be used. The server allocates
dynamic storage for general (e.g. output buffering) or specific (e.g. response
headers) uses.
2.3 - ASTs
With server functions having AST capability, in particular $QIO, the server is designed to rely on the AST routine to report any error, including both those that occur during the IO operation and any that occur when initiating the IO (which would normally prevent it being queued) even if that requires directly setting the IO status block with the offending status and explicitly declaring the AST. This eliminates any ambiguity about under what conditions ASTs are delivered ... ASTs are always delivered.
If a call to a server function with AST capability does not supply an AST
routine then it must check the return status to determine whether it can
continue processing. If it supplies an AST routine address then it
must not act on any error status returned, it must allow the AST routine
to process according to the IO status block status.
2.4 - Tasks
Each request can have one or more tasks executed sequentially to fullfil the request. This occurs most obviously with Server-Side Includes (SSI, the HTML pre-processor) but also, to a more limited extent, with directory listing and its read-me file inclusion. A task is more-or-less defined as one of:
Some tasks can only be called once per request. For example, image mapping, file transfer using cache, file upload, menu interpretation.
Other tasks have the possibility of being called within other tasks or multiple times serially during a request. An example is the transfer file task (non-cache), which can be used within directory listings to insert read-me files, and when <!--#includeing multiple files within an SSI document.
Two tasks, the directory listing and SSI interpretation tasks, can be
called multiple times and can also have concurrent instances running. For
example, an SSI file can <!--#include another SSI file,
nesting the SSI execution. The same SSI document can have an embedded
directory listing that contains an SSI read-me file with another directory
listing. Can get quite convoluted! The tasks are inplemented using a
linked-list FILO stack allowing this nesting. SSI documents have a maximum
depth for nesting, preventing recursive document inclusion.
2.5 - Memory Management
Memory management is exclusively done using VMS system library virtual memory routines. Using these rather that generic C library routines is a deliberate design decision, and done with the following considerations.
Per-request memory is managed in three distinct portions.
When a dynamic structure is required during request processing it is allocated from a request-thread-specific zone of virtual memory. This list is released in one operation at thread disposal, making it quite efficient. Maintaining a thread-specific heap of vritual memory also makes it easier to avoid memory leakage.
These structures are used to store task-specific data. If a task is used multiple times within the one request (see above) the previous allocated and now finished-with (but not deallocated) task structures can be reused, reducing overhead.
To reduce the number of individual network writes, and thus provide significant improvements in efficiency, generated output can be buffered into larger packets before sending to the client. Not all modules use this (e.g. File.c) and not all modules use it all of the time, but all modules work to implement a seamless integration of output via this mechanism (best seen in the SSI.c module).
The output buffer functionality underwent a complete redesign for v5.0. It is now based on a list of one or more buffers that can be used in two modes.
The first mode is used for general buffering (e.g. SSI and directory
listings), streaming data to the client in a sequence of larger aggregates.
The second mode is useful for functions that must block (e.g. those reporting
on data structures such as the file cache), write a lot of output for a
report, and not want to block general server activity for a
long-ish period due to network throughput (e.g. again the caching
reports). In these cases the entire report can be written to buffer, then
simply asynchronously output, unblocking any resource it may have held.
2.7 - Rule-Mapping
A fundamental aspect of any HTTPd implementation is the rule mapping used
to create a logical structure for the Web file system. The HTTPd mapping
function is designed to be flexible enough that script programs can also use
it. As a result it is text-file based, and opened and read when mapping. This
method of mapping provides a good deal of flexibility, coupled with acceptable
performance. The function has received a high level of attention in an effort
to optimize it.
2.8 - Auto-Scripting
The WASD VMS HTTP server has the facility to automatically invoke a script
to process a non-HTML document (file). This facility is based on detecting
the MIME content data type (via the file's extension) and causing a
transparent, local redirection, invoking the script as if it was specified in
the original request.
2.9 - Internal Directives and "Scripts"
The HTTPd server detects certain paths and query strings as directives about its behaviour. Certain paths are interpreted as pseudo, or internal scripts, handled internal to the server. Other directives are passed in the query string component of the request, and as reserved sequences cannot occur in normal requests (an unlikely combination of characters has been selected).