Brief Introduction to HTTPd Code

WASD Hypertext Services - Technical Overview

[next] [previous][contents]

14 - Brief Introduction to HTTPd Code

This section is designed to be only a broad overview of the basic functionality of the HTTPd server. It also does not cover the full suite of WASD VMS Hypertext Services software.

NOTE: Some code descriptions have become very brief in the transition from version 3 to version 4 of the HTTPd, during which some significant changes occured to data structures and processing. Also I have not been rigorous in my revision of the descriptions, so they may be misleading (though not grossly). Apologies! I'll blame time constraints, but more probably this is a rationalization of procrastination :^)

The source code should also be consulted.

14.1 - Multi-Threaded

The WASD HTTPd is written to exploit VMS operating system characteristics allowing the straight-forward implementation of multi-threaded code. The server is written to be I/O event driven. Asynchronous System Traps (ASTs), or software interrupts, at the conclusion of an I/O (or other) event allow functions to be activated to post-process the event. The event traps are automatically queued on a FIFO basis, allowing a series of events to be sequentially processed. When not responding to an event the process is quiescent, or otherwise occupied, effectively interleaving I/O and processing, and allowing a sophisticated client multi-threading.

Multi-threaded code is inherently more complex than single-threaded code, and there are issues involved in the synchronization of some activities in such an environment. Fortunately VMS handles many of these issues internally. After connection acceptance, all of the processing done within HTTPd is at USER mode AST delivery level, and for all intents and purposes the processing done therein is atomic, implictly handling its own synchronization issues.

HTTPd is written to make longer duration activities, such as the transfer of a file's contents, event-driven. Other, shorter duration activites, such as accepting a client connection request, are handled synchronously.

It is worth noting that with asynchronous, and AST-driven output, the data being written must be guaranteed to exist without modification for the duration of the write (until completion AST delivery). This means data written must be static or in buffers that persist with the thread. Function-local (automatic) storage cannot be used. The HTTPd server allocates dynamic storage for general (e.g. output buffering) or specific (e.g. response headers) uses.

14.2 - Tasks

Each request can have one or more tasks executed sequentially to fullfil the request. This occurs most obviously with the HTML pre-processor, but also, to a more limited extent, with directory listing and its read-me file inclusion. A task is more-or-less defined as one of:

Each one of these modules executes relatively independently. Before commencing a task, a next-task pointer can be set to the function required to execute at the conclusion of that task. At that conclusion, the next-task functionality checks for a specified task to start or continue. If it has been specified control is passed to that next-task function via an AST.

Some tasks have the possibility of being called multiple times during a request, others do not. An example of the former is the send file task, which is used when <!--#includeing multiple files within an SSI document. An example of a task that can only possibly be called once is the interpretation of an SSI document within a request.

14.3 - Memory Management

Per-Thread memory is managed in three distinct portions.

  1. A fixed-size structure of dynamic memory is used to contain the core request thread data. This is released at thread disposal.

  2. A heap of dynamically allocated memory is maintained during the life of a thread structure.

    When a dynamic structure is required this heap is expanded by calloc()ing memory, placing this in a double-linked list structure, and returning a pointer to the usable portion of the newly allocated memory. This list is released in one operation at thread disposal, by traversing the list and free()ing each individual chunk (making it easier to avoid the memory leaks associated with making autononmous allocations for each dynamic memory structure required).

  3. Per-task data structures are allocated using the above heap.

    These structures are used to store task-specific data. If a task is used multiple times within the one request (see above) the previous allocated and now finished-with (but not deallocated) task structures can be reused, reducing overhead.

14.4 - Output Buffering

To reduce the number of individual network writes, and thus provide significant improvements in efficiency, output generated from all modules except File.c and Dcl.c, is buffered into larger packets before sending to the client. All modules, including File.c, work to implement a seamless integration of output via this mechanism (best seen in the SSI.c module).

The AST-driven nature of the server means this functionality is moderately complex. A form of double buffering is employed, allowing the buffer to overflow and be flushed to the network asynchronously, without overwriting, losing or needing to re-request data. Two buffer spaces are employed. When one fills it is written to the network and a pointer to the two areas exchanged, allowing the supplied (and overflowed) data to be immediately buffered (without a synchronous wait for the network write to complete), and an immediate return to further AST-driven processing. The alternate buffer continues to be used until it fills, when the process is repeated.

The possibility of an asynchronous write with every buffered output introduces complexity. Every buffered output call must be used as if it is an asynchronous network write, an AST function address supplied with every call on the off-chance (and eventuality) that an actual network write will occur. If a network write does not occur (most of the time) the AST is explicitly declared for delivery. This need to supply an AST function with every buffered write, basically means only one buffered write may occur per AST-executed function.

14.5 - Rule-Mapping

A fundamental aspect of any HTTPd implementation is the rule mapping used to create a logical structure for the hypertext file system. The HTTPd mapping function is designed to be flexible enough that script programs can also use it. As a result it is text-file based, and opened and read when mapping. This method of mapping provides a good deal of flexibility, coupled with acceptable performance. The function has received a high level of attention in an effort to optimize it.

14.6 - Auto-Scripting

The WASD VMS HTTP server has the facility to automatically invoke a script to process a non-HTML document (file). This facility is based on detecting the MIME content data type (via the file's extension) and causing a transparent, local redirection, invoking the script as if it was specified in the original request.

14.7 - Internal Directives and ``Scripts''

The HTTPd server detects certain paths and query strings as directives about its behaviour. Certain paths are interpreted as pseudo, or internal scripts, handled internal to the server. Other directives are passed in the query string component of the request, and as reserved sequences cannot occur in normal requests (an unlikely combination of characters has been selected).

14.8 - HTTPd Modules

The HTTPd server comprises several main modules, implementing the obvious functionality of the server, and other, smaller, support modules.

14.8.1 - ADMIN.C

Code for: Admin.c

(INCOMPLETE AS YET!)

This module provides the on-line server administration menu and functionality. Some administration pages are provided by the Upd.c module, ``piggy-backed'' into normal editing dialogues.

14.8.2 - AUTH.C

Code for: Auth.c

(INCOMPLETE AS YET!)

Sorry, its fairly complex module so I'll have to plead being too busy!

HTAdmin.c module helps administer the authentication databases.

The authorization module handles user authentication and path method authorization for all requests received by the server. Authenticated username/password information is cached in a balanced binary tree, improving performance compared to on-disk checking each time a request is received.

HTTPd-specific authentication databases are binary files with fixed-length 512 bytes records. Within the record is provision for:

Server-host SYSUAF authentication is also provided.

14.8.3 - BASIC.C

Code for: Basic.c

This module provides authentication functionality for the BASIC method.

14.8.4 - DCL.C

Code for: DCL.c

(MAJOR REVISION FOR v4.2!)

The DCL execution functionality must interface and coordinate with an external subprocess. It too is asynchronously driven by I/O once the subprocess has been created and is executing independently. Communication with the subprocess (IPC) is via mailboxes.

Process creation by the VMS operating system is notoriously slow and expensive. This is an inescapable factor when scripting using child processes within the environment. An obvious strategy is to avoid, at least as much as possible, the creation of subprocesses. The only way to do this is to share subprocesses between multiple scripts/requests. The obvious complication becomes isolating the potential interactions due to changes made by any script to the subprocess' enviroment. For VMS these changes are basically symbol and logical name creation, and files opened at the DCL level. In reality few scripts need to make logical name changes and symbols are easily removed between uses. DCL-opened files are a little more problematic, but again, in reality most scripts doing file manipulation will be images.

The conclusion becomes that for almost all environments scripts can quite safely share subprocesses with great benefit to response latency and system impact. If the local environment requires absolute script isolation for some reason then this subprocess-persistance may easily be disabled with a consequent trade-off on performance.

The term zombie is used affectionately to describe these subprocesses when persisting between uses (the reason should be obvious, they are neither ``alive'' (processing a request) nor are they ``dead'' (deleted) :^)

The DCL facility is used in three modes:

  1. To execute independent DCL commands.
    This is used to provide DCL command output for SSI (pre-processed HTML). One new subprocess per DCL command is created. Multiple subprocesses may be created per request (and SSI document).

  2. To execute standard CGI scripts.
    One new subprocess per script is created. The subprocess terminates when the script execution and request completes.

  3. To execute CGIplus scripts.
    Subprocesses may exist over multiple requests. Technically the CGIplus script only executes once and then remains blocking until a request is provided to it. It then processes the request, provides output, then blocks again. See 9.6 - CGIplus Scripting.

The DCL module creates a data structure that allows subprocesses to be managed independently of any request. This allows both CGIplus and standard CGI subprocesses (in the form of zombies) to persist across multiple requests. There are a fixed number of subprocesses that can exist for all purposes. This is set by the subprocess hard-limit configuration parameter. Each of these structures is created and populated on an as-required basis, linked into a list growing from zero to maximum based on demand and the life of the server. Subprocesses can come and go depending on requirements. CGIplus script subprocesses and any zombies are semi-permanent. In summary:

The four mailboxes serve as the subprocess' IPC:

  1. SYS$COMMAND
    This stream controls the subprocess execution, providing DCL commands to the subprocess' CLI. For DCL commands and standard CGI it creates DCL symbols representing the CGI variables then the command or script is executed. For CGIplus the CGIPLUSIN stream provides CGI variables.

  2. SYS$OUTPUT
    The subprocess simply writes output to SYS$OUTPUT (<stdout>). Due to buffering in the C RTL binary-mode streams are more efficient and faster and record-mode. See any of the CGI applications in this package for example code for changing script <stdout> to binary-mode. CGIplus scripts must indicate the end of a single request's output using a special EOF string which is specifically detected by the output functions. As this mailbox persists between requests it is essential to ensure no output from a previous request lingers in the mailbox due to request concellation or abnormal subprocess termination.

  3. SYS$INPUT
    For CGI script execution, available for the subprocess access to the HTTP data stream as <stdin>. A synonym exists for backward compatibility, the logical name HTTP$INPUT, a stream which can be explicitly opened. Note that unlike some CGI environments which supply only the body, this supplies both header and body. It is an easy enough modification to skip the header by reading until the header-separating empty (blank) line is encountered.

  4. CGIPLUSIN
    For CGIplus this mailbox provides access to a request's CGI variables. The first line of any request can always be discarded (for synchronization) and end-of-request vriables is indicated by an empty record (blank line). For standard CGI and DCL commands this mailbox is not used.

DCL Module Processing

  1. The primary DCL function ensures any required file specification exists (e.g. script procedure). It the allocates a slot to the request. Slot allocation is a very fundamental activity:

    A function writes to the CGIPLUSIN, creating a number of logical names, CGI-compliant symbol names and executing the command or invoking the execution of a DCL procedure, etc., and supplies the CGIplus variable stream if appropriate. If the use of zombies is enabled then DCL to clean up the environment as much as possible is provided first.

  2. When the subprocess writes to the SYS$OUTPUT stream the I/O completion AST routine associated with reading that mailbox is called.

    If CGIplus script execution the I/O is always examined for the CGIplus end-of-output signature bits. It must always be at the start of the record and if detected the request is concluded.

    If CGI script execution, the first I/O from this stream is analyzed for CGI-compliance. It is determined whether a raw HTTP data stream will be supplied by the script, or whether the script will be CGI-compliant (requiring the addition of HTTP header, etc.) and whether HTTP carriage-control needs to be checked/added for each record.

    A CGI local redirection header (partial URL) is a special case. When this is received all output from the subprocess is suppressed until the script processing is ready to be concluded. At that time the ``Location:'' information of the header is used to reinitiate the request, using the same thread data structure.

    When normal SYS$OUTPUT processing is complete the record received can be handled in one of two ways. If it is raw HTTP it is asynchronously written to the network. The AST completion routine specified with the network write will queue another read from subprocess' SYS$OUTPUT. If it is record-oriented I/O (e.g. from DCL output), it has it's carriage-control checked for HTTP compliance before asynchronously writing the record. Hence a script supplying its own raw, HTTP-compliant data stream is much more efficient than line-by-line.

    The SYS$OUTPUT stream is a little problematic. For standard CGI and DCL command execution at subprocess exit there may be one or more records waiting in the mailbox for reading and subsequent writing to the client over the network, delaying processing conclusion. Detection of completion is accomplished by making each QIO sensitive to mailbox status via the SS$_NOWRITER status, which indicates there is no channel assigned to the mailbox, and the mailbox buffer is empty. It then becomes safe to dispose of the client thread without loss of data.

  3. If CGI-script execution is for a POST or PUT metthod, the HTTP data stream made available is also AST driven. If the subprocess opens the stream and reads from it, the I/O completion routine called queues another asynchronous read from the buffered request header and/or body.

14.8.5 - DESCR.C

Code for: Descr.c

(INCOMPLETE AS YET!)

The Descr.c module generates a file description by searching HTML files for the first occurance of <TITLE>...</TITLE> or <Hn>...</Hn> tags, using the description provided there-in. It is primarily used by the directory listing module, but can also be used by the menu module.

It does this search asynchronously (surprise-surprise!)

To asynchronously locate a description in an HTML file, the file is opened and then each record asynchronously read and examined for the <TITLE> element. Once obtained a synchronous call is made to a function to list the file details. After the file details are listed another asynchronous search call is made, with the file search function specified for AST completion. The function then immediately completes.

14.8.6 - DIGEST.C

Code for: Digest.c

This module provides authentication functionality for the DIGEST method.

This module uses code derived in part from RSA Data Security, Inc., under licence:

granted to make and use derivative works provided that such works are identified as ``derived from the RSA Data Security, Inc. MD5 Message-Digest Algorithm'' in all material mentioning or referencing the derived work.

14.8.7 - DIR.C

Code for: Dir.c

There is some fairly complex and convoluted behaviour in this code!

This module implements the HTTPd directory listing functionality. Directories are listed first, then files. File detail format customizable, with the default resembling the default CERN and NCSA server layout. Output from this module is buffered to reduce network writes, improving I/O efficiency. HTML files have the <TITLE></TITLE> element extracted as a ``Description'' item.

Essential behaviour:

  1. The primary function obtains the file specification from the request data structure. Server directives, controlling some features of the directory listing beahaviour, are checked for and parsed out if present. The directory listing layout is initialized. The directory specification (path information) is parsed to obtain the directory and file name/type components. After successfully parsing the specification it generates an HTTP response header if required.

  2. Column headings and (possibly) a parent directory item are buffered in an asynchronous function call. An RMS structure is initialized to allow the asynchronous search for all files in the specified directory ending in ".DIR".

  3. For each directory file found the directory search AST completion function is called. Status is checked for success or otherwise. If an error the status is reported to the client and the request processing concluded. If the directory contained no directory files, or the directory files are exhausted a call to a function to begin a listing of non-directory files is made and the function then completes.

    If a directory file was returned a synchronous call to list the details of that directory is made and then another asynchronous search call made with an AST completion function again back to this function.

  4. When the directory files are exhausted the RMS structure is reinitialized to allow the search for all specification-matching, non-directory files in the directory. An asynchronous search call is made.

  5. For each matching file found the file search AST completion function is called. Status is checked for success or otherwise. If an error the status is reported to the client and the processing concluded. If the directory contained no matching files, or the files are exhausted, the processing is concluded and the function immediately completes.

    If a file was returned a call is made to the Descr.c module to check whether a file description can be obtained (HTML files only). If it can then this module is use to generate it and the function completes. If no description can be obtained a synchronous call is made to a function to list the file details. After the file details are listed another asynchronous search call is made, with the same function specified for AST completion. The function then immediately completes.

14.8.8 - FILE.C

Code for: File.c

This module implements the file transfer functionality. It obtains the file specification and mime content type information from the request data structure. It handles VARIABLE or VFC files differently from STREAM, STREAM_LF, STREAM_CR, FIXED and UNDEFINED. With STREAM(_*), FIXED and UNDEFINED files the assumption is that HTTP carriage-control is within file itself (i.e. at least the newline (LF), all that is required required by most browsers), and does not require additional processing. With VARIABLE, etc., files the carriage-control is implied and therefore each record requires additional processing by the server to supply it. Record-oriented files will have multiple records buffered before writing them collectively to the network (improving efficiency). Stream and binary file reads are by Virtual Block, and are written to the network immediately making the transfer of these very efficient indeed! The essential behaviour however is much the same.

If conversion to STREAM_LF files is enabled this module will, upon encountering a VARIABLE or VFC file, initiate its conversion to STREAM_LF record format. This is done using the StmLf.c module.

(Versions prior to 3.2 used a configuration directive for the MIME content-type to determine whether a file was transfered record-by-record or in binary. This is no longer required.)

  1. The primary function allocates a task structure. This function then gets some file information using ACP I/O. If the file does not exist it immediately returns the error status to the calling routine for further action (this behaviour is used to try each of multiple home pages by detecting file- not-found, for example). If it does the ACP information provides modification date/time, size and record-format. If the record format is VARIABLE and STREAM-LF conversion is enabled, conversion is initiated. If the request specified an ``If-Modified-Since:'' header line the modification date is checked and a possible ``304 Not Modified'' response generated.

  2. After successfully opening the file it generates an HTTP response header if required. It then calls one of either two functions to queue the first read from the file, one for variable-record files (record-oriented transfer), another for stream (STREAM-LF and stream record formats) text and binary files (block-oriented transfer). After the read is queued it returns with a success status code.

  3. When the asynchronous file read completes one of either two AST completion functions (one for record the other for block) is called to post-process the I/O. Status is checked for success or otherwise. If an error the status is reported to the client, the file closed, and the request thread concluded.

    If end-of-file, the file is closed, for record-oriented files the buffer checked and if necessary flushed. If an end task function was specified control is now passed to that, otherwise the thread is concluded.

    If not end-of-file, for record files multiple records may be buffered before writing to the network. If the buffer is full (the read was unsuccessful due to insufficient space) the contents are asynchronously written to the network, with the network write completion routine specifying a function to re-read the the file record that just failed. If there is still space in the buffer another asynchronous read of the file is queued in an attempt to append the next record into the buffer. After the read is queued the function completes.

    If not end-of-file, for stream and binary files a successful read results in a call to the network write function to send this to the client. This call contains the address of the function to read the next blocks from the file as an AST completion routine. After the asynchronous network write is queued the function completes.

For text files the contents can be encapsulated as plain text. This involves prefixing the file send with a <PRE> HTML tag and postfixing it with a </PRE> tag. The buffer is filled as per normal but when ready to output a function is called that escapes all HTML-forbidden characters first (e.g. ``<'', ``>'', ``&'', etc.) This is used by the SSI.C module.

14.8.9 - HTADMIN.C

Code for: HTAdmin.c

(INCOMPLETE AS YET!)

The HTAdmin.c module allows on-line administration of the HTTPd-specific authentication databases.

14.8.10 - HTTPD.C

Code for: HTTPd.c

This is the main() module of the server. It performs server startup and shutdown.

14.8.11 - ISMAP.C

Code for: IsMap.c

The clickable-image support module provides this functionality as an integrated part of the server. It supports image configuration file directives in either of NCSA or CERN formats. Extensive configuration specification error reporting has been implemented.

Acknowlegement:

Three coordinate mapping functions have been plagiarized from the NCSA imagemap.c script program. These have been inserted unaltered in the module and an infrastructure built around the essential processing they provide. Due acknowlegement to the original authors and maintainers of that application. Any copyright over portions of that code is also acknowleged:

  ** mapper 1.2
  ** 7/26/93 Kevin Hughes, kevinh@pulua.hcc.hawaii.edu
  ** "macmartinized" polygon code copyright 1992 by Eric Haines, erich@eye.com

Essential behaviour:

  1. The primary function allocates a task structure and then attempts to open the map configuration file. If unsuccessful it generates an error report and concludes processing.

  2. After successfully opening the configuration file it extracts the client-supplied coordinate from the query string. A call is then made to asynchronously read a record (line) from the configuration file. Configuration file processing is asynchronous from that point.

  3. The record (line) read AST function checks for end-of-file, when it will return the default URL (if supplied). After end-of-file the file is closed and the processing is concluded.

    If not end-of-file, a function is called to parse the record for an image mapping directive. When the components have been parsed the NCSA imagemap.c routines are used to determine if the click coordinates are within the specified region coordinates.

    If it is within the region the click has been mapped and the URL is placed in heap memory and the thread's redirection location pointer set to it. The file is closed and the processing conclusion function called. This function detects the redirection location and if a local URL instead of disposing of the thread generates a new, internal request from the redirection information. In a non-local URL the client is sent a redirection response and then the thread concluded.

    If not within the region a call is made to asynchronous read the next record from the configuration file.

14.8.12 - LOGGING.C

Code for: Logging.c

The logging module provides an access log (server logs, including error messages are generated by the detached HTTPd process, see sections Server Process Logging Directory and 4.3.2.2 - Logging).

The access log format is that of the Web-standard, ``common''-format, allowing processing by most log-analysis tools. Each entry (record, line) comprises:

  client_host r_ident auth_user [time] "request" reponse_status bytes_sent
where:

In addition to legitimate request entries the server adds bogus entries to time-stamp server startup, shutdown, and the log being explicitly opened or closed (see 4.3.2.2 - Logging). These entries are correctly formatted so as to be processed by a log analysis tool, and are recognisable as being ``POST'' method and coming from user ``HTTPd''. The request path contains the event and a hexadecimal VMS status code, that represents a valid exit status only in ``END'' entries.

Clickable-image requests are logged as ``302'' entries, and the resulting, redirected request entry logged as well.

When a log entry is required the file is opened if closed. The file is again closed one minute after the initial request. This flushes the contents of the write-behind buffers.

14.8.13 - MENU.C

Code for: Menu.c

This module implements the WASD menu interpretation functionality. It obtains the file specification from the request data structure. Output from this module is buffered to reduce network writes, improving I/O efficiency.

Essential behaviour:

  1. The primary function allocates a task structure and then attempts to open the file. If unsuccessful it immediately returns the error status to the calling routine for further action (this behaviour is used to try multiple home pages, for example). No checking of modification date/times is done as menu documents are considered dynamic in a similar way to SSI documents.

  2. After successfully opening the file it generates an HTTP response header if required. A call is then made to asynchronously read a record from the file opened. After the asynchronous file read is queued the function returns with a success status code.

    When the asynchronous file read completes the AST completion function is called to interpret the line, dependant on the section number it occurs in. Status is checked for success or otherwise. If an error the status is reported to the client, the file closed, and the request concluded. If end-of-file, the file is closed and the processing concluded. For a successful record read the line can either be title, description or menu item. When the line is interpreted and written to the network another read is queued, with an AST completion routine again specifying the contents interpretation function. The function then completes.

14.8.14 - NET.C

Code for: Net.c

This module handles all TCP/IP network activites, from creating the server socket and listening on the port, to reading and writing network I/O. It manages request initiation and rundown, and controls connection persistence. The network read and write functions have provision for specifying I/O completion AST function addresses. If these are provided then the function is called upon completion of the network I/O. If not provided then the I/O completes without calling an AST routine.

The server begins by creating a network socket and then binding that to the HTTP port. The server then enters an infinite loop, waiting for IP connections.

When a connection request is received the remote host is checked as an allowed connection. If allowed, a request data structure is created from dynamic memory, and an asynchronous read is queued from the network client. The pointer to this dynamic data structure becomes the request thread, and is passed from function to function, AST routine to AST routine. The AST completion routine of the network request read(s) specifies a request analysis function. The function then returns to the connection acceptance loop.

When the network read(s) complete an AST completion function in the Request() module is called to process the HTTP request.

This module also contains the code for the BufferOutput() function described above, 14.4 - Output Buffering.

14.8.15 - PUT.C

Code for: Put.c

(INCOMPLETE AS YET!)

The PUT module allows files to be uploaded to, and stored by, the server. It also allows the deletion of files, and the creation and deletion of directories. This same module handles PUT, POST and DELETE methods appropriately. It requires authorization to be enabled on the server. Created files have a three-version limit applied.

The Request.c module controls the size of any request POSTed or PUTed via a configurable parameter limiting the size in Kilobytes. The request is completely read by that same module before being parsed and handed over to the Put.c module (or DCL.c if a script). Hence the Put.c module has a complete request body pointed in-memory that it can process.

POSTed or PUTed requests are processed differently according to the MIME content-type:

The parent directory of any file/directory operation is always checked for permission to modify its contents. This permission is usually granted to the HTTPd account via an ACL. Files are written using Virtual Block I/O. This can make them very efficient. They are handled asynchronously, not disturbing the multi-threading of the server.

14.8.16 - REQUEST.C

Code for:Request.c

This module reads the request header from the client, parses this, and then calls the appropriate task function to execute the request (i.e. send a file, SSI an HTML file, generate a directory listing, execute a script, etc.)

The request header is contained in the network read buffer. If it cannot be completely read in the first chunk, the read buffer is dynamically expanded so as to be read in multiple chunks. The request header is addressed by a specific pointer that allows the parse-and-execute function to process either a genuine, initial client request header, or a pseudo-header generated to effect a redirection.

The method, path information and query string are parsed from the first line of the header. Other, specific request header fields are also parsed out and stored for later reference. Once this has been done the header is not further used.

Once the relevant information is obtained from the request header processing commences on implementing the request. This comprsises the rule-mapping of any path information, the RMS parsing of any resulting VMS file specification and decision-making on how to execute the request.

This functionality is used to parse and execute both the initial client request and any pseudo-request generated internally to effect a redirection.

If a POST/PUT method is used the entire request body (using the ``Content-Length:'' header line to determine length) is also read into dynamic memory before passing to the PUT.C or DCL.C modules for processing.

14.8.17 - SSI.C

Code for: SSI.c

The Server Side Includes (HTML pre-processor) module provides this functionality as an integrated part of the server. Output from this module is buffered to reduce network writes, improving I/O efficiency.

Essential behaviour:

  1. The primary function attempts to open the file. If unsuccessful it immediately returns the error status to the calling routine for further action (this behaviour is used to try multiple home pages, for example). SSI documents can be dynamic in undetectable ways so no checking of file modification date/time is done.

  2. After successfully opening the file it generates an HTTP response header if required. A call is then made to asynchronously read a record from the file opened. The record read AST function scans the record (line) looking for pre-processor directives embedded in HTML comment directives. If no directive is found the record is output buffered and another queued to be read.

  3. If a directive is detected any part of the line up the directive is output buffered and a function called to parse the directive. This function reports an error if the directive specified is not supported (unknown, etc.) If a supported directive a specific function is called according to the directive specified. These functions provide the pre-processor information in one of four ways:

    1. Internally

      Information such as the system time, current document information, etc., can be provided from information contained in the request data, etc., or in the case of specified document/file information obtained via the file system. These directives have the relevant information buffered and then the function returns to the directive parsing function.

    2. Via DCL Execution

      Information that must be obtained through DCL execution is obtained using an asynchronous call to the Dcl() module. The next-task function is specified as the line parsing function. When the DCL module has finished executing the required command control is passed back to this function.

    3. Sending a File

      If a file is #included this is provided with an asynchronous call to the File() module. The next-task function is specified as the line parsing function. When the File() module has finished transfering the included file control is passed back to this function.

    4. Directory Listing

      If a directory listing is requested this is provided via an asynchronous call to the Dir() module. The next-task function is specified as the line parsing function. When the Dir() module has finished generating the listing control is passed back to this function.

  4. Directives continue to be parsed, and executed, asynchronously if necessary (as just described), from within a line until the end-of-line is reached. Any remaining characters are output buffered. Lines continue to be read from the file using the AST mechanism until end-of-file.

14.8.18 - STMLF.C

Code for: stmLF.c

The stmLF.c module converts VARIABLE format records to STREAM-LF. It is only called from the File.c module and only then when STREAM-LF conversion is enabled within the server configuration.

The conversion is done asynchronously (naturally) concurrently with any reads being performed by the File.c module in transfering the file to the client. After successful conversion the original file is purged.

14.8.19 - SUPPORT.C

Code for: Support.c

The support module provides a number of miscellaneous support functions for the HTTPd (well go-o-o-lee!).

14.8.20 - UPD.C

Code for: Upd.c

(INCOMPLETE AS YET!)

The Upd.c module implements the on-line web directory administration and file editing and upload facility. It requires authorization to be enabled on the server. File and directory modification are still performed by the Put.c module. The Upd.c is an overly large body of code generating all of the dialogues and editing pages. It also provides additional functionality for the server administration, adding admin-specific portions of dialogues as required.


[next] [previous][contents]