Document Searching

WASD Hypertext Services - Environment Overview

[next] [previous][contents]

7 - Document Searching

The query and extract scripts provide real-time searching of plain-text and HTML documents, and document retrieval. The search is a simple-string search, not a GREP-style search. It is designed to provide a useful mechanism for locating documents containing a keyword, not for document analysis. It has the useful feature for plain-text documents of allowing the selective extraction of only the portion near the hit.

- NOTE -

Searching is a notoriously CPU and I/O intensive activity. Longer searches progressively decrease scheduling priority by one every 10 seconds, from normal to zero, helping to reduce impact on any interactive users of the server system. The search algorithm itself is efficient, but searching will take longer on a more heavily loaded system because of this mechanism.

Only files with a plain-text or HTML MIME data type (see 3 - Document Access and Specification) will be searched. Others may be specified, or be selected from wildcard file specification, but they will not actually have their contents searched.

7.1 - Plain-Text Search

A search of a plain-text file is straight-forward. Each line in the file is searched for the required string. The first time it is encountered is considered a hit. The line is not searched for any further occurances.

Searches of plain text files allow the subsequent selection of partial documents (i.e. the retrieval of only a number of lines around any actual hit). This allows the user to selectively extract a portion of a document, avoiding the need to explcitly scan through to the section of interest.

7.2 - HTML Search

A search of an HTML file is a little more complex. As might be expected, only text presented in the document text is searched, markup text is ignored. That is, all text not part of an HTML tag construct is extracted and searched. For example, out of the following HTML fragment

  <!-- an example HTML document -->
  <P>
  The document entitled <A HREF="example.html">"Example Document"</A>
  provides only an <I>overview</I> of the full capabilities of HTML.
only the following text would actually be searched
  The document entitled "Example Document" provides only an overview
  of the full capabilities of HTML.

The mechanism for partial document retrieval available with plain-text files is not present with HTML documents. HTML files generally must be treated as a whole, with the formatting of current sections often very dependent on the formatting of previous sections. This makes extracting a subsection perilous without extensive syntactical analyis. On the positive side, HTML documents tend to be already divided into meaningful subdocuments (files), making retrieval of a hit naturally more-or-less within context.

7.3 - Search Syntax

A search may be initiated in one of two ways:

  1. Appending a question-mark and search string to a file specification (the simple syntax of ISINDEX-style searching). This is standard HTTP, and of course must conform to HTTP syntax.
  2. Forms-based search, which allows the format and mechanism of the search to be controlled.

7.3.1 - ISINDEX Search

Placing the HTML tag <ISINDEX> within a document's text is sufficient to inform the browser that searching is available for that document. The browser will inform the user of this and allow a search of that document to be initiated at any time. Note that it is limited to the one document.

Using the keyword search syntax explicitly is another method of initiating a search, and additionally can use a wildcard in the document specification. For example:

  /web/html/html-primer/*.html?problem

7.3.2 - Forms-Based Search

A ``forms-based'' search is initiated by the server receiving a file specification, which of course may contain wildcards, followed by a search parameter. This is a typical HTML forms format URL. For example:

   *.txt?search=SIMPLE
   /web/.../*.*?search=THIS
   sub_directory/*.*?search=THAT
   ../sibling_directory/*.HTML?search=OTHER

The following link provides an online demonstration search. It searches all files (plain-text and HTML) in the current directory for the keyword "formatted". Note the difference in the way plain-text file hits are presented compared with those of HTML files.

Search for "formatted"

7.3.3 - Search Options

Additional URI components may be appended after the initial ``search='' parameter. These are appended with intervening ``&'' characters.

7.3.4 - Example Search Form

To allow the client to enter a search string and submit a search to the server a HTML level 2 form construct can be used. Here is an example:

  <FORM ACTION="/web/html/.../*.html">
  Search HTML documents for: 
  <INPUT TYPE=text NAME="search">
  <INPUT TYPE=submit VALUE="[execute]">
  </FORM>

The following provides an online demonstration of the form used above:

Search HTML documents for:

Bells and Whistles

A form providing all the options refered to in 7.3.3 - Search Options is shown below (some additional white-space introduced for clarity):

  <FORM ACTION="/web/html/.../*.html">

  Search HTML documents for: 
  <INPUT TYPE=text NAME="search">
  <INPUT TYPE=submit VALUE="[execute]">

  <BR><TT><A HREF="/web/?about=search">About</A> this search.</TT>

  <BR><TT>Output By:
  line <INPUT TYPE=radio NAME="hits" VALUE="line" CHECKED>
  document <INPUT TYPE=radio NAME="hits" VALUE="document"></TT>

  <BR><TT>Case Sensitive:
  no <INPUT TYPE=radio NAME="case" VALUE="no" CHECKED>
  yes <INPUT TYPE=radio NAME="case" VALUE="yes"></TT>

  </FORM>

The following provides an online demonstration of the form used above:

Search HTML documents for:
About this search.
Output By: line document
Case Sensitive: no yes


[next] [previous][contents]