How to Enable and Interpret the Pop3.log File (155515)

MORE INFORMATION

The remainder of this article is organized into two sections: "POP3 Basics" and "Interpreting the Pop3.log File."

POP3 Basics

For information about how to enable the Pop3 log in Outlook Express, please see the following article in the Microsoft Knowledge Base: This section includes a general introduction to POP3 design, connections, and session commands and syntax.

Special conventions used in this article follow.

The following abbreviations are used frequently:

• POP3 Design

The Post Office Protocol version 3 (POP3) is a text-based protocol using the US ASCII set of characters, a subset of the ASCII character set containing octet values 00h-7Fh (0d-127d) inclusive. Its sole requirements for a network-layer transport mechanism are ordered, reliable packet delivery.

POP3 was developed to satisfy a need for a dynamically accessible mail delivery system. In the early days of ARPA, the ultimate destination of an email message was usually a large host system that was continually connected to the network. There was no need for mail to be stored for later retrieval as the message was delivered to the user's terminal when it was received. However, with the proliferation of inexpensive dial-up access to the Internet, the need arose for a mail storage mechanism because users are not always connected to their service providers.

The POP3 protocol is used only as a delivery mechanism between a mail client and a mail host (post office). The messages that reside in the client's mail drop are delivered there by the Simple Mail Transfer Protocol (SMTP). For more information on SMTP and its use with Microsoft products, please see the following article in the Microsoft Knowledge Base: POP3 transfers involve a client-server model where each participant can send, receive, and interpret session status messages from the other. The client usually initiates and terminates the connection to the host.

One very important difference between SMTP and POP3 is that SMTP, being basically a mail relay agent, does not require user authentication before allowing access to SMTP functions. Since POP3 is a 'post office protocol,' and POP3 servers have multiple mail drops, it implements several methods to authenticate clients.

NOTE: This article only covers the use of the USER and PASS keywords for POP3 authentication. For more information regarding POP3 authentication techniques, please see either the POP3 specification [1] or RFC 1734.

A second important difference between SMTP and POP3 servers is that the POP3 server employs a state machine, similar to the finite-state automaton in PPP, that passes through several states during the course of a connection. State transitions are driven by events triggered by the client.

• POP3 Connections

A POP3 server, typically implemented as a daemon process on a computer running a UNIX-based operating system, listens for traffic on TCP port 110. Although this port assignment is standard, many client software packages have a user-configurable option to change the port number to compensate for non-standard server implementations.

NOTE: To change this value in Microsoft Internet Mail and News (IMN), use the following steps:

1. On the Mail menu, click Options.
2. On the Server tab, click Advanced Settings.
3. Enter the appropriate value in the Incoming Mail (POP3) Server box.
4. Click OK.

To change the server name and/or the user name and password information, use the following steps:

1. On the Mail menu, click Options.
2. Click the Server tab.
3. Enter the appropriate values in the appropriate boxes.
4. Click OK.

• POP3 Session Commands & Syntax

For the purpose of discussion, the peers in a POP3 session will be differentiated into client and host. The client is assumed to be mail client software. The host is always a POP3 server.

• POP3 Client Commands

Commands are composed of a case-insensitive keyword and any arguments that are needed to complete the command. Commands have the following format: The <arguments> field may contain several parameters, each being up to 40 characters long. Not all command keywords have arguments and not all command keywords that use arguments require that they are used.

• POP3 Host Responses

Responses are composed of a status indicator and an implementation- specific ordering of a keyword/additional information string. Responses have the following format: With regards to responses, the following semantics must be followed:

• The host must send a response to every command. This includes unimplemented, unrecognized, and syntactically incorrect commands.
• Responses may be single or multiple-line. Single-line responses can be a maximum of 512 octets, including the terminating CRLF.
• Multiple-line responses are terminated by a special octet sequence. The sequence is five octets long and consists of the termination octet '.' (29h, 41d) bounded by CRLF pairs. This is easily identified as a period on a line by itself.
• The <status indicator> field is either positive (+OK) or negative (-ERR) and is always in uppercase.
• The <keyword> field in an error response is usually returned in uppercase regardless of how it was sent.

For the purpose of this article, the definition of the response, positive or negative, within a given state and in response to a given command keyword is shown in parenthesis. The important parts of responses are noted where necessary.

The scope of this article covers the commands most likely to be seen while examining the Pop3.log produced by IMN, with the exception of the TOP command. For a complete listing and explanation of POP3 commands, see RFC 1939.

The rest of this section will be organized according to the commands that you should expect to see used within a given POP3 state. Not all commands can be issued in all states. The primary commands that can be issued within a given state will be covered first, and then any additional commands that may be common to multiple states will follow.

The order of the commands presented below does not imply that there are No other possible sequences of commands. The following commands and explanations outline a simple connection.

• Connection Established

When a client wishes to retrieve mail, it first opens a connection to TCP port 110 on the POP3 host.

When the connection is established, the POP3 host responds with a one-line greeting in the following format: If when troubleshooting a POP3 connection a greeting is not received, the server may be malfunctioning or down.

For more information on troubleshooting POP3 connections, please see the following article in the Microsoft Knowledge Base: After issuing the greeting, the POP3 state machine transitions to the AUTHORIZATION state.

• AUTHORIZATION State

After receiving the host's greeting, the client must identify and authenticate itself. This is achieved through the use of the USER and PASS commands. The USER command identifies a mail drop on the POP3 host, and is issued prior to the PASS command.

• USER Command

    Syntax : USER SP <user-id> CRLF

    Expected responses : +OK (name is a valid mailbox)
                         -ERR (never heard of mailbox name)
				

NOTE: Some POP3 implementations, in order to guard against intrusion attempts based on repetition, may not issue a negative response, but will instead request a password. An example of this is provided later.

If the USER command generates a positive response, the client will then send the PASS command.

• PASS Command

    Syntax : PASS SP <password> CRLF

    Expected responses : +OK (mailbox locked and ready)
                         -ERR (invalid password)
                         -ERR (unable to lock mail drop)
				

The +OK response will usually contain a message that indicates the number of waiting messages and total mail drop size in octets. Since no significant information is derived from this response (other than the user was logged in successfully), only the +OK portion is needed.

The second -ERR response usually indicates that the user is already logged in on another connection. It is also possible that the user's mail drop could be corrupted and the host cannot parse it, or the resource containing the mail drop be temporarily unavailable. Finally, in some POP3 server implementations, particularly on UNIX hosts, the 'unable to lock mail drop' response is typical of other error conditions present on the server. As such, the error message may not properly represent the problem. Either way, the user needs contact whomever is responsible for administrating their POP3 server.

• QUIT Command

The QUIT command is a multiple-state command which has the following effect when issued in the AUTHORIZATION state.

   Syntax : QUIT <no arguments> CRLF
   Expected response : +OK (host closing connection)
				

When the QUIT command is issued in the AUTHORIZATION state, the POP3 server does not transition to the UPDATE state (covered later), it simply closes the current connection and beings listening for another.

If authentication succeeds, the POP3 state machine will transition to the TRANSACTION state.

• TRANSACTION State

After the user has been authenticated, messages can be accessed within the user's mail drop. POP3 only provides simple functionality where message manipulation is concerned, more often than not, messages are retrieved then deleted. Once in the TRANSACTION state, the POP3 host will accept many different commands, which can be issued multiple times by the client. This state is where the majority of client-host interaction occurs. The commands valid in this state are described below.

• STAT Command

The STAT command is issued by the client to determine the number of messages (n) in and the total size (yyyy) of the mail drop. This is known as a 'drop listing.'

   Syntax : STAT <no arguments> CRLF
   Expected Response : +OK n yyyy
				

The +OK response is the only possible response as defined in the current specification for POP3 [1].

• LIST Command

The LIST command returns information regarding the size of a single message or all the messages in a mail drop. This is known as a 'scan listing.'

   Syntax : LIST SP <message number> CRLF [argument is optional]
   Expected Responses : +OK (scan listing follows)
                        -ERR (no such message)
				

The format of the +OK response is the same whether or not the <message number> argument is specified or not:

   +OK n yyyy
				

Where 'n' is a decimal value indicating either an individual message number or the total number of messages in a mail drop; and 'yyyy' is a decimal value representing either the size of an individual message or the entire mail drop in octets. What follows depends on whether or not the <message number> argument is used.

If the <message number> argument is used, it must refer to a message number that has not been marked for deletion using the DELE command(covered later). The host will echo the message number and return the size of the message in octets. Being a single-line response, there is no termination sequence. For example:

   C: LIST 1
   H: +OK 1 2039
				

If the <message number> argument is not used, this command returns a multi-line response. The listing starts with the +OK response containing the mail drop statistics, followed by one line listings for each message, and then terminated with the CRLF.CRLF sequence.

NOTE: In Pop3.log, all that is shown is the +OK response.

For example, this is what was actually sent from the POP3 host: This is what Pop3.log shows:

   POP3: 10:25:43 [tx] LIST
   POP3: 10:25:43 [rx] +OK 7 messages (15423 octets)
				

• UIDL Command

The UIDL command returns the 'unique-id listing' for a single message or all the messages in a mail drop. It is composed of a unique character string that is a maximum 70 octets in length, each having a value between 21h-7Eh (33d-126d). This range of values excludes all non-printable characters (00h-19h), SPACE (20h), and DEL (7Fh).

It is stated in the POP3 specification [1] that the unique-id value should never be duplicated in a mail drop. This idea is similar to not being able to have to files of the same name in the same directory - even if it was possible, how would you tell them apart?

If Internet Mail is configured to leave a copy of the messages on the POP3 server, this information is cached in a file called POP3UIDL.DAT. Later, when the user checks for mail, the UIDL command is issued to the POP3 server and the information returned is checked against what is in POP3UIDL.DAT. If a message's unique-id is not found in the DAT file, it is retrieved. If it is found, it is left alone.

If this file is deleted, IMN will download all the messages on the host, regardless of whether or not they have been retrieved previously.

If the user disables the "Leave a copy of messages on server" option, two things will happen the next time they connect to the POP3 server:

1. Mail that has been stored will be deleted.
2. The information in the POP3UIDL file is cleared, and the file is reset to its default size of 312 bytes.

NOTE: To ensure suitable randomness, the value is recommended to be calculated from a hash of the message contents. How it is actually done depends on the mail client implementation.

   Syntax : UIDL SP <message number> CRLF [argument is optional]
   Expected Responses : +OK (unique-d listing follows)
                        -ERR (no such message)
				

The format of the +OK response is the same whether the <message number> argument is supplied or not:

   +OK n yyyy
				

Where 'n' is the message number, and 'yyyy' is the unique-id listing. If the <message number> argument is provided, the response will be one line. If it is not, the responses will be formatted similarly, but will follow multiple-line conventions.

NOTE: In Pop3.log, all that is shown is the +OK response.

For example, this is what was actually sent from the POP3 host: This is what Pop3.log shows:

   POP3: 10:25:43 [tx] UIDL
   POP3: 10:25:43 [rx] +OK
				

This truncated version of the host's output, as was done with the LIST command's response, is done largely to keep the size of the logfile down and because the actual response information is not important to the user.

NOTE: Not all servers implement the UIDL command. In the event the server indicates that the UIDL command is not supported, IMN will fall back to the TOP command, and use the message-id contained in the message's headers. This value, like the unique-id, is assumed to be completely unique.

• TOP Command

The TOP command is used to display a given message's headers and any number of lines of the body text of the message.

   Syntax : TOP SP <message number> SP <nn> CRLF
   Expected Responses : +OK (top of message follows)
                        -ERR (no such message)
				

The <message number> argument specifies which message the user would like to see the TOP of. The <nn> argument is a non-negative integer value indicating the number of lines of the message to be sent.

If <number of lines> is zero, only the message's headers will be sent by the host.

If <number of lines> is greater than zero the message headers, a blank line, and then 'nn' lines of the message will be sent by the host.

• RETR Command

The RETR command is used by the POP3 client to retrieve a message from the host.

   Syntax : RETR SP <message number> CRLF
   Expected Responses : +OK (message follows)
                        -ERR (no such message)
				

When the RETR command is issued, the message referenced in the argument is transmitted in its entirety to the client. Note that the use of this command and subsequent retrieval of a message does not imply that the message will be deleted afterwards. This is the job of the DELE command, covered next.

Pop3.log does not log the contents of each message, rather, it only logs the command being issued and whatever immediate response is returned. For example, here is what the host sends when this command is issued: This is what Pop3.log shows:

   POP3: 10:25:43 [tx] RETR 1
   POP3: 10:25:43 [rx] +OK 476 octets
				

• DELE Command

The DELE command is used to by the POP3 client to remove a message from the mail drop.

   Syntax : DELE SP <message number> CRLF
   Expected Responses : +OK (message deleted)
                        -ERR (message already deleted)
                        -ERR (no such message)
				

When the DELE command is issued, the message referenced in the argument is marked for deletion, it hasn't actually been removed from the mail drop yet. The significance of being marked for deletion is that the client can no longer issue a command with that message's number as an argument, any such attempt will generate an error.

The DELE command can be countermanded if the client issues the RSET command, which is covered later. The command sequence below illustrates the use and effect of the DELE command.

   H: +OK pop3.host.com POP3 server ready
   .
   . <client logs into the POP3 host>
   .
   H: +OK user's mail drop has 1 messages (476 octets)
   C: LIST 1
   H: +OK 1 476
   C: DELE 1
   H: +OK Message 1 deleted.
   C: RETR 1
   H: -ERR Message 1 was deleted.
   C: RSET
   H: +OK user's mail drop has 1 messages (476 octets)
   C: LIST 1
   H: +OK 1 476
   .
   . <session continues>
   .
   C: QUIT
   H: +OK pop3.host.com closing connection. Good Night.
				

• RSET Command

If there are any messages marked for deletion, the RSET command 'undeletes' them. An example of the command is above, and the syntax for the command is below.

   Syntax : RSET CRLF
   Expected Response : +OK
				

The POP3 specification [1] does not mandate that any information follow the '+OK' portion of the response. There is no provision for an -ERR response, the server must respond with the positive reply.

• NOOP Command

The NOOP command has the same function as it does in SMTP, it serves to let the server know that the client has not hung or become inoperable. If the POP3 client is designed to process messages (formatting, attachment decoding, etc.) on-the-fly, this command is useful if there are significant delays incurred.

   Syntax : NOOP CRLF
   Expected Response : +OK
				

A negative response is not provided. This command has no effect on the server's state machine or any messages in the user's mail drop.

• QUIT Command

The QUIT command is a multiple-state command which causes the server to transition to the UPDATE state when issued in the TRANSACTION state.

• UPDATE State

The UPDATE state is only time the server attempts to permanently remove all messages in user's mail drop marked for deletion. No input from the client is processed in this state.

   Syntax : QUIT <no arguments> CRLF
   Expected response : +OK (host closing connection)
                       -ERR (some messages note removed)
				

Normally, the positive response is returned to the client, and the host terminates the session. If the host encounters a problem while removing messages, the negative response is issued.

If the session terminates abnormally, that is, the connection was dropped or terminated, the host will not enter the UPDATE state and any messages marked for deletion will be reset.

Interpreting the Pop3.log File

POP3 logging is not enabled by default. To enable POP3 logging in Outlook Express, click Options on the Tools menu, click the Advanced tab, and then click the Mail Transport check box to select it. To enable POP3 logging in Internet Mail and News, you must add or change the following entries in the registry::

   HKEY_CURRENT_USER\Software\Microsoft\Internet Mail and News\Mail

   Value Name : "Log POP3 (0/1)"  (without quotes)
   Value Type : dword
   Value Data : 00000001

   Value Name : "Log File (POP3)" (without quotes)
   Value Type : string
   Value Data : <path to log file>
				

In order to fit all the pertinent data for each entry on one line, the 'POP3:' tag and the time stamp at the beginning of each line have been removed.

To view the Pop3.log file, follow these steps:

1. Click Start, point to Find, and then click Files Or Folders.
2. In the Named box, type "pop3.log" (without quotation marks), and then click Find Now.
3. In the list of found files, double-click the Pop3.log file to open it.

Log Example - Typical Session

 01) Microsoft(r)  Plus!  for  Windows(r)  95 4.70.1155
 02) POP3 Log started at Sun Aug 18 16:27:52
				

Lines 1 and 2 are plugged in every time Internet Mail and News is launched.

 03) [db] Connecting to 'pop3.host.com.'
 04) [db] srv_addr = nnn.nnn.nnn.nnn
				

Lines 3 and 4 indicate the friendly name and the IP address of the POP3 server specified in the Internet Mail and News configuration.

 05) [rx] +OK pop3.host.com POP3 server ready Sun, 18 Aug 1996 14:28:31 -
          0700
 06) [tx] AUTH twinkie
 07) [rx] -ERR Command unknown: AUTH
 08) [tx] USER joe
 09) [rx] +OK Password required for joe
 10) [tx] PASS ********
 11) [rx] +OK joe's mail drop has 28 messages (215096 octets)
				

Line 5 indicates that the communications channel between the client and the host has been successfully opened.

In Line 6, the AUTH command [2] is issued to the server with a dummy value of 'twinkie.' The AUTH command is an authentication method used by a post office specification known as the Internet Message Access Protocol - 4 (IMAP4) [3]. The command's failure is indicated by the negative response in Line 7. This command was included to test for authentication methods supported by the POP3 server. In Line 8, the client begins authenticating itself by issuing the USER command with 'joe' as the user-id. Line 9 indicates that the user-id has been accepted and prompts for a password. As mentioned before, POP3 servers will typically not indicate that a user-id is invalid. The only time a negative response will be seen here is if a character outside of the 21h-7Eh range finds its way into the string.

Line 10 shows that the password has been transmitted. Since the password is transmitted in the clear, it is masked by asterisks in the Pop3.log file.

The positive response on Line 11 indicates that authentication was successful. The server transitions to the TRANSACTION state at this time.

 12) [tx] STAT
 13) [rx] +OK 28 215096
				

The STAT command is issued to obtain the number of messages in the mail drop, and its size.

 14) [tx] UIDL 1
 15) [rx] +OK 1 19960814211633540.AAA85@smtp.host.com
 16) [tx] UIDL
 17) [rx] +OK
				

The UIDL command is issued in Line 14 with the parameter '1.' The server returns the unique-id string of the first message in the mail drop. Then, in Line 16, the command is issued again with no parameters. This will return the unique-id strings for all the messages in the mail drop.

 18) [tx] LIST
 19) [rx] +OK 28 messages (215096 octets)
				

In line 18, the LIST command is sent with no parameters. The response generated is multiple-line, and all that is shown in Pop3.log is the response status. This command is useful to determine the size of each message in the user's mailbox. If the user has specified that they do not want to download messages over a given size, that information is checked here.

 20) [tx] DELE 1
 21) [rx] +OK Message 1 deleted
 22) [tx] DELE 2
 23) [rx] +OK Message 2 deleted
 24) [tx] DELE 3
 25) [rx] +OK Message 3 deleted
				

Here is where the information gained from the UIDL command is used. This log file was generated after the "Leave copy..." was disabled. The first three messages in the mail drop had already been retrieved prior to this connection, and are now being removed.

 26) [tx] RETR 4
 27) [rx] +OK 8875 octets
 28) [tx] DELE 4
 29) [rx] +OK Message 4 deleted
 .
 .
 .
 30) [tx] RETR 28
 31) [rx] +OK 14288 octets
 32) [tx] DELE 28
 33) [rx] +OK Message 28 deleted
				

Line 26 shows the beginning of message retrieval. Unless the user has chosen to preserve the messages on the server, each message is marked for deletion after it is downloaded. Remember that the message is not actually removed from the mail drop until the QUIT command is issued.

 34) [tx] QUIT
 35) [rx] +OK pop3.host.com POP3 server closing connection
 36) [db] Connection to 'pop3.host.com' closed.
				

After all messages have been downloaded, the client issues the QUIT command, which triggers the POP3 server's transition to the UPDATE state. This is when messages are removed from the mail drop. When all the messages have been removed, the +OK response is issued, and the host closes the TCP/IP connection.

Log Example - Failed Login

In this example, the user account is valid, but the password is wrong.

 01) [db] Connecting to 'pop3.host.com.'
 02) [db] srv_addr = nnn.nnn.nnn.nnn
 03) [rx] +OK pop3.host.com POP3 server ready Sun, 18 Aug 1996 14:28:31 -
          0700
 04) [tx] AUTH twinkie
 05) [rx] -ERR Command unknown: AUTH
 06) [tx] USER joe
 07) [rx] +OK Password required for joe
 08) [tx] PASS ******
 09) [rx] -ERR Password failed for joe
 10) [tx] QUIT
 11) [rx] +OK pop3.host.com POP3 server closing connection
 12) [db] Connection to 'pop3.host.com' closed.
				

In this example, the user account is invalid, so the password does not matter.

 01) [db] Connecting to 'pop3.host.com.'
 02) [db] srv_addr = nnn.nnn.nnn.nnn
 03) [rx] +OK pop3.host.com POP3 server ready Sun, 18 Aug 1996 14:28:31 -
          0700
 04) [tx] AUTH twinkie
 05) [rx] -ERR Command unknown: AUTH
 06) [tx] USER jdoe
 07) [rx] +OK Password required for jdoe
 08) [tx] PASS ******
 09) [rx] -ERR jdoe unknown, or access denied from [xxx.xxx.xxx.xxx]
 10) [tx] QUIT
 11) [rx] +OK pop3.host.com POP3 server closing connection
 12) [db] Connection to 'pop3.host.com' closed.
				

This article is intended as a reference to be used when all other troubleshooting has failed. The majority of POP3 transmission errors are going to be related to either hardware configuration errors (modem, serial port, initialization string), TCP/IP connectivity problems, or client-side malfunction.

As a general rule, always rule out any other possible source of error before troubleshooting an odd log file entry.