MORE INFORMATION
General Information
Error -1216 is triggered when comparison of the header information in the databases and log files indicates that there has been replacement or removal of critical files. This error is a warning to the administrator that if soft recovery is run, it might allow the storage group to start, but will leave the storage group running with some data missing. If you run recovery, it might be difficult or impossible to re-integrate the missing information into the storage group.
NOTE: Not all database startup failures can be attributed to this problem, and the steps that are described in this article to resolve -1216 errors are not effective for all startup problems. If other startup errors are reported instead of or in addition to -1216 errors, refer to appropriate Microsoft Knowledge Base articles for advice about resolving those errors.
An Exchange 2000 database is composed of three kinds of files:
- Database files (.edb and .stm matched file pairs)
- Transaction log files (.log files)
- A checkpoint file (.chk)
All of these files have "headers." A header is the first page in the file; the header contains information that defines the file's relationships with other files in the storage group. You can run Eseutil to view headers as plain text for any of the three file types. The
/mh,
/ml, and
/mk switches show the contents of database, transaction log, and checkpoint files, respectively, for example:
eseutil /mh Priv.edb
eseutil /ml E00.log
eseutil /mk E00.chk
Header information in the database and checkpoint files is updated frequently while the system is running, and contains detailed current state information about the database. Header information in log files is generated when the log file is created and is never changed afterward.
Whenever a storage group is started, whether the last shutdown was abnormal or not, header information is analyzed by the system; if the system's comparison of file headers reveals anomalies, a -1216 error is returned.
After you run recovery against a given data set, you might not be able to re-integrate information that was missing at the time of recovery. For example, if log files are missing, and you later discover and restore those log files, they cannot be replayed after you run recovery against a particular copy of the database files. If you are certain that the available file set is the most complete set that you can construct, you can often use Eseutil to override the -1216 error. Before you run Eseutil to force soft recovery, you must know:
- The log prefix for the storage group.
- The transaction log location for the storage group.
- The system path location for the storage group.
All three of these values are located on the
General tab of the storage group object's properties in Exchange System Manager.
To force recovery, type the
ESEUTIL command exactly as follows, including the parameter order (the command is not case-sensitive):
ESEUTIL /R Enn /I /L[log_path] /S[system_path]
For example:
ESEUTIL /R E00 /I /LD:\MDBDATA /SC:\MDBDATA
In earlier versions of Exchange, if no Edb.log file exists in the log folder that
ESEUTIL /R /L points to, an Edb.log file is created to allow recovery to start. In Exchange 2000, an E0n.log file must already exist in the log folder; this is an additional safeguard against running recovery against the wrong file set.
-1216 Errors When One or More Database Files Are Inconsistent
If a storage group is stopped unexpectedly, some or all of the database files in the storage group are likely to be inconsistent. If the problem that caused the storage group to stop leaves one or more database files unavailable, an error message that is similar to the following is logged in the Application event log:
Event Type: Error
Event Source: ESE98
Event Category: Logging/Recovery
Event ID: 0
Date: 4/24/2001
Time: 6:20:18 PM
User: N/A
Computer: EXCHANGE1
Description: Information Store (4312) Database recovery failed with error -1216 because it encountered references to a database, 'D:\exchsrvr\mdbdata\PRIV2.edb', which is no longer present. The database was not brought to a consistent state before it was removed (or possibly moved or renamed). The database engine will not permit recovery to complete for this instance until the missing database is re-instated. If the database is truly no longer available and no longer required, please contact PSS for further instructions regarding the steps required in order to allow recovery to proceed without this database.
You can determine if a database file is Inconsistent by running the following
ESEUTIL command against the database file:
eseutil /mh database_name | find /i "consistent"
: Exchange 2000 Service Pack 2 and later do not report the database state as "Consistent" or "Inconsistent" but as "Clean Shutdown" or "Dirty Shutdown." The meaning of "Clean Shutdown" is the same as "Consistent", and the meaning of "Dirty Shutdown" is the same as "Inconsistent". For Exchange 2000 Service Pack 2 or later, replace the preceding command line with:
eseutil /mh database_name | find /i "Shutdown"
The following is a sample of the output of this command:
D:\mdbdata>eseutil /mh priv.edb | find /i "consistent"
State: Inconsistent
Last Consistent: (0x2CC7,1F14,1F7) 04/04/2001 18:07:14
D:\mdbdata>eseutil /mh priv.stm | find /i "consistent"
State: Inconsistent
Last Consistent: (0x2CC7,1F14,1F7) 00/00/1900 00:00:00
If any database in a storage group is inconsistent, you must use extreme caution with all transaction log files. A database file is inconsistent if all of its data from all of the transaction log files has not been applied to it. The missing data is spread through the log files between the checkpoint log and the current log file (E0n.log, where nn is 00, 01, 02, or 03). Typically, this data is in the most recent three or four log files.
All of these logs must be present to recover the database successfully. If you are missing any of these logs, recovery of the database does not work, which leaves you with only two alternatives:
- Restore the database from a backup.
- Repair the database. To do this, run the eseutil /p command, run the eseutil /d command, and then run the isinteg -fix command.
For more information about the eseutil commands, click the following article number to view the article in the Microsoft Knowledge Base:
259851
Ramifications of running the eseutil /p or edbutil /d /r command in Exchange
You can determine which log files each database needs for recovery by using Eseutil to examine the Logs Required line in each header:
eseutil /mh database_name | find /i "log required
The following is a sample of the output of this command:
D:\mdbdata>eseutil /mh priv.edb |find /i "log required"
Log Required: 11521-11524
You must translate the range that is listed from decimal to hexadecimal to determine which log files are required. In the preceding example, the log file range from E0n02d01.log through E0n02d04.log is required.
TIP: To convert from decimal to hexadecimal, start the Calc.exe utility, click
View, and then click
Scientific. Type a decimal number, and then click
Hex to convert the number.
In most cases, you will not find the highest log that is required on disk, because this log is still named E0n.log (it is renamed and numbered when it is full). To verify that E0n.log is the highest log, run the following command:
eseutil /ml E0n.log | find /i "lgeneration"
The following is a sample of the output of this command:
D:\mdbdata>eseutil /ml e00.log |find /i "generation"
lGeneration: 11524 (0x2d04)
If inconsistent database files are removed before soft recovery has a chance to restore consistency to all of the databases in the storage group, a -1216 error is generated during the next attempt to mount any database in the storage group. (Soft recovery runs against a storage group as a whole, not against an individual database in the group.)
At this point, you can restore the missing database from a backup, or you can use Eseutil to bypass recovery of the database. After the other databases in the storage group have been mounted, there are several methods that you can use to restore the damaged database to service:
- Restore the database from a backup, and then play log files forward to recover data up to the time of the crash.
- Use Eseutil and Isinteg to repair the database. This method results in some data loss even if repair is successful.
- Delete the database, and then mount it. Exchange 2000 prompts you to create new database files. All of the information in the database is lost, but the database is restored to service.
-1216 Errors When All Database Files Are Consistent
You are most likely to encounter this situation after you restore an offline backup. You might also encounter this situation after you repair an inconsistent database and rejoin the database to the storage group when no other databases in the storage group have been successfully mounted in the interim. Even if all of the databases are consistent, a -1216 error is returned when the last consistent value in one of the database headers references a log file that is missing and the checkpoint file is invalid or has been removed.
Do not remove the checkpoint file unless you intend to replay all of the log files from the lowest to the highest last consistent logs. For additional information, click the article number below
to view the article in the Microsoft Knowledge Base:
296788 XADM: Offline Backup and Restoration Procedures for Exchange 2000 Server
Recovering by Removing All Transaction Log Files
If you remove all of the log files, you can bypass the -1216 error. If there is no log file header information to be analyzed, a -1216 error cannot occur. When all of the databases are consistent, no log files are required to start the database. (If you examine the Log Required entry in the header of a consistent database, the Log Required entry reads 0-0.)
If you remove the log files, you cause the creation of a new set of log files, starting with E0n000001.log. If you remove all of the log files, immediately take a full backup of the entire storage group. You will not be able to replay log files from earlier backups past the point at which the log sequence was reset.
Recovering Without Removing All Transaction Log Files
To avoid removing all of the log files, you can force recovery, but the method to force recovery differs depending on whether the missing log files are at the beginning or end of the required sequence.
To determine the required sequence, use Eseutil to examine the
Last Consistent value for every database:
eseutil /mh database_name | find /i "consistent"
The following is a sample of the output of this command:
D:\mdbdata>eseutil /mh PRIV.EDB | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,2692,1ED) 04/12/2001 20:07:46
I:\mdbdata<eseutil /mh PRIV.stm | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,2692,1ED) 00/00/1900 00:00:00
E:\mdbdata>eseutil /mh PRIV2.edb | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,2685,171) 04/12/2001 20:07:41
J:\mdbdata>eseutil /mh PRIV2.stm | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,2685,171) 00/00/1900 00:00:00
F:\mdbdata>eseutil /mh PRIV3.edb | find /i "consistent"
State: Consistent
Last Consistent: (0x2ac8,87,1FC) 04/12/2001 20:05:04
K:\mdbdata>eseutil /mh PRIV3.stm | find /i "consistent"
State: Consistent
Last Consistent: (0x2ac8,87,1FC) 00/00/1900 00:00:00
G:\mdbdata>eseutil /mh PRIV4.edb | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,268C,19B) 04/12/2001 20:07:43
L:\mdbdata>eseutil /mh PRIV4.stm | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,268C,19B) 00/00/1900 00:00:00
H:\mdbdata>eseutil /mh PUB.EDB | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,2699,181) 04/12/2001 20:07:46
M:\mdbdata>eseutil /mh PUB.stm | find /i "consistent"
State: Consistent
Last Consistent: (0x2cc7,2699,181) 00/00/1900 00:00:00
The output in the preceding example confirms that all of the files are consistent and that each database file pair (.edb and .stm) is synchronized. A database file pair is considered synchronized if both files are consistent and both files were last shut down at the same time. You can determine whether or not a file pair was shut down together by examining the three numbers in parentheses on the Last Consistent line for each file.
If you use the preceding Pub.edb and Pub.stm output as an example, you can see that both contain (0x2cc7,2699,181) on the Last Consistent line. These numbers designate a log file, a page offset in the log file, and a byte offset in the page. Both of these files were detached from the same log file at the same point; therefore, they are synchronized. The time stamps on the Last Consistent line are irrelevant.
After you confirm that all of the database files are consistent and matched, you must identify the range of log files that is required to start these databases without a -1216 error. This range is from the lowest Last Consistent entry to the highest. In the preceding example, this range of transaction logs is from E0n02ac8.log through E0n02cc7.log, inclusive. Because you are encountering -1216 errors, you probably do not have all of these logs available. Your next steps depend on whether you are missing the first log, the last log, or a log in the middle of the range.
Recovering If Log Files Are Missing from the Beginning of the Required Range
If you are missing log files at the beginning of the required range, when you try to mount the storage group, you receive an error message that is similar to the following:
Event Type: Error
Event Source: ESE98
Event Category: Logging/Recovery
Event ID: 0
Date: 4/16/2001
Time: 7:53:27 PM
User: N/A
Computer: EXCHANGE1
Description: Information Store (3128) Database recovery on 'D:\exchsrvr\MDBDATA\PRIV4.edb' failed with error -1216. The database is not in the state expected at the first reference of this database in the log files. It is likely that a file copy of this database was restored, but not all log files since the file copy was made are currently available. Please contact PSS for further assistance.
This error message indicates that you will not be able to replay any log data at all into the referenced database. You can only replay log data for a particular database if the last consistent log is present, and then only if there is an unbroken stream of logs past this "anchor" log. Eseutil is also unable to recover the database if there is not a valid checkpoint file.
To recover, move all of the database files that are missing Last Consistent logs to a safe location. You can then recover the remaining databases by simply mounting them. After you start these databases successfully, a valid checkpoint file is generated. The presence of this file allows the other databases to mount normally after they are moved back into place.
Recovering If Log Files Are Missing from the Middle or End of the Required Range
Log files might be missing from the middle or end of the required range if a disaster has destroyed or damaged the current transaction log file drive. You might still have log files that were previously backed up, but you might be missing the most recent log files, or you might be missing log files in the middle of the required set.
If a log file is missing in the middle of the set, all of the subsequent log files are invalidated. The log file before the gap becomes the new end of the available range, and you must remove all higher numbered log files before you run Eseutil.
Soft recovery requires that the highest available log file in the transaction logs folder be named E0n.log. If this is not the case, you receive an error message that is similar to the following:
Event Type: Error
Event Source: ESE98
Event Category: Logging/Recovery
Event ID: 455
Date: 4/24/2001
Time: 5:01:08 PM
User: N/A
Computer: EXCHANGE1
Description: Information Store (736) Error -1811 (0xfffff8ed) occurred while opening logfile D:\exchsrvr\MDBDATA\E00.log.
If you receive this error message, you must rename the highest numbered log file that is available to the name that is listed in the error description.
IMPORTANT: The following instructions apply only if all of the databases that are present are consistent at the time that you perform the procedure.
If you are missing log files at the end of the required range, when you try to mount the storage group, you receive an error message that is similar to the following:
Event Type: Error
Event Source: ESE98
Event Category: Logging/Recovery
Event ID: 0
Date: 4/16/2001
Time: 7:32:52 PM
User: N/A
Computer: EXCHANGE1
Description: Information Store (2548) Database recovery failed with error -1216 because it encountered references to a database, 'D:\exchsrvr\MDBDATA\PRIV3.edb', which is no longer present. The database was not brought to a consistent state before it was removed (or possibly moved or renamed). The database engine will not permit recovery to complete for this instance until the missing database is re-instated. If the database is truly no longer available and no longer required, please contact PSS for further instructions regarding the steps required in order to allow recovery to proceed without this database.
The database that the error message references probably both exists and is consistent, but the database was brought consistent in the "future" of the log file in which the error is encountered. (The database that is listed in the error varies depending on which database in the storage group happens to be listed first in the log file header.)
You can recover all of the databases, along with information in the available log set, by running the
ESEUTIL /R /I /L /S command, as described in the "General Information" section of this article. You must point the
/S parameter to an existing location that does not contain an E0n.chk file. If Eseutil finds a checkpoint file, Eseutil tries to use the information in the checkpoint file to determine where to begin recovery. The likely result is that Eseutil will skip the replay of any log files.