Pre-staging the File Replication service replicated files on SYSVOL and Distributed file system shares for optimal synchronization (266679)

MORE INFORMATION

In Windows Server 2003, the Active Directory Installation wizard (Dcpromo.exe) contains a Source from Media feature, which enables the Active Directory to be sourced from a recent copy of the database on a CD-ROM as opposed to performing a full synchronization of the Active Directory over the network.

In Windows 2000 build 2195 and Windows XP, the release of FRS supports a similar feature when the target folders on new replica members are restored by the NTBackup program for any existing members before joining the replica set. This operation can be used on new or reinitialized SYSVOL and Dfs replica members:

1. Set up at least two Dfs alternates, such as, \\Server1\Apps, \\Server2\Apps, and \\ServerX...\Apps.
2. Enable replication only between two replica members, such as, \\Server1 and \\Server2. You can designate any server as primary, but the replicated folders must be empty when the computers are added to the Dfs/FRS replica set.
3. Copy the files destined for the replica set into the replicated \\Server1\Apps folder.
   
   Because \\Server1 has at least one outbound partner (\\Server2), when you copy a file into \\Server1, it causes FRS to generate a staging file and a change order is sent to \\Server2. An MD5 (a hash algorithm) checksum is computed during the staging file generation and the result is saved in the IDTable on \\Server1 and in the change order sent to \\Server2. When \\Server2 processes this change order it saves the MD5 checksum in the IDTable on\\Server2. This process is the only way an MD5 checksum is saved in the IDTable and the use of the MD5 is necessary to avoid overhead when new members are added later.
   
   When step 3 is finished, the replicated files should exist on both \\Server1 and \\Server2, and both IDTables should have MD5 checksums for each file and folder.
4. Use NTBackup or a third-party equivalent to backup the contents of the replica tree from either \\Server1 or \\Server2. NTBackup saves and restores the object identification (ID) attribute associated with each file and folder. Neither the Windows NT nor the MS-DOS copy commands preserve this information when files are copied from \\Server1 to \\Server2. This object ID must be restored with the files when new members are added later.
5. If fewer than seven days have passed since the replica set containing Server1 and Server2 was created, the outbound log must be cleared so that a full vvjoin is triggered when the next member joins.
   
   Note Setting the following registry value to 0 clears the outbound log:

   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NtFrs\Parameters
   Key Name: Outlog Change History In Minutes (REG_DWORD)
   Value:0

6. On \\Server3 and all future replica members, restore the backup to the \\Server3\Apps replicated folder (using the Restore files to menu) to an "alternate location" before you add the computer to the replica set.
7. To enable replication to \\Server3\Apps, FRS on \\Server3 moves all files from the target folder to the pre-existing folder, and then initiates a full synchronization (also referred to as a version vector join operation) from all computers that \\Server3 has inbound Windows NT Directory Services (NTDS) connection objects. In the case of Dfs replica sets with a full mesh topology preferred by the Windows 2000 Dfs snap-in, the sets can include all servers participating in the replica set, such as, \\Server1 and \\Server2. The Windows XP release of the Dfs snap-in supports more optimal topologies including a custom option.
   
   The key requirement in this situation is that \\Server3 has inbound connections from an upstream partner, \\Server1 and \\Server2 in this case, whose IDTABLE contains MD5 checksums for files contained in the replica sets of interest.
   
   FRS on \\Server1 enumerates all the files and folders in its IDTable and sends directed (that is, single target) change orders to \\Server3. Because the IDTable has an MD5 checksum, it is included in the change order. As \\Server3 processes these change orders, this server takes the object ID for the file or folder from the change order and attempts to locate the corresponding file in the pre-existing folder. If the server locates the file, it re-computes the MD5 checksum on the content of that file, compares the result to the MD5 checksum it received in the change order and, if they match, uses the pre-existing file instead of attempting to obtain the file from \\Server1. If \\Server3 does not find the file, or if the MD5 checksum does not match, the server obtains the file from \\Server1. Any change to the file content, such as, to the access control lists, data streams, or attributes can cause an MD5 mismatch and the file is obtained from \\Server1 or other upstream partner.
   
   Meanwhile, FRS on \\Server2 (and all other upstream partners of the new or reinitialized replica member) is performing the same process as \\Server1. \\Server3 processes a change order for a given file or folder from either Server1 or Server2, whichever arrives first. The other change is ignored.
   
   When all replication activity has settled out, the IDTables on all three servers have an identical MD5 checksum and identical file content in the replicated folder. Repeat steps 5 and 6 to add additional servers to the replica set.

Optimizing the Initial or VV Join Process

The current VV join is inherently inefficient. During normal replication, upstream partners build a single staging file, which can source all downstream partners. In a VV join, all computers that have outbound connections to a new or reinitialized downstream partner build staging files designated solely for that partner. If 10 computers do an initial join from \\Server1, the join builds 10 files in stage for each file being replicated. Optimizations to limit the impact of the VV join include:

• Pre-stage content on new members using NTBackup (previously discussed).
• Reduce the number of servers building staging files for new or reinitialized downstream partners.
• Remove or reduce the number of files in the replicated folder until all the computers have completed the VV join phase.
• Enable only one join to occur on a given upstream partner.

Reduce the Number of Servers Building Staging Files for New or Reinitialized Downstream Partners

Replica domain controllers (backup domain controllers) joining existing Windows-based domains attempt to replicate the SYSVOL folder from the same domain controller used to source the Active Directory. The SYSVOL source server is identified in the Replica Set Parent value of the registry. You must confirm that FRS is running and responsive on the designated source server.

For SYSVOL replica sets being reinitialized with a non-authoritative restore (Burflags=D2), administrators can limit the generation of staging files to a specific server by setting the Replica Set Parent registry key to point to a same site or designated staging server.

For more information about the Replica Set Parent registry key, click the following article number to view the article in the Microsoft Knowledge Base: The Replica Set Parent registry key optimization is not possible for Dfs replica members. Possible workarounds include:

• Turning off the FRS service on all possible upstream partners (those computers that the new or reinitialized member has inbound connections from) so that sourcing occurs from the only remaining server.
  
  This work around is not the preferred solution because stopping the FRS service does not stop the accumulation of change records in the NTFS change journal. If the journal wraps (overflows), a VV join is needed when the service is later restarted. If you know that there is negligible or no file modification activity (such as, creates, deletes, renames, updates) on any of the disk volumes hosting a replica set, the risk of journal wraps is likely to be low.
• Remove or reduce the number of files in the replicated folder until all the computers have completed the VV join phase (discussed in the following section).
  
  This operation may not be a viable alternative if the central servers are connected to the new branch servers by means of low bandwidth links and you have gigabytes of file data to initialize. However, you should consider the following option.
• Control the number of inbound connections from source servers available to the new or reinitialized member (that is, the join occurs with a single inbound connection).
• Propagate the data from the hub servers over the weekend or evening hours.
  
  Even with a 64 kilobit link at 75 percent available bandwidth, you can move 21 MB of data each hour or 506 MB each day. With two hub computers and 200 branches connected by means of 64 kilobit links, you can initialize them with 1 GB of content over a two day weekend. If you obtain an average compression ratio of 50 percent, you can move 2 GB of data during a weekend. This operation requires no backup or restore operations, no phasing of branch startup to avoid overwhelming the hub servers, and all progress monitoring can be done from the hub servers using the ntfrs sets command and the Connstat report tool to check for any backlogs to specific branches. The staging space and staging limit parameter on the hub servers must be large enough to hold all the data because the staging file generation can easily outpace the branch data delivery over the slow links.

Remove or Reduce the Number of Files in the Replicated Folder Until all Computers Have Completed the VV Join Phase

Typically, you want all replica set members to join replica sets with empty or nearly empty folders to avoid the inefficient generation of staging files on multiple servers. This process is less of an issue for SYSVOL because the servers are built incrementally, the contents are typically smaller than Dfs shares, and the Replica Set Parent registry key means FRS attempts to source from a single upstream partner.

For large Dfs replica sets, where replication is typically enabled instantly on 2 to 50 servers for tens of gigabytes of content, the impact is greater. Consider adding the majority of the computers to FRS-replicated Dfs shares after they have been deployed. Also, you want the replicated folder on the primary server to be empty so that the VV join occurs without having to replicate files. Files can be added, perhaps incrementally, with normal efficiency.

An empty or minimal VV join can be used to recover a deployment where Active Directory and/or FRS experienced a "melt down" and needs to be reinitialized. After you confirm that Active Directory replication is functional, move files out of the replicated folder on the primary server, and then reinitialize the replica members. In the case of SYSVOL, keep the default domain and domain controller policy in the \Policies folders intact on the primary server (Burflags="D4" or remaining source servers) so that reinitialized domain controllers can replicate in and apply the policy (for example, the "Access This Computer from Network and Other Required Rights" policy) for proper domain and client operation.

Enable Only One Join to Occur on a Given Upstream Partner

For large-sized Dfs replicas containing tens of gigabytes of files, you may consider adding only one member at a time to FRS-replicated folders. Specifically, let the new member complete the full synchronization and move out of VV Join mode. The upstream partner should clean up their files from the staging folder before adding additional members.

In addition, set the staging space limit (defined in the following article, Q221111, "Description of FRS entries in the registry") on all potential source servers equal or greater than the largest 128 files being replicated by the upstream partners (the number of VV joins occurring at any given point).

For more information, click the following article number to view the article in the Microsoft Knowledge Base: