MORE INFORMATION
The Delta Import run profile and the Full Import run profile
Importing from a connected datasource occurs by discovering the connected datasource and processing the selected objects and attributes configured on the management agent. There are two types of imports that can be processed against the connected data source, a delta import and a full import. Descriptions of both types are provided in this article.
Running a delta import
To run a delta import, the connected datasource must have some mechanism to allow delta operations. For example, the Active Directory directory service uses Update Sequence Number (USN) numbers to keep track of changes to the directory. You can use these USN numbers to import only the changes that have been made since the last MA run. Other directories use different mechanisms.Microsoft SQL Server has to have a delta table that is configured with a column that specifies the change type of the transaction to process. Sun ONE directory uses a changelog. If the changelog is enabled on that server, Microsoft Identity Integration Server (MIIS) can read from the changelog and process only what is written to the changelog. Delta imports, and the fact that only changes are processed, are typically more efficient to run than running full imports.
Running a full import
Full imports are typically much less efficient to run than delta imports. This is because running a full import requires importing the whole directory and not a subset of changes. However, there are a number of conditions where you may have to run a full import against the connected datasource:
- The initial MA import must be run as a full import because there is not a watermark on the MA to handle deltas. After the MA has run a full import, the watermark value is set.
- The connected datasource does not support delta imports.
- A USN-based system should run a full import if either of the following is true:
- The directory has been restored from backup.
- The latency between import runs exceeds your tombstone lifetime.
- When discovering against a connected datasource that uses change logs, and the change log has been truncated for some reason.
Run full imports periodically at non-peak times to catch any potential loss of transactions. The frequency of and the requirement for this depends on the connected data source.
Delta Synchronization and Full Synchronization run profiles
During the synchronization phase, MA and metaverse rules are processed. MAs use a set of rules to determine if and how objects in the connector space are synchronized with the metaverse. These rules determine how metaverse objects are created or linked to, how connector space objects are handled after a deletion, and how attributes of a connector space object are synchronized with a metaverse object. The metaverse also uses a set of rules to determine how changes to metaverse objects are provisioned to connector space, and how metaverse objects are handled after they are deleted.
Running Delta Synchronization
To run a Delta Synchronization, the MA processes changes only to those objects and attributes staged in the connector space that have changed. Those objects and attributes are then synchronized with the metaverse or disconnected, depending on rules criteria. Therefore, any object with a pending add, modify, or delete will be evaluated and only attributes with changes will process attribute flow rules and update the metaverse. Any attribute that has not been updated on the object will not process.
Running Full Synchronization
Full Synchronization is used to evaluate and apply synchronization rules to all objects in the connector space. Those objects and attributes are then synchronized with the metaverse or disconnected depending on rules criteria. Therefore, regardless of what attribute has been updated in the connector space or metaverse, all rules will be evaluated and applied to all objects.
Typically, Full Synchronization is required with a rules change.
Multi-step run profile
A multi-step run profile is typically configured to allow multiple operations to be handled by one run profile. A multi-step run profile is typically used in the following situations:
- Export run profiles
- Multiple partitions
Export run profiles
After running an Export to the connected datasource, a delta import and Delta Synchronization should immediately follow to confirm the hologram and make sure that the system state has been confirmed. It is typically easier to add a second step to the Export run profile. By doing this, the Delta Import/Delta Synchronization will be called immediately after the export has been run. If delta imports are not supported for the connected datasource, a Full Import/Delta Synchronization should be configured instead. Combining these run profiles can make scheduling management agent runs less complex.
Multiple partitions
Another common reason for creating a multi-step run profile is the requirement of processing multiple partitions from a connected data source. Import, synchronization, and Export run profile steps must be defined for each partition configured in the management agent. It is much less complex, from a scheduling of MA runs perspective, to add a step for each partition that requires processing per the MA phase.
For example, if you process an Export to an Active Directory forest that has three domains, you create an Export run profile that has three steps, one for each domain partition, and then you create a separate run profile for Delta Import/Delta Synchronization of each domain partition.
Two-step run profiles and one-step run profiles
There are three run profiles that are considered one-step run profiles.
- Delta Import/Delta Synchronization
- Full Import/Delta Synchronization
- Full Import/Full Synchronization
When you run a delta import and Delta Synchronization, you can do Full Import/Delta Synchronization and Full Import/Full Synchronization by using two separate run profile steps, or you can combine them into single-step run profiles, as shown earlier. There are reasons to use either method. Make the decision based on the different functionality that each provides. Three things to consider before making this decision are:
- Processing disconnectors
- The speed of processing
- The ability to look for staged deletions for connectors that are authoritative for object existence
Processing disconnectors
In an MA that does not have projection rules defined, and where the join rules are evaluated and are not met because a corresponding object has not been projected into the metaverse, the object would be remain a typical disconnector. MIIS will re-evaluate a join when a Delta Synchronization is run. For example, if a Delta Import/Delta Synchronization is run, a pending change would have to be imported from the connected datasource for join rules to be applied. In this situation, it is recommended to periodically run a Delta Synchronization to re-evaluate the typical disconnectors to make sure that join conditions are applied. Running a two-step process of Delta Import (stage only) and a then a Delta Synchronization would make sure that typical disconnectors are always evaluated for join every time the delta synchronization is run.
This does not affect Full Import/Full Synchronization as all objects will be processed on a full synchronization.
The speed of processing
There is a difference in performance when running a two-step delta, or full import, and Delta Synchronization, instead of a Delta Import/Delta Synchronization or a Full Import/Delta Synchronization. This difference will vary, based on the number of disconnectors that have to be evaluated under a particular MA.
This does not affect Full Import/Full Synchronization because all objects are processed on a Full Synchronization.
The ability to look for staged deletions for connectors that are authoritative for object existence
If the import and synchronization runs are separated into two run profiles, Windows Management Instrumentation (WMI) code can be used to check the number of pending import deletions before running the synchronization. Checking the number of import deletions in automated MA runs gives the ability to abort the automated run process if a pre-determined threshold is exceeded.
This should be seriously considered if the MA is authoritative for object existence in the metaverse. Determining if the MA is authoritative for object existence depends on the metaverse deletion rules.
A sample WMI script that demonstrates how to calculate the import deletion ratio can be found in the Simple Account Provisioning scenario. The Simple Account Provisioning scenario is located in the Scenarios folder on the product media.
The third-party products that this article discusses are manufactured by companies that are independent of Microsoft. Microsoft makes no warranty, implied or otherwise, regarding the performance or reliability of these products.