Multicast Support Enabled for the Cluster Heartbeat (307962)

SUMMARY

Windows Server 2003 cluster nodes exchange multicast heartbeats rather than unicast heartbeats. In Microsoft Windows NT 4.0 and Microsoft Windows 2000, all heartbeat messages were unicast.

Multicast networking refers to transmitting network data to a group of interfaces instead of a single destination interface. Only the other interfaces in the same multicast group can use the transmitted data. Using multicast technology permits several unicast messages to be replaced with a single multicast message. Unlike broadcast, multicast traffic does not have to be flooded throughout the network, reducing the chance that unnecessary CPU cycles are expended filtering traffic on nodes outside the cluster. As the number of nodes increases, the ability to replace several unicast messages with a single multicast message improves node performance and decreases network bandwidth consumption.

Clusters that are comprised of fewer than three nodes will not send multicast heartbeats. The Cluster service also does not send multicast heartbeats in mixed version clusters. For instance, a four-node Windows Datacenter Server cluster that has three nodes running Windows Server 2003 Datacenter Server and the remaining node running Windows 2000 Datacenter Server reverts to unicast heartbeats. Otherwise, the Cluster service automatically configures and uses multicast for the cluster heartbeats by default.

The cluster Network Driver (Clusnet.sys) is the core cluster driver that has been rewritten to support multicast. Multicast is not used for all intracluster communications. Multicast messages replace unicast messages in two components of Windows Clustering:

• Heartbeats: The Windows Clustering failure detection engine is based on a scheme whereby nodes send heartbeat messages to other nodes. Specifically, for each network interface, a node sends a heartbeat message to all other nodes with interfaces on that network. Heartbeat messages are sent every 1.2 seconds. In the common case where each node has an interface on each cluster network, there are N * (N - 1) unicast heartbeats sent per network every 1.2 seconds in an N-node cluster. With multicast heartbeats, the message count drops to N multicast heartbeats per network every 1.2 seconds, because each node sends 1 message instead of N - 1. This represents a reduction in processing cycles on the sending node and a reduction in network bandwidth consumed.
• Regroup: The Windows Clustering membership engine executes a regroup protocol during a membership view change. The regroup protocol algorithm assumes the ability to broadcast messages to all cluster nodes. To avoid unnecessary network flooding and to properly authenticate messages, the broadcast primitive is implemented by a sequence of unicast messages. Converting the unicast messages to a single multicast message will conserve processing power on the sending node, reduce network bandwidth consumption, and more accurately reflect the intentions of the algorithm.

Configuration

Multicast configuration primarily involves selecting and agreeing upon a multicast group address for each cluster network. A multicast group address can be any IPv4 class D address (224.0.0.0 to 239.255.255.255). The multicast group address can be configured in one of three ways:

• From a Multicast Address Dynamic Client Allocation Protocol (MADCAP) server
• Automatically by self-selecting a multicast address
• Manual configuration by the administrator

From a MADCAP Server

If a cluster administrator did not manually configure a multicast address, the cluster tries to obtain a multicast address from a MADCAP server. MADCAP servers lease multicast addresses much like DHCP servers lease unicast addresses.

The following is the procedure that the Cluster service uses to obtain a multicast address when there is a MADCAP server on the network:

1. Send a MADCAP request to obtain a list of multicast scopes.
   
   A scope is a range of multicast addresses that are configured so that traffic that is sent to these addresses can be limited to a subset of the network. A MADCAP server may offer addresses in several scopes. When you request a MADCAP address, the cluster must specify the scope from which the address should be selected.
2. From the scopes advertised by MADCAP servers, determine those that are valid for the cluster network.
   
   A scope is valid for the cluster network only if it meets both of the following criteria:
   • The scope is available on the local interface for this network (as opposed to a different network interface on this node).
   • The scope is not in the range of multicast addresses reserved for single-source traffic (232.*.*.*).
3. Rank the valid scopes that remain according to the following criteria:
   • Scope falls in the administrative local scope (239.255.*.*)
   • Proximity of MADCAP server with regard to the cluster node (same subnet as cluster node)
   • Scope has the largest range of addresses of those enumerated
   • Scope has the lowest time-to-live attribute (TTL) of those enumerated
   
   
4. Request an address lease from each scope in ranked order until a lease is obtained. Much like a DHCP assigned address, the MADCAP assigned address has an expiration time by which the lease must be renewed.
   
   If no MADCAP server responds or if there are no valid scopes with available leases, the Cluster service automatically selects an address.

Automatically by Self-Selecting a Multicast Address

If an administrator did not specify an address and if MADCAP requests fail, the Cluster service automatically selects an address. The Cluster service randomly selects an address from a subset of all IPv4 class D addresses. The range of multicast addresses to use for self-selection is determined as follows:

1. Check the cluster registry to see if administrator specified a multicast address range. If no range was specified, the administrative local scope (239.0.0.0 to 239.255.255.255) [RFC 2365] is chosen as the default range.
2. Apply the following exclusions to the address range:
   • Upper 256 addresses in both the administrative local scope and the administrative organizational scope, which are reserved for relative allocations as per RFC 2365 [RFC 2365]
   • The single-source scope (232.*.*.*)
   • Any valid scopes recently obtained from a MADCAP server
   If the range of addresses for self-selection is not empty after the exclusions are applied, the Cluster randomly selects an address. The Cluster configures self-selected addresses to expire in 24 hours in case a MADCAP server is deployed on the network (or a lease becomes available).

Manual Configuration by the Administrator

An administrator can manually configure a multicast address for a cluster network. If a class D IPv4 address is manually configured, that address is used without question. A manually configured address persists, even if the cluster service is stopped and restarted, until the administrator manually configures a different address. If an administrator specifies an invalid multicast address (for example, not an IPv4 class D address), the cluster service tries to automatically configure a valid multicast address.

An administrator can also disable multicast either on the whole cluster or for individual networks so that heartbeat traffic reverts to unicast. If multicast is disabled for a network, there will be no attempt to obtain a multicast address and the Cluster service uses unicast until the network is reconfigured.

An administrator configures multicast by modifying the private properties of individual networks and the cluster. To view the private properties for a network with regard to the multicast configuration; use the /privproperties parameter without any options. For example, to see the heartbeat multicasting properties for the cluster network PrivNet, type the following at the command prompt: The following table lists all of the configuration parameters for a particular cluster network with regard to multicasting. The writable column indicates whether or not the parameter can be set with Cluster.exe.

Name	Writable	Description
MulticastAddress	Yes	Sets the Class D IPv4 multicast address for the network. If the address is not a valid Class D IPv4 address (224.0.0.0 to 239.255.255.255), the Cluster ignores the specified address and tries to automatically obtain a valid address.
MulticastDisabled	Yes	Controls whether or not heartbeat multicasting is used for two classes of intracluster traffic (heartbeats and node-to-node communication to verify node failures during cluster configuration changes) for this network. This setting overrides the cluster MulticastClusterDisabled setting for the entire cluster. The choices are: 0 - This is the default. This enables heartbeat multicasting for this cluster network. 1 - This disables heartbeat multicasting for this cluster network. If MulticastDisabled is missing, a value of 0 is the default
MulticastConfigType	No	0: Manual configuration 1: MADCAP configuration 2: Self-configuration
MulticastLeaseServer	No	MADCAP Server from which multicast address was obtained, undefined for manual configuration and self-selection
MulticastLeaseRequestId	No	Lease ID of the multicast address, undefined for manual configuration and self-selection
MulticastLeaseObtained	No	UTC time that lease was obtained or address was self-selected, undefined for manual configuration
MulticastLeaseExpires	No	UTC time that lease expires or self-selection expires, undefined for manual configuration
MulticastAddressRangeLower	Yes	Lower bound (inclusive) of range from which a multicast address should be chosen if MADCAP server not found - if not specified, will default to 239.255.0.0 Note: This lower address range will only be used if the following conditions are met: Values for both MulticastAddressRangeLower and MulticastAddressRangeUpper are specified and the addresses supplied for MulticastAddressRangeLower and MulticastAddressRangeUpper are valid Class D IPv4 addresses.
MulticastAddressRangeUpper	Yes	Upper bound (inclusive) of range from which a multicast address should be chosen if MADCAP server not found- if not specified, will default to 239.255.254.255 Note: This upper address range will only be used if the following conditions are met: Values for both MulticastAddressRangeLower and MulticastAddressRangeUpper are specified and the addresses supplied for MulticastAddressRangeLower and MulticastAddressRangeUpper are valid Class D IPv4 addresses.

The MulticastClusterDisabled parameter, which is writable, indicates whether multicast should be used for all cluster networks. Each cluster network MulticastDisabled setting overrides the global cluster MulticastClusterDisabled setting. The Cluster Service has to be restarted for the global disable to take effect. The table below describes whether or not heartbeat multicasting is disabled for a specific network as determined by the MulticastDisabled value for that network and the MulticastClusterDisabled value for the entire cluster:

MulticastClusterDisabled (Global to the entire cluster)	MulticastDisabled (per cluster Network)	Multicast on the Heartbeat?
Not Set	Not Set	Yes
Not Set	0	Yes
Not Set	1	No
0	Not Set	Yes
0	0	Yes
0	1	No
1	Not Set	No
1	0	Yes
1	1	No

* - "Not Set" implies that the parameter is absent

Examples

The following examples are sample uses of Cluster.exe to handle multicast settings:

To turn off heartbeat multicasting for the cluster network named PrivNet, type the following at the command prompt: To set the multicast address for network PrivNet to 224.1.2.3, type the following at the command prompt: To set the range of multicast addresses for network PrivNet from which an address should be automatically chosen when no MADCAP server is detected to 239.255.0.0-239.255.0.255, type the following at the command prompt: To disable multicast for the whole cluster, type the following at the command prompt:

Key for Signing and Verifying Messages

The final step for the multicast configuration is agreeing on a key for signing and verifying messages. The key must be secret, but it must be agreed upon by all cluster nodes. Periodically, for each cluster network where multicast is not disabled, one cluster node generates a large random number to serve as the new multicast key. That key is encrypted and distributed to all the other cluster nodes. When each node receives the new key, each node stops signing messages with the old key. Multicast keys are never recorded in the cluster database and are always encrypted before distribution across a network.