We’re up to the fourth technical preview of Windows Server 2016 and my coworkers and I have been digging into it to see what’s coming. Last week I covered the enhancements and new features to Hyper-V, and our licensing specialist Seth made sure to clarify how licensing for Windows Server 2016 is going to be different.
A failover cluster is a group of servers that work together to increase the availability of services and applications. When a failure occurs on one machine in a cluster, the resources get redirected and the workload redistributed to another machine in the cluster. Failover clusters are a great way to ensure that users have nearly constant access to important server-based resources. If one of the nodes fails, another node picks up to provide service, hence “failover.”
Failover clustering in some form has been around for a while, but was called server clusters before Windows Server 2008. In the latest preview of Windows Server 2016, there are quite a few new features to failover clustering.
This new feature of 2016 allows for data at a volume level to be replicated to another site without any lag. Data writes are not acknowledged at the source site until they are acknowledged by the destination side, which means that the two are always fully in sync. If the storage fails at any point in time, the other has a full copy of all acknowledged IOs and can be brought online as needed.
It works over generic SMB3 protocols over the standards everyone is familiar with. There is also an option for asynchronous replication for potential disaster recovery scenarios. This type of functionality is vital for stretch or metro clusters in particular, as both sites need to have an up-to-date copy of the data. As with any significant architecture change, a lot of careful design is required to avoid potential bottlenecks and delays. For example, if the second site has a high latency link, the main site will never be able to write faster than the round-trip latency to the second site.
One of the biggest problems with stretch or metro clusters (separating your datacenter across two geographic areas) is achieving quorum. If I have a datacenter stretched across two separate physical locations, something needs to serve as the third party that arbitrates who is authoritative should the link between them fail. Cloud Witness sets up a third party in Microsoft Azure that functions as a witness for the failover cluster. If site A goes offline, site B can still talk to Azure; if site B goes offline, site A can still talk to Azure. If both sites go offline, neither can talk to Azure and therefore neither is authoritative. It’s a simple way of eliminating one more point of failure.
Site-Aware Failover Clusters
Another major problem with stretch and metro clusters is that the products that run on top of it are ignorant to the fact that they are stretched. This leads to simple things like a host failure causing some servers to boot up in a different site from where they previously ran. The new Site-Aware Failover Cluster allows hosts to be specified as being in a site, so that when small failures occur, the cluster knows to bring those resources back with local resources, rather than failing over to the other site. This also helps a lot with initial placement of virtual machines (VMs) as well as heartbeat, quorum and failover. When coupled with Cloud Witness, this can greatly increase the ability to build stretch Hyper-V clusters.
Virtual Machine Redundancy
In modern, far more resilient datacenters, it’s much less likely for entire environments to fail or massive components to fully disappear. It’s far more likely that something will break for a short length of time and then be fixed (ex: storage controller failover). Because of this, 2016 failover clusters are far more resilient to these types of failures and include two new states for the nodes that participate. There are two states, isolated and quarantined.
In the isolated state, a node has lost access to the other hosts through networking. In this state the host can potentially continue to run VMs, but with some conditions. If the VM is on SMB storage, it will be considered online and healthy (though it has to be managed directly on the host, since the host is unhappy with the rest of the cluster). If the VM is on block storage, since Cluster Shared Volumes (CSVs) are a function of the cluster, the VM will be paused to await the host’s return to the cluster, where it will resume its former state as normal. If the host has consistent problems (ex: leaving the cluster three times in two hours) then the host enters the second state, quarantined. The cluster knows something is awry with the node, so it puts forth an effort to gracefully pull resources off that node while it is functional, and doesn’t let it rejoin the cluster without manual intervention.
Rather than the whole cluster freaking out (for lack of a better term) when storage is lost, in 2016 the VM will pause, keeping its full state intact, and then resume when the storage re-appears. This is very similar to VMware’s feature for when a LUN runs out of storage, it pauses the VM and holds its state until the underlying issue is fixed.
Multi-Domain and Workgroup Clusters
2016 now supports the ability to create a cluster between hosts in multiple domains or in a workgroup. While multi-domain clusters are rarely a serious problem, smaller customers having a domain dependency for their cluster to function is often a serious problem.
For example, say there are only three Hyper-V hosts in a cluster. All VMs, including all Domain Controllers run on these hosts, and there are no other sites or physical servers. If a power outage occurs at this small site, bringing this environment back up can be very tricky. Even though 2012 R2 is supposed to be able to boot the cluster without any Active Directory dependencies, its functionality has been questionable. Full workgroup support means that the hosts can be in a workgroup, fully independent from the environment that runs on top of them, and recovery can be much simpler.