Skip to main content
Hyper-VWindowsWindows Server 2012Windows Server 2019Windows Server 2022

Windows Updates Cause DC Boot Loops and Hyper-V Failure

By February 9, 2022No Comments

Microsoft released several patches in last weeks January 2022 Patch Tuesday that fixed some serious vulnerabilities. You can look at the entire list of updates here.

The Mirazon Group has seen some of these updates cause severe outages caused by domain controllers being stuck in repetitive boot loops, Hyper-V failing to start, and inaccessible ReFS volumes. These issues continue until the updates are removed / rolled back.

Windows Domain Controller Boot Loops

After having installed these updates, Windows domain controllers enter an endless boot loop every few minutes after reaching the login screen. This appears to be related to the LSASS.exe process; the process consumes the CPU and eventually terminates, resulting in a reboot.

The LSASS process is critical to the Windows Operating System to operate and thus will reboot when this process crashes.

If you manage to grab event viewer logs quick enough or have them centrally stored, you will find the following:

The process wininit.exe has initiated the restart of computer [computer_name] on behalf of user for the following reason: No title for this reason could be found Reason Code: 0x50006 Shutdown Type: restart Comment: The system process ‘C:\WINDOWS\system32\lsass.exe’ terminated unexpectedly with status code -1073741819. The system will now shut down and restart.

Hyper-V cannot Start Guest VMs

This is a big one. Once a Hyper-V host receives these updates and reboots, the host will no longer be able to start the Hyper-V Services. Therefore, no guest VMs will run.

When attempting to manually start a guest VM, you will receive the following error:

Virtual machine xxx could not be started because the hypervisor is not running.

ReFS becomes inaccessible after updates

We have seen multiple Windows Resilient File System (ReFS) volumes that are no longer accessible, or are seen as RAW (unformatted) after installing these updates.

Go figure, the resilient file system is brought down by it’s creator. NTFS attached volumes appear to be unaffected by these updates.

Out of Band Updates to Fix

On January 18th, Microsoft released OOB updates in an attempt to resolve these wide spread issues.

All OOB updates can be downloaded on the Microsoft Update Catalog here. Some of them can also be installed directly through Windows Update as optional updates.

The following updates can only be downloaded and installed via the Microsoft Update Catalog:

Updates for these Windows versions are also available through Windows Update as an optional update:

Uninstall Updates to Fix

The only other option is to entirely remove the Windows Updates and reboot the host. Due to the way that Microsoft bundles all security patches into a single rollup, removing the cumulative updates to fix the aforementioned issues will also remove the fixes that Microsoft implemented.

Run the following commands within an elevated Command Prompt to manually remove these updates:

  • Windows Server 2012 R2:

wusa /uninstall /kb:5009624

  • Windows Server 2012 R2:

wusa /uninstall /kb:5009595

  • Windows Server 2012:

wusa /uninstall /kb:5008604

  • Windows Server 2019:

wusa /uninstall /kb:5009557

  • Windows Server 2022:

wusa /uninstall /kb:5009555

These commands can be run from Safe Mode as well.

If, however, you are not able to get the host stable enough to issue a Command Prompt, we have been successful with booting to installation media and accessing Command Prompt via the Recovery Console.

The commands are different within this environment and level DISM to achieve the same result.

You will have to used the following command in order to pull the package name:

dism /online /get-packages

It may be useful to pipe this to the clipboard and paste into Notepad:

dism /online /get-packages | clip

Once you have the package name, you can construct an uninstall command as such:

dism /online /Remove-Package /PackageName:Package_for_KB5008604~31bf3856ad364e35~amd64~~9200.23518.1.0

 

If you have any additional questions or concerns, please call 502-240-0404 or send us an email at info@mirazon.com