Examining SymCrypt DoS Vulnerability

Early last week, Tavis Ormandy released a new DoS vulnerability affecting the SymCrypt library. While it provided some details, I decided to look at it a little further and realized it makes for a nice quick beginner tutorial on using IDA’s debugger and verifying a vulnerability. We start by picking the relevant details from the original release, then verifying that the vulnerability exists using the provided POC, and finish by looking at some other concerns for those attempting to patch the vulnerability. We will also discuss why this bug is such a pain for the industry.

This post’s target audience is beginner reverse engineers. If you aren’t so concerned with RE but are interested in this bug, I’d recommend reading the first two sections and then skipping to API Bugs are Difficult.

Extracting relevant info

Bug information can be found on Project Zero’s website here.  There are three relevant pieces of information present that we need to focus on:

  1. The vulnerability affects the SymCrypt library, specifically bcryptprimitives!SymCryptFdefModInvGeneric. This is the Microsoft symbol format (e.g., when using Windbg). The part before “!” is the affected library, and the part after the “!” is the function containing the vulnerability.
  2. The vulnerability is an infinite loop that results in a DoS in certain MS Windows applications, possibly requiring a reboot. In third-party applications, it may cause the application to deadlock.
  3. To test the vulnerability, use C:\> certutil.exe testcase.crt

We won’t necessarily focus on these elements in order, but each represents a particular string we need to pull: What the vulnerability affects, the kind of vulnerability, and how to test it.

Background research

By research, I mean, “looking around the interwebs using your favorite search engine.” Searching for SymCrypt yields the SymCrypt Github. Presumably, this is the source code for Microsoft’s bcrypt library. In many cases, we may not have the source code, but we lucked out, so clone the repo, and search around for FdefModInvGeneric, and you’ll eventually find the function inside of the fdef_mod.c file.

We know from the published vulnerability details that we’re looking for an infinite loop, so we don’t necessarily need to read every single line of code in this function. Instead, we’re looking for loop constructs (and possibly gotos). Eventually, you’ll find this:

FdefModInvGeneric For loop
FdefModInvGeneric Vulnerable Loop

There’s a for-loop with no loop condition. It starts with a while-loop and based on the output of that for-loop, it decides whether or not to exit the for-loop. The for-loop extends to the end of the function. Thus if either the while-loop never completes, or the break statement never executes, this function will not return. Likely, this structure is the source of the infinite loop.

It’s also important to keep in mind that the source code we’re looking at may not be the version of source code that was compiled to produce the DLLs on the affected versions of Windows. So hopefully, the code structure will be similar, but there are no guarantees. Further, verification of this vulnerability on a particular version of Windows doesn’t imply that this specific source code is vulnerable. We’d have to compile and test this source separately to be sure.

Quick Aside: Mathematics (scary!)

You can verify this vulnerability without knowing much about what these algorithms are supposed to implement, but you won’t necessarily understand why the function loops infinitely, or even perhaps how to fix it. Especially when it comes to cryptography, that’s a pretty reasonable situation to find yourself in. If you are curious about the purpose of this function and others in this library, this writeup by Conrado Gouvêa on Montgomery Reductions is a good start. If you don’t have time to read that post but are still faintly curious, the TL;DR is that calculating modulo is expensive if you use division. Thus, a real smart mathematician created a different way, called Montgomery Reductions, to calculate modulo using only multiplies and adds which are more easily optimized.

Testing the Vuln (Basic)

(Note: the following was done on a Windows 10 64 bit VM).

From the source code, we have a vague idea of what we’re looking for, and from the vulnerability disclosure, we know both the library (bcryptprimitives.dll), and also how to test the vuln (certutil.exe). So first, let’s try the given test case by running certutil on the modinv.cer file from the vuln disclosure:

Basic Run Test
Testing the Vuln with Certutil

It tries to display part of a signature, and then hangs indefinitely. If we weren’t focused on RE, that would be the end of it, but I imagine you didn’t read this far for a command prompt screenshot. So, let’s see what happens in the debugger since it’s better to be sure that the infinite loop is indeed inside of the FdefModInvGeneric function.

Testing the Vuln (IDA)

The first step is to find the affected DLL. We know from the vuln disclosure that the function is inside of bcryptprimitives.dll. Standard Windows DLLs are typically located in System32, so search for it in there, and copy that file to wherever you’d like to work with it. Load that file into IDA, let IDA download the symbols, and locate SymCryptFdefModInvGeneric.You can either use the functions view that’s by default on the left of the IDA-view, or hit Ctrl-P to bring up the functions window and search.

Click Ok, and you should see something similar to this:

SymCryptFdefModInvGeneric Function Start
SymCryptFdefModInvGeneric Function Start

Zoom out of the graph view, and you’ll find a structure very similar to what we saw in when we looked at the SymCrypt source code. There’s a large loop at the end of the function with one exit out of the loop.

FdefModInvGeneric Function view
Control flow breakdown in IDA

Since we want to test if this loop ever exits, we’ll set a breakpoint at the target of the jump. Press F2 on that line, and it will become red to denote the breakpoint. On my system, the address is 0x1800142BB. If this code indeed infinitely loops as reported, then the target of the jump will never hit, and we’ll never be able to confirm that the function or the vulnerable code even executed. To make sure that code execution makes it to this loop, we’ll also set a breakpoint at the conditional jump that governs whether to continue looping. On my system, this address is at 0x1800141C6. If you want to be extra careful, you should also set a breakpoint at the function start and return.

FdefModInvGeneric Function Breakpoints
Put breakpoints on conditional jump and jump target

Next, we need to set up our debugger. Select “Local Windows Debugger”, since we’ll be running the test case on the same system that IDA is executing on. Next, in IDA’s top-level menu bar select Debugger, then Process Options. This window lets us tell IDA what application we want to execute, and what we intend to debug. In this case, that’s the bcryptprimitives.dll. You should set up your process options similar to the following image.

IDA's Process Options
Process Options Setup
  • Application – The binary to execute, in this case, it’s certutil.exe, which is the test case published by Project Zero.
  • Input File – This is the binary that was used to create the IDB. In this case, that’s the bcryptprimitives DLL.
  • Directory – The directory to launch the application from.
  • Parameters – Command line options to pass to the Application. Here we pass modinv.cer, since we need to provide the input to trigger the vulnerability.

Once the Process Options are set up, we’re ready to debug. Hit the green play button at the top, or hit F9, and IDA will execute certutil.exe passing in the modinv.cer. Sometime during process startup, you will see a popup similar to the following.

Screen Shot 2019-06-18 at 4.04.09 PM

IDA is trying to figure out which loaded DLL matches the DLL we used to create the IDB. It needs this info to map that DLL onto our IDB, with addressing that matches the process memory. Once IDA knows the address of the target DLL, it can populate our function names, comments, and other annotations to display at the correct location in our debugger view. More importantly, this allows our breakpoints to get set at the right address. Since we copied the DLL we are analyzing from System32, select “Same”, and let the load and follow-on execution continue. Eventually, execution should pause, and one of the lines we set a breakpoint on will turn purple. This line coloring is IDA’s visual way of telling the analyst that execution has paused on a line with a breakpoint.

Breakpoint hit
Execution Paused at the conditional jump

Execution paused at the conditional jump inside of the big loop in FdefModInvGeneric. This breakpoint hitting tells us that this function’s code indeed executes. If you hit play again (F9), you’ll see that execution pauses at the same spot. At minimum, returning to this conditional tells us that this loop’s dependent functions all return for the first iteration.

To further convince yourself, remove the breakpoint from the conditional jump (F2), and hit play. The breakpoint set at the jump target never triggers. If you bring up the certutil application that IDA started as part of this debugging session, you’ll see that the application paused in the same place as when we ran it previously outside the debugger.

Screen Shot 2019-06-18 at 4.20.40 PM

Let this run while you contemplate the difficulty of the Halting Problem, and once you’ve convinced yourself that this is an infinite loop, exit the debugger. If you’d like to experiment some more, step through each instruction in the loop to make sure each function indeed returns correctly, and the loop isn’t getting stopped by one of its dependent functions.

API bugs are difficult

We’ve verified that there’s a user-mode infinite loop in the bcryptprimitives.dll, and this is a big problem, to say the least. When developers write software, they rely on these DLLs and exported functions (APIs), like bcryptprimitives.dll and its parent bcrypt.dll, so that they don’t have to re-write every single piece of code needed to interact with the operating system. A Windows API is a “contract guarantee” between Microsoft and the developer community: “Trust us. We know how to do this operation. If you pass us these inputs, we promise to provide these outputs”.

Developers choose to rely on these functions, and trust that when their code executes on a Windows system, Microsoft’s implementation of an exported function to behave as promised. (If you’d like to know more about how this process works, read up on Dynamic Linking, and Import Address Tables.) A library like bcrypt that implements common cryptographic operations such as signature verification is a dependency of a wide variety of software beyond just Microsoft developed code. Tavis refers to this fact in the vuln disclosure:

Obviously, lots of software that processes untrusted content (like antivirus) call these routines on untrusted data, and this will cause them to deadlock.

This bug isn’t just a Windows Denial of Service; it’s also potentially a denial of service against any piece of software that relies on that particular API. On the bright side, an API bug means that Microsoft only needs to fix and patch one bug, and as a result, all the dependent software should work as intended. However, in this case, there’s an extra wrinkle.

Common Source code means common problems

Microsoft Windows doesn’t just implement these cryptographic algorithms in user-mode. The kernel also exports these cryptographic operations for drivers to use. This library is documented on MSDN here. As an example API, look at BCryptVerifySignature. In the remarks section, it says:

To call this function in kernel mode, use Cng.lib, which is part of the Driver Development Kit (DDK).

If you look in, System32\drivers, you’ll find a corresponding cng.sys. Here are some of its functions.

Screen Shot 2019-06-18 at 5.17.55 PM

At this point, these function names should look familiar, in particular SymCryptFdefModInvGeneric. So it appears that both the user-mode and kernel-mode crypto libraries are dependent on a version of SymCrypt source (not necessarily the same version though).

I compared the kernel level implementation of SymCryptFdefModInvGeneric with the user-mode one, and they appear to be very similar in structure. Obvious things were different such as calling KeBugCheck on failure, and not so obvious things. For example, the kernel version only uses general purpose registers, while the user-mode version uses SSE registers in certain places. It is possible that this change in implementation means that cng .sys’ version is unaffected. For lack of time, I didn’t test to see if the kernel version was explicitly vulnerable.

Microsoft, on the other hand, will almost certainly make time (at least I hope) to test for a corresponding kernel level DoS bug, as such a bug could have cascading effects on the operating system. I wouldn’t be surprised if the patch to fix this changes both cng.sys and bcryptprimitives.sys.


We’ve walked through how to verify the SymCrypt DoS vulnerability, and looked into the potential impact of such a bug. Since the bug affects a core Microsoft DLL, it has the potential to DoS not only Microsoft written software but also any third-party supplied software that relies on bcryptprimitives!SymCryptFdefModInvGeneric. Since cng.sys also relies on a version of SymCrypt source code, there exists the potential for a kernel-mode vulnerability in cng!SymCryptFdefModInvGeneric. More than likely, Microsoft is testing this possibility as well, and if no public reports are released testing this possibility, we’ll be able to tell when the July patch gets released.