What Components are Involved?This error involves two sides: a "client" and a server. Client is in quotes because it can be, and often is, an application consuming a web service or similar. On the server side this problem generally occurs on Windows 2008 or newer. The "client" can be any platform.
What Errors Again?
Generally, but not always, these errors are manifested into following events:
- System Log, Schannel source, EventID 36888
- System Log, Schannel source, EventID 36874
These errors can occur on either side, provided obviously that side is Windows. What errors you receive on the other side depend entirely on the platform.
What is Happening?At a high level, the client and server are failing to agree on a way to talk to each other securely. To communicate securely, the server and client must agree on a methodology to communicate involving 4 main components. Those are:
- How to authenticate each other (Key Exchange)
- How to encrypt data to be exchanged (Encryption Cipher)
- How to verify the message hasn't been tampered with (Message Authentication Code)
- How to determine random numbers for seeding keys (Pseudorandom Function)
The client and server must agree to the same implementation of each of these items. Bundled together, these are referred to as a cipher suite.
The client and server each have preferences as to which portions of the cipher suite hold which priority. Based on this prioritization, a set of supported cipher suites is compiled and proposed at the beginning of any SSL/TLS connection. The client first proposes what it would like, then the server compares the client list to its own list and selects the first matching suite.
So therein lies the problem: Your server doesn't like any of the proposals from the client.
This is why I decided to write this article. While there are several hits on the internet regarding this problem, I have yet to see one that nails it. Initially (and originally published in this article) I suspected the problem was due to an incorrect cryptographic service provider but thanks to some insights from one of my colleagues I took another look. Turns out that due to the nature of this problem it can appear sporadically and be difficult to troubleshoot. If you're experiencing this problem the following may be true of your environment:
- Internal CA (Certificate Authority)
- You're using certreq.exe to create a CSR (Certificate Signing Request)
- Your template for the CSR is "quite short"
If those things aren't true, don't worry because here are the details: If your CSR is requesting a certificate that is valid for signing only, rather than signing and encryption, and your CA has a policy that allows for encryption even when the request was signing only, then you will likely see this problem... sometimes. Clearly a certificate requested for signature only shouldn't work at all when used for encryption, but if your CA overrides the request to allow for encryption that will create a situation where encryption will work, but only under circumstances when the client supports a couple specific protocol suites. Identifying certificates causing this problem is complicated; since the CA overrode the We'll cover the specifics further in the next two sections...
Detecting The Problem
Feel free to skip this section if you want to jump to the fix. Detection can be pretty easy using tools like Wireshark. Fire up the tool on either the client or server with the proper capture filters to reduce noise, and then attempt the failing connection. You will see only a handful of packets (5 or so) as the rejection happens pretty quickly. To see the detail appropriately, you'll need to tell Wireshark this is SSL/TLS by right clicking->decode as->SSL.
If a protocol negotiation is the issue, you'll see the connection reset by the server immediately after the client suggests a list of cipher suites. This packet from the client will have the info of "client hello" followed immediately with a TCP RST (reset) from the server.
If you drill into the details of the "client hello" packet you will be able to see the suites the client is proposing.
You can then attempt a successful TLS connection if you are able to produce one (if not just jump to the fix and try it) using the same methodology. I found that while using the affected cert type listed above, my server only supported TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA and TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, clearly a very limited subset.
To remediate this issue you'll need to make sure that certificate ordered is for the correct purpose. Rather than recreate that article I'll direct you to my favorite one here, however note that the [strings],[Extensions],and [RequestAttributes] sections may not be needed depending on your situation. The main takeaway from that article is that at the very least the KeySpec and KeyUsage settings need to be specified (see link under references for more info). Request, retrieve, and install this certificate.
You can use any other method you would like to obtain a certificate (perhaps you do), but it's critical to ensure your request has the correct parameters including the certificate usage. If you are using Windows PKI with AD integrated templates, you can "hard code" this in the templates if you like.
If this fix didn't work for you, wait for the "Wait There's More" section because it's likely due to a misconfigured set of cipher suites. Speaking of that...
Wait There's More
As a security best practice, you should also control (restrict) your available cipher suites on Windows/IIS. This is pretty easy to do; it can be done via Group Policy for large sets of servers and one-by-one with registry settings or better yet with this easy tool from Nartac. For more guidance check out these three links.
Thanks for reading and feel free to add your own experience below!