Verification of the server name when alt names are present
yesterday I tried to establish a SSL/TLS communication to a particular e-mail server for the first time. I used mbedtls 2.5.1.
The handshake failed with the flag
MBEDTLS_X509_BADCERT_CN_MISMATCH set, so I sniffed the next session using Wireshark. The certificate looked valid.
Debugging of the session led me to the function
This method checks the
SUBJECT_ALT_NAME extension values, but once it runs out of them, it does not check the subject proper, which is in fact only fetched a few lines before.
And that was precisely problem. The certificate of the server had regular subject
(something).cz, with alt names
exchange.(something).cz, imap.(something).cz, mail.(something).cz.
So as long as I tried to connect just to
(something).cz, the handshake failed, because the subject was not checked, only the alt names, of which none matched.
Once I changed the URL to one of the alt names, the handshake was successful.
I think this is an error. I read the subject alt name extension definition
and it does not seem to imply that the subject is to be repeated among the alt names.
This problem would be fixed by a minor overwrite of the function, should suggest one?
When just looking at RFC 5280 you might come to that conclusion.
Specifically for using X.509 certificates with TLS, you should look at RFC 6125.
In Section 6.4.4, you can read that when a subjectAltName extension is present, CN must be ignored. Underlying reason is that the CN is a pretty human name, and could not mean to actually represent a FQDN at all.
Hope this helps.