Mbed TLS is now part of TrustedFirmware.org.

TLS Handshake fails

Jan 10, 2017 13:45
Daniel Sexton

I am working with your mbedtls client example: https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-tls-tls-client/file/05377fa05603/main.cpp

I have modified it to send a request to another host “api-sandbox.mediumone.com” and the TLS handshake fails – the following message is printed out

Starting the TLS handshake...

mbedtls_ssl_handshake() failed: -0x3b00 (-15104): PK - The pubkey tag or value is invalid (only RSA and EC are supported)

I can get a successful ssl connection with the server using curl and the certificate in the example so I know that the server will accept a secure connection. Here is the message received from curl when I look at the certificate chain:

Server certificate
subject=/OU=Domain Control Validated/OU=PositiveSSL Wildcard/CN=*.mediumone.com
issuer=/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain
 Validation Secure Server CA
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
SSL handshake has read 6083 bytes and written 434 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

The mbedtls documentation says you support these algorithms so I don’t understand why I am not making the connection Any suggestions?

Jan 13, 2017 17:29
Andres Amaya Garcia


Browsing through the mbed TLS source code, I think that this is the only line of code that can throw a MBEDTLS_ERR_PK_INVALID_PUBKEY. It seems that there is a failure while verifying the RSA public key (for more information see mbedtls_rsa_check_pubkey()). However, it is difficult to point out the exact cause of the problem. As a starting point to aid debugging, I suggest enabling the debug prints in the application by changing the DEBUG_LEVEL to a higher number (say 3) and manually inspecting the logs.

Finally, I would like to mention that mbed TLS does support a range of cipher suites and features. However, the library is designed to be very modular and the features can be enabled/disabled through a configuration file (see include/mbedtls/config.h). Please bear in mind that it is possible that some of the features that your application requires might be disabled by default in a platform such as mbed OS. I suggest reading the articles linked below and checking the mbed OS config.h for mbed TLS.

I hope this information is useful.

Kind regards, Andres AG, mbed TLS Team Member

Apr 26, 2018 07:58

Hi, was this problem solved?

I have the same problem with Comodo certificate.

Apr 26, 2018 12:33

More details in debug mode:

ssl_tls.c:3531: |3| input record: msgtype = 22, version = [3:3], msglen = 4600
ssl_tls.c:3703: |4| dumping 'input record from network' (4605 bytes)
ssl_tls.c:3703: |4| 0ff0:  00 01 a3 81 f4 30 81 f1 30 1f 06 03 55 1d 23 04  .....0..0...U.#.

ssl_tls.c:3134: |3| handshake message: msglen = 4600, type = 11, hslen = 4600
ssl_tls.c:4179: |3| send alert level=2 message=<b>42</b>

Only first 4092 bytes from 4600 received are presenting in the dump. I tried set MBEDTLS_SSL_MAX_CONTENT_LEN=16384, but result is the same.

Can anybody help me?

Apr 27, 2018 20:47

Solving: #define MBEDTLS_MPI_MAX_SIZE 512

Reason: Long keys in the certificates chain (4096 bits)

Apr 27, 2018 20:50

Only first 4092 bytes from 4600 received are presenting in the dump. I tried set MBEDTLS_SSL_MAX_CONTENT_LEN=16384, but result is the same.

Reason: Const max value in the debug library