PolarSSL is now part of ARM Official announcement and rebranded as mbed TLS.

Testing the DTLS release

Adding a new protocol to an existing product is not trivial. In order to make sure we can have assurance in the new DTLS protocol, the team did extensive testing and even created new tools in the process for our automated testing framework.

The branch that adds support for DTLS 1.0 and 1.2, was released recently as a preview, still under the old PolarSSL 1.4 name. As every released version, it was thoroughly tested.

In addition, for this version we again expanded our usual automated testing framework.

UDP proxy emulating unreliable connection

The main feature of DTLS, compared to TLS, is that it is able to run over an unreliable connection (with UDP, packets might be lost, re-ordered or duplicated). Since we run our tests on the loopback interface, which happens to be 100% reliable even with UDP, we wrote a UDP proxy to emulate these issues. This is standard practice, recommended in this paper by the original authors and implementors of DTLS, and used by other libraries.

In this proxy, we chose to use a combination of fully deterministic options (to make sure we get exactly the expected reaction to some particular events, such as the ChangeCipherSpec message being delayed) and pseudo-random "issues", to cover interesting conditions that would not happen with simpler behaviour such as "drop every Nth packet".

We believe this has been effective, because during development, we actually ran into many corner cases. Our proxy has an option to specify a seed for the pseudo-random event, so that when we ran into an issue, we were able to reproduce it reliably until it was solved.

The use proxy is integrated in our test script ssl_opt.sh for automated testing with various options and handshake flows. Here is a sample output (in case you're curious, nbio stands for non-blocking I/O, AD for Application Data and FS for Forward Secrecy):

DTLS proxy: reference .................................................. PASS
DTLS proxy: duplicate every packet ..................................... PASS
DTLS proxy: duplicate every packet, server anti-replay off ............. PASS
DTLS proxy: inject invalid AD record, default badmac_limit ............. PASS
DTLS proxy: inject invalid AD record, badmac_limit 1 ................... PASS
DTLS proxy: inject invalid AD record, badmac_limit 2 ................... PASS
DTLS proxy: inject invalid AD record, badmac_limit 2, exchanges 2 ...... PASS
DTLS proxy: delay ChangeCipherSpec ..................................... PASS
DTLS proxy: 3d (drop, delay, duplicate), "short" PSK handshake ......... PASS
DTLS proxy: 3d, "short" RSA handshake .................................. PASS
DTLS proxy: 3d, "short" (no ticket, no cli_auth) FS handshake .......... PASS
DTLS proxy: 3d, FS, client auth ........................................ PASS
DTLS proxy: 3d, FS, ticket ............................................. PASS
DTLS proxy: 3d, max handshake (FS, ticket + client auth) ............... PASS
DTLS proxy: 3d, max handshake, nbio .................................... PASS
DTLS proxy: 3d, min handshake, resumption .............................. PASS
DTLS proxy: 3d, min handshake, resumption, nbio ........................ PASS
DTLS proxy: 3d, min handshake, client-initiated renego ................. PASS
DTLS proxy: 3d, min handshake, client-initiated renego, nbio ........... PASS
DTLS proxy: 3d, min handshake, server-initiated renego ................. PASS
DTLS proxy: 3d, min handshake, server-initiated renego, nbio ........... PASS
DTLS proxy: 3d, openssl server ......................................... PASS
DTLS proxy: 3d, openssl server, fragmentation .......................... PASS
DTLS proxy: 3d, openssl server, fragmentation, nbio .................... PASS
DTLS proxy: 3d, gnutls server .......................................... PASS
DTLS proxy: 3d, gnutls server, fragmentation ........................... PASS
DTLS proxy: 3d, gnutls server, fragmentation, nbio ..................... PASS
------------------------------------------------------------------------
PASSED (27 / 27 tests (0 skipped))

Testing interoperability and finding bugs in other implementations

The first goal of testing against other widely-used implementations is to make sure we don't have bugs or misinterpretations of the standard that would prevent us from interoperating properly with them. During the development phase, it regularly helped pinpoint problems with our development code.

However, sometimes it also happens to uncover bugs or issues with the implementations against which we're testing. We do not mean to pick on them (every large enough piece of software has bugs) but we do believe this speaks for the quality of our testing process.

Specifically, during DTLS development, we reported the following bugs, which were quickly acknowledged and fixed by the developers of the respective libraries:

  • OpenSSL #3449: ECC (ie ECDHE and ECDSA) could not be negotiated with DTLS.
  • OpenSSL #3452: PSK and -Verify: discrepancy between TLS and DTLS.
  • OpenSSL #3453: s_server: -www & Co break DTLS connections.
  • GnuTLS: Client failed to retransmit in some case when session tickets were in use. (Mailing-list thread.)

Fuzzing with Codenomicon Defensics

While this is not strictly related to DTLS, it happens we're currently in the process of testing our code with the great fuzzing suites developed by Codenomicon. We're still in the preliminary phase, which mean coverage is probably not maximal yet, but we're happy to report that our DTLS server has successfully survived its first encounters with the fuzzer.

The server used was built with ASan, so surviving the tests means it never performed out-of-bounds access (read or write) to a buffer (either on the heap, the stack, or global) and never used a local variable after return or a dynamic buffer after it was freed, and did not leak any memory.

We're working on improving coverage and running more extended test sessions, and we'll report on this in some time.

Like this?

Section:
Blog

Author:


Published:


Last updated:
Jul 24, 2015

Sharing:


Want to stay up to date?