Given our large and growing backlog of bugs, it has become apparent that our current approach to defect management is insufficient. We are trying a new tack. Learn more about how we are changing our approach and why.
We are embarking on a new journey (see previous blog post). The new Mbed TLS 3.0 will separate crypto primitives into their own project, with a clear aim to support as much crypto hardware as possible. We will soon introduce a complete re-engineering of the TLS layer to make it more modular and enhance support for lossy networks. Those changes are intentionally aimed at opening the gates for community contributions, whether they are crypto drivers, or additional network protocols, or enhancements to the existing ones, and everything else the Mbed TLS community cares about.
Looking at the backlog of GitHub issues reveals no less than 200 issues and contributions waiting for integration work. This backlog of work has slowly piled up and has become largely unmanageable, requiring some urgent decision to help us move forward.
In many cases, the contributions we received were built against a part of the code that has evolved too much for patches to be applicable without a great deal of work. Time alone raises the bar for contributions to be integrated, only because they are not reviewed as often as they should. The slower we react, the more we create work for ourselves.
What's changing and why?
There are at least two ways to run a bug tracker:
One can use it as a complete inventory of all potential bugs ever encountered, with all non-resolved issues as "open"
A list of bugs that one plans to fix and address, with all "open" bugs considered important enough to fix
We are migrating from the first approach to the second approach. This is for a few reasons:
Inventory has a cost
We spend time a lot of time prioritizing and managing our bug list. For a mature project such as Mbed TLS, this list is very large as it's a complete inventory of all potential bugs ever encountered. Prioritizing and scheduling fixes for a long list of bugs takes a long time. This time is better spent fixing important bugs that our community cares about. Without a manageable number of bugs, it's not practical to decide which bugs to fix and when, so it's in everyone's best interest to keep the number of open bugs manageable.
We prefer to be honest about what we are capable of fixing and don't want to give the impression that we are working towards fixing all known bugs. We'll close more bugs as "won't fix" if we know we won't be able to prioritize spending time to fix them. A bot will mark issues as "Archived" if they haven't had activity in over a year, signaling that we are not realistically able to prioritize work on fixing the bug and that there is little interest in having the bug fixed.
Pull requests are still welcome for archived issues. Pull requests are the best way to communicate the importance of a bug, since it shows enough care to create a fix and to work with us to land the fix. When raising a pull request for an archived issue please link to and ask for the issue to be re-opened, to signal active work on the issue.
We are not deleting any bugs, so one can still use our bug tracker as a list of all potential bugs ever encountered. No information is lost. The reason an issue is closed will be clearly labelled, and all issues remain searchable (whether closed or not).
We are introducing the following labels to indicate the reason a bug was closed:
wontfix - The cost of fixing the bug or managing it in the bug list is not justified.
notabug - The bug is not considered to be a defect, but by design.
archived - The bug has not had activity in over a year, on either the community interest front or the able to fix it front, and is being closed
Archiving is blunt
We recently reviewed and closed a number of issues on GitHub. One of our criteria for closing was based on activity. Any issues that have not seen any interest or progress for over a year have been archived. Archiving is, of course, a very blunt tool and we have probably set aside some topics worthy of attention. Anything important enough can be unarchived, made into an active issue, and reprioritized.
If you are reading this, you are probably an Mbed TLS contributor. If you raised an issue on GitHub and found it to be recently closed, please ask for the issue to be re-opened it with a valid justification: why is this needed? what does it offer? how important is this to you and your projects?
Please give bugs you care about a pull request or at least a thumbs up. If you care about a bug that gets archived, please ask for the issue to be re-opened. We use PRs, thumbs up, and re-opened status as an indication of interest in having a bug fixed and prioritize accordingly.
Mbed TLS would not be where it is today without the many contributions we receive on a daily basis. We cannot thank the community enough for their invaluable contributions. We want to enhance our thanks by providing the support you are legitimately expecting, and this slate-cleaning exercise is part of our commitment.
If you have feedback on this new approach, feel free to contact us to share your thoughts. We take stewardship of this project seriously and want to do what's best for future of Mbed TLS.
Please bear with us and keep up the good work!