Discreet Log #7: UI Security Testing with Fuzzbot
14 May 2021
The user interface remains the largest and riskiest part of the Cwtch system. While we can (and do) strictly scope the code involved in the Cwtch protocol, we are at the mercy of large UI frameworks when it comes to displaying and interacting with the protocol in a way that is usable and cross-platform.
We can’t vet every single line of code in UI frameworks, and even if we did these frameworks often produce intermediate code during their compilation (e.g. to generate specific code that will work on Window, Linux, iOS or Android) which in turn invokes dynamic libraries that exist on those systems with their own security concerns and considerations.
Fuzz testing allows us to battle harden Cwtch against possible failure cases in a way that is efficient and scalable, and for that we have Fuzz Bot!
Fuzz Bot is a small Cwtch bot whose goal is to break Cwtch. As we expand the capabilities of the UI we also expand the capabilities of Fuzz Bot. As of today Fuzz Bot has several supported fuzz cases:
- Message Content
- UI Overlay Envelopes
- Invitation Overlay Messages
The rest of this post will dive into the workings of each case.
Fuzzing Chat Message Content
Content injection bugs are any bugs that causes user-tainted input to be parsed outside of the widget and scope it was intended for - in many frameworks such bugs can be converted to deanonymization vulnerabilities (e.g. by inserting an element that requests a remote resource, like an image), and as such they are our primary UI security testing concern.
Fuzz Bot provides support for fuzzing “chat” message content via the blns command which stands for big list of naughty strings.
In response to this command FuzzBot sends every “naughty” string as a chat message which we can then check in the UI for rendering glitches, exceptions or content injection. As a side effect blns also tests emoji rendering, the UI handling
of many, different unicode control characters, several unicode scripts and symbols, and the UIs response to receiving
a large number of messages in quick succession.
Fuzz Bot also supports the blns-mutate variant of the command which randomly transforms the malicious input prior to
sending it to the target (see below for more information on mutation).
Fuzzing Overlays and Functionality
Cwtch is intended to be a platform on which different kinds of applications can be implemented. Our original prototype supported chat, lists and a bulletin board.
In the new Cwtch UI group invites and peer recommendations are also implemented as overlays that display distinct actions in the chat pane.
Different applications are specified in the protocol as “overlays”. The UI may respond to different overlay types in different ways. Some of this functionality may be non-trivial (e.g. a message on the list overlay might add an item to a list and also notify the parties in the chat that the list has been updated).
Starting at the most trivial, Fuzzbot provides the random command which constructs completely random envelopes. Fuzzing the overlay envelope itself, a malformed message should not result in arbitrary behaviour and should simply display a malformed message error in the UI.
Fuzzing Invitations
When it comes to fuzzing the specific overlays themselves, support for each overlay needs to be added to Fuzz Bot individually.
We recently added support for fuzzing group invites as these have been promoted to their own mini overlay in the Cwtch UI.
The invite-me <number> command will ask Fuzz Bot to generate a specified number of invitations, to which Fuzz Bot
will randomly insert rogue strings from blns into the Group Name and Group ID.
The future of Fuzz Bot - Larger Corpuses, Overlay Grammars and Mutations
While various iterations of Fuzz Bot have existed throughout the lifetime of the Cwtch project, it is still a rather small bot with a limited feature set. As a Cwtch Beta release nears completion there are several expansions that I would like to add to grow its capabilities and usefulness.
Fuzz Bot currently lacks any way to tell if any fuzzed input resulted in undefined behaviour, and this has three large consequences:
The first is that a human is required to assess the output, watch log files and check to ensure that nothing weird happened.
The second is that Fuzz Bot has no basis on which to intelligently mutate corpus inputs; as such all mutation is random and unguided.
Finally, it means that Fuzz Bot has to ultimately rely on larger input corpuses and manually specified overlay constructions.
None of these are inherently bad, but they do limit our ability to deploy Fuzz Bot as part of our CI pipelines.
In the future I can imagine incorporating some additional debug capabilities into the Cwtch UI that can feed FuzzBot a stream of log messages so that it can better guide mutation choices.
In addition, one of our goals for the second half of this year is to more formally define the overlay protocols for lists and bulletins such that Cwtch clients (including Fuzz Bot) can treat them as a consistent standard, rather than a set of loosely related applications.
The Impact Of Fuzz Bot
Upfront investment in fuzzing infrastructure has paid for itself many times over. In addition to the countless UI glitches that Fuzz Bot has materialized over the years, it has also on occasion found actual security issues (from HTML rendering in our old QML UI to a server crash bug caused by a failure to check a length).
We were able to find these issues during development, rather than hitting them during testing or having people encounter them in the wild.
If you’d like to help us with Cwtch security testing initiatives like Fuzz Bot, then please consider donating! or checking out the code.
