Discreet Log #11: Integrating FFI processes with Android services

In Discreet Log #9, Dan talked about how we use FFI to connect our Flutter frontend to our Go backend Cwtch library on both desktop and Android platforms. In the leadup to the release of the Cwtch Beta last week, we finished the Android service work mentioned in that post and are going to go over how it works in this week’s edition of Discreet Log!

In addition to needing to make plain ol’ method calls into the Cwtch library, we also need to be able to communicate with (and receive events from) long-running Cwtch goroutines that keep the Tor process running in the background, manage connection and conversation state for all your contacts, and handle a few other monitoring and upkeep tasks as well. This isn’t really a problem on traditionally multitasking desktop operating systems, but on mobile devices running Android we have to contend with shorter sessions, frequent unloads, and network and power restrictions that can vary over time. As Cwtch is intended to be metadata resistant and privacy-centric, we also want to provide notifications without using the Google push notification service.

The solution for long-running network apps like Cwtch is to put our FFI code into an Android Foreground Service. (And no, it’s not lost on me that the code for our backend is placed in something called a ForegroundService.) With a big of finagling, the WorkManager API allows us to create and manage various types of services including ForegroundServices. This turned out to be a great choice for us, as our gomobile FFI handler happened to already be written in Kotlin, and WorkManager allows us to specify a Kotlin coroutine to be invoked as the service.

If you’d like to follow along, our WorkManager specifications are created in the handleCwtch() method of MainActivity.kt, and the workers themselves are defined in FlwtchWorker.kt. These links go to the latest versions of the files, which may have been updated since this post was written to incorporate feedback and data from our Beta testers; at the time of writing, we were on commit 9bb23de09078d813a2620e5c6dc982d3bc5b358d.

Our plain ol’ method calls to FFI routines are also upgraded to be made as WorkManager work requests, which allows us to conveniently pass the return values back via the result callback.

One initial call (aptly named Start) gets hijacked by FlwtchWorker to become our eventbus loop. Since FlwtchWorker is a coroutine, it’s easy for it to yield and resume as necessary while waiting for events to be generated. Cwtch’s goroutines can then emit events, which will be picked up by FlwtchWorker and dispatched appropriately.

FlwtchWorker’s eventbus loop is not just a boring forwarder. It needs to check for certain message types that affect the Android state; for example, new message events should typically display notifications that the user can click to go to the appropriate conversation window, even when the app isn’t running in the foreground. When the time does come to forward the event to the app, we use LocalBroadcastManager to get the notification to MainActivity.onIntent. From there, we in turn use Flutter MethodChannels to forward the event data from Kotlin into the frontend’s Flutter engine, where the event finally gets parsed by Dart code that updates the UI as necessary.

Messages and other permanent state are stored on disk by the service, so the frontend doesn’t need to be updated if the app isnt open. However, some things (like dates and unread messages) can then lead to desyncs between the front and back ends, so we check for this at app launch/resume to see if we need to reinitialize Cwtch and/or resync the UI state.

Finally, while implementing these services on Android we observed that WorkManager is very good at persisting old enqueued work, to the point that old workers were even being resumed after app reinstalls! Adding calls to pruneWork() helps mitigate this, as long as the app was shut down gracefully and old jobs were properly canceled. This frequently isn’t the case on Android, however, so as an additional mitigation we found it useful to tag the work with the native library directory name:

    private fun getNativeLibDir(): String {
        val ainfo = this.applicationContext.packageManager.getApplicationInfo(
                "im.cwtch.flwtch", // Must be app name
                PackageManager.GET_SHARED_LIBRARY_FILES)
        return ainfo.nativeLibraryDir
    }

…then, whenever the app is launched, we cancel any jobs that aren’t tagged with the correct current library directory. Since this directory name changes between app installs, this technique prevents us from accidentally resuming with an outdated service worker.

Cwtch Beta update

Since the launch of the Cwtch Beta, tonnes of people have tried out Cwtch and given us feedback that has already started making it into our nightly releases. (We don’t know exactly how many, thanks to Cwtch’s metadata-resistant architecture!) Bug reports are immensely helpful to us as a small group of nonprofit developers, and help ensure Cwtch is stable on as many devices as possible. We have already gotten feedback that the new Beta UI is much more stable and fluid on older/slower devices than the Alpha builds were, which is very rewarding to hear. Fixes have already started getting pushed, and we have a roadmap for the Cwtch 1.1 release now.

As ever, if you’d like to support Open Privacy’s efforts to develop Cwtch and bring open source metadata-resistant and privacy-first infrastructure to marginalized communities, please consider donating.