Discreet Log #17: Filesharing FAQ
01 Oct 2021
Welcome to Discreet Log! A fortnightly technical development blog to provide an in-depth look into the research, projects and tools that we work on at Open Privacy. For our 17th edition Erinn Atwater answers some questions about the new filesharing feature in Cwtch 1.3.
Today’s release of Cwtch 1.3 includes support for a new experimental feature: file sharing! Windows, Mac, Linux and Android users can enable the File Sharing experiment from the global settings panel. Files are sent peer-to-peer directly over Cwtch/tor connections. In this week’s Discreet Log, I thought I’d answer some questions about the finer points of how it all works.
Note that this is specific to the 1.3 release, as we have many plans to expand on the feature in future releases. If you found our work useful or you’d like to help speed it along, please consider donating to Open Privacy.
How does file sharing with groups work? Are my files stored on a server somewhere?
Files are sent through onion-to-onion Cwtch connections directly from the person offering the file to the person receiving it. The initial offer to send a file is posted as a standard Cwtch conversation/overlay message. For groups, this means that the initial offer (containing the filename, size, hash, and a nonce) is posted to the group server, but then each recipient connects to you individually to receive the actual file contents.
Does that mean I have to be online to send a file?
Yes. If the person offering the file goes offline, you will have to wait for them to come online to resume the file transfer. The underlying protocol splits the files into individually-requestable, verifiable chunks, so that in a future release you will be able to “rehost” a file posted to a group, and even download from multiple hosts at once (sort of like bittorrent).
Why are new contacts popping up in my list?
This is due to how Cwtch currently handles connections from unknown addresses. Since posting a file to a group results in group members connecting to you directly, some of those members might not be in your contact list already and so their download connection to you will appear in your list as a contact request.
What is “SHA512”?
SHA512 is a cryptographic hash that can be used to verify that the file you downloaded is a correct copy of the file that was offered. Cwtch does this verification for you automatically, but you’re welcome to try it yourself! Note that we also include a random nonce with file offers, so people can’t just ask you for any random hash you might have, or files from conversations they’re not part of.
Why is it so slow? Is there protocol overhead?
The primary bandwidth bottleneck is tor, as connections happen through hidden service connections (with some additional crypto layered on top). There is currently a ~30% overhead on file size due to base64 encoding in the protocol, which we hope to eliminate soon. There is also an additional ~1% overhead resulting from file chunk manifests.
If a transfer seems particularly slow or struggling, you can try resetting the tor daemon from the onion panel to get a new hidden service circuit.
Is there a file size limit?
The current limit is 10 gigabytes per file.
What are these .manifest files?
The .manifest files are used while downloading the file to verify that individual chunks are received correctly, and support resuming interrupted transfers. They also contain the info from the original file offer. You can safely delete them once the download is complete. On Android, the manifests are stored in the app’s cache, and can be cleared through your system settings.
What about file metadata?
We send the file’s name as a suggestion and to help distinguish it from other file offers. The full path is stripped before sending the offer. You should be wary of hidden metadata that might be stored in the file itself, which varies depending on the file’s format. For example, images might contain geolocation info and information about the camera that took them, and PDF files are notorious for containing hidden information such as the author’s name or the machine they were created on. In general, you should only send and receive files with people you trust.
Can I preview images? Audio clips?
Not at this time, but it is on our roadmap. Previews have been known to cause metadata leaks and security vulnerabilities in other apps, so we want to give this feature extra scrutiny before releasing it, even hidden behind the “experimental” gate. Once we’re satisfied, it can even be used to power cool new features like custom profile pictures and stickers.
Why does the Android app request new permissions?
Previously we had no need to read or write files outside app local storage. While we do support the new Android 11/API 30 method of requesting individual file permissions already, the Play Store beta (now available to Patrons) is currently built from the same source as the APK on cwtch.im, which is intended to support older devices in addition to sideloading.
Can I download files automatically?
We have no plans to include such a feature at this time.