SQlite & iOS Crashes

[nplasterer]

Context

iOS app is actually made of 2 processes that need to share data: the app itself, and the Notification Extension

You may want to modify the content of a remote notification on a user’s iOS device if you need to:
Decrypt data sent in an encrypted format.

By default, on iOS, these 2 processes are considered as 2 apps, with sandboxed data.

To share data between those 2, we need them to be part of the same App Group. Apps in the same App Group have access to a “Shared Container”, a place to store shared data.

Sharing data

The goal for the app extension is to decrypt incoming notifications. For that, the app and the extension need to share some data.

In V2 (i.e. our current app) :

• from the app, we export conversations topic data
• we store it (encrypted) using MMKV into the shared container accessible from the app & the extension
• when a notification is received, the notification extension is triggered
• it loads the topic data from shared MMKV, and imports it into an XMTP Client instance
• it decodes the notification and displays it

In V3, it seems that there is no export topic / import topic feature, and that everything is stored inside an encrypted SQLite database managed by libxmtp. So for the notification extension to be able to decode V3 notifications, we might need the SQLite database to be stored inside the Shared Container.

The issue

It is pretty hard to share an SQLite database between two processes on iOS.

We have experienced it firsthand because we currently already have our own SQLite database stored inside the shared container.

All the crash logs from those show it is a 0xDEAD10CC exception.

The reason why the app crashes while you’re not using it is because it is killed by iOS.

The reason it is killed by iOS is because the OS tries to move the app from BACKGROUNDED to SUSPENDED state but the app is actually keeping / opening a lock on the SQLite file (which is usually the case when using SQLite, there is a connection / a pool of connections open between the app & the SQLite file).

Potential fixe(s)

On our current app, we recently tried just not storing the SQLite database in the Shared Container and relying only on MMKV to share data between the 2 processes. We don’t see any crash anymore 👍

How to prevent this in v3?

⇒ Easiest would be to just stop sharing the sqlite db between the app and the extension, but I think v3 relies too much on this database to make it work. We would need to be able to export topic data from the app and reimport it from the extension, but still they would probably be considered 2 installations so we might need to share even more data.

⇒ If not, it seems it is possible to make the SQLite db sharing work.

Indeed, an SQLIte db in WAL mode seems to be the only kind of files that does not trigger 0xDEAD10CC exceptions (any WRITE after suspension still triggers it, see 2.).

So to make it work:

• the OS needs to be able to know it is an SQLite db opened in WAL mode, and if using SQLCipher we also need to make the header plaintext for the OS to determine this. (see 1 & 4)
  • It is a lot simpler to have this from the beginning, because migrating from a non plaintext header to a plaintext header seems to be a pain (see 3)
• the Swift SDK needs to detect when the app is suspended / active again
• during suspension, libxmtp should not trigger any WRITE in the SQLite database but throw an exception (see 2)

More information on this:

1. https://swiftpackageindex.com/groue/grdb.swift/v6.18.0/documentation/grdb/databasesharing#How-to-limit-the-0xDEAD10CC-exception
2. https://github.com/groue/GRDB.swift/blob/master/GRDB/Core/Database.swift#L1172-L1217
3. Important fix for using SQLCipher in shared app container groue/GRDB.swift#302
4. https://www.zetetic.net/sqlcipher/sqlcipher-api/#cipher_plaintext_header_size
5. https://developer.apple.com/documentation/uikit/uiapplication/1623071-didenterbackgroundnotification

[nplasterer]

Step one is going to be to allow unencrypted databases.

[nmalzieu]

To reproduce this crash while debugging the app :

• https://forums.developer.apple.com/forums/thread/126438 (last message)
• https://developer.apple.com/forums/thread/728957

And some helpful lifecycle information:

• https://www.codingexplorer.com/execution-states-swift-ios-app/
• https://stackoverflow.com/a/55118026

Interesting technical details on this specific issue:

• iOS apps are terminated every time they enter the background if they share an encrypted database with an app extension. sqlcipher/sqlcipher#255 (comment)