Fix E2EE encryption surviving app restarts
Six interconnected fixes to make Olm/Megolm state persist correctly across restarts: - Fix OlmAccount/OlmSession unpickle argument order (data, key) — was reversed, causing INVALID_BASE64 on every restore - Fix pickle byte encoding: pickle() returns ByteArray, not StringBuffer; encode/decode with ISO_8859_1 - Track publishedOtkCount in SharedPreferences to avoid evicting existing OTKs on restore (only generate MAX - published new keys) - Delay startSync() until after uploadKeys() on fresh login to avoid concurrent OlmAccount access (race condition causing BAD_MESSAGE_KEY_ID) - Persist encryptedRooms to SharedPreferences so rooms stay encrypted across restarts - Check m.room.encryption in both state and timeline events (server returns it in timeline for rooms where encryption was enabled early) Also adds SAS device verification UI, DM naming improvements, and updates state.md with full architecture and debugging notes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
ltadeu6
parent
b30e83898c
commit
3747172488
8 changed files with 781 additions and 58 deletions
85
state.md
Normal file
85
state.md
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
# State - matrix-android
|
||||
|
||||
Current snapshot of the project as of 2026-05-17.
|
||||
|
||||
## What is working
|
||||
|
||||
- NixOS dev shell is available through `flake.nix`.
|
||||
- Android builds work from the CLI with `nix develop -c gradle --no-daemon assembleDebug`.
|
||||
- The app installs and launches over ADB on the test device without wiping user data.
|
||||
- Login, room list, chat, send, and receive flows are in place.
|
||||
- The UI uses the terminal theme and respects the status bar on the main screens.
|
||||
- Direct message naming prefers the other member's display name.
|
||||
- Local 16 KB-compatible `libolm` packaging is integrated.
|
||||
- SAS device verification is implemented and can be initiated from the Android side.
|
||||
- The app exposes a "sync this device" CTA on the home screen when the current device is unverified.
|
||||
- **E2EE encryption survives app restarts** — messages can be sent and received encrypted after closing and reopening the app.
|
||||
|
||||
## Encryption state
|
||||
|
||||
Encryption works across restarts. All three flows verified:
|
||||
- Messages from matrix-commander (CLI) → decrypted on Android after restart ✓
|
||||
- Messages from Android → appear encrypted in Element Web ✓
|
||||
- Messages from Element Web → decrypted on Android ✓ (after Element Web rotates its Megolm session with fresh OTKs)
|
||||
|
||||
### Root causes fixed
|
||||
|
||||
**1. OlmAccount pickle/unpickle argument order**
|
||||
`unpickle(key, data)` was wrong — the Java API is `deserialize(data, key)`. Passing the key bytes as the first argument caused `INVALID_BASE64` since the key contains underscores (invalid base64). Fixed to `unpickle(data, key)` for all three Olm object types (`OlmAccount`, `OlmOutboundGroupSession`, `OlmSession`).
|
||||
|
||||
**2. Pickle byte encoding**
|
||||
`pickle()` returns `ByteArray`, not `StringBuffer`. The original cast `as StringBuffer` crashed or silently failed. Fixed to `String(pickled as ByteArray, Charsets.ISO_8859_1)` for persist and `stored.toByteArray(Charsets.ISO_8859_1)` for restore.
|
||||
|
||||
**3. OTK eviction on restore**
|
||||
`uploadOtks()` on session restore called `generateOneTimeKeys(50)` unconditionally. Since the Olm OTK ring buffer has a fixed max (50), this evicted all existing OTKs. Senders who had claimed old OTKs got `BAD_MESSAGE_KEY_ID`. Fixed by tracking `publishedOtkCount` in SharedPreferences and calling `generateOneTimeKeys(needed)` where `needed = MAX - publishedOtkCount`. This only fills the gap without evicting.
|
||||
|
||||
**4. Login OTK upload race condition**
|
||||
`startSync()` was called immediately in `onLoginSuccess`, before `uploadKeys()` completed. The first sync could report `otkCount=0` and trigger `uploadOtks()` concurrently with `uploadKeys()` — both calling `generateOneTimeKeys` on the non-thread-safe `OlmAccount`. Fixed by moving `startSync()` inside the upload coroutine for fresh login (so sync only starts after keys are uploaded).
|
||||
|
||||
**5. `m.room.encryption` only checked in state events**
|
||||
The server returns `m.room.encryption` in the **timeline** (not in `state`) for this room, because the encryption was enabled early in the room's history and the timeline window covers it. The check was `roomData.state?.events.any { it.type == "m.room.encryption" }` — always false. Fixed to also check timeline events.
|
||||
|
||||
**6. `encryptedRooms` not persisted**
|
||||
`encryptedRooms` was in-memory only, reset on every restart. After restart, the room appeared unencrypted and messages were sent as plaintext until the first sync populated it. Fixed by persisting to `matrix_session` SharedPreferences under key `encrypted_room_ids`, loaded on init before sync starts.
|
||||
|
||||
### Remaining known issue
|
||||
|
||||
Element Web (`e6CqkKos`) keeps sending `BAD_MESSAGE_KEY_ID` PRE_KEY messages. This is because it has cached OTKs from earlier broken sessions (before the fix) that were evicted from the account. Element Web will recover automatically when it rotates its outbound Megolm session (after ~100 messages or 7 days), at which point it re-claims fresh OTKs. New sessions (like matrix-commander) work immediately.
|
||||
|
||||
## Architecture notes
|
||||
|
||||
- `MatrixSession` (singleton) owns all runtime state. Always via `MatrixSession.getInstance(context)`.
|
||||
- `CryptoService` holds and persists: `OlmAccount`, inbound Megolm sessions, outbound Megolm sessions, Olm 1:1 sessions. All in `olm_crypto` SharedPreferences.
|
||||
- `uploadKeys()` — uploads device keys + OTKs. Called only on **fresh login**. Causes `device_lists.changed`.
|
||||
- `uploadOtks()` — uploads only OTKs (no `device_keys`). Called on **session restore** and when sync reports OTK count ≤ 9. Does NOT cause `device_lists.changed`. Only generates `MAX - publishedOtkCount` new keys to avoid eviction.
|
||||
- `encryptedRooms` — persisted in `matrix_session` prefs. Populated from both `state` and `timeline` events in sync.
|
||||
- `logout()` wipes both `matrix_session` and `olm_crypto` prefs.
|
||||
|
||||
## Test infrastructure
|
||||
|
||||
- Android emulator: Pixel_3a_API_35 (emulator-5554), launched via `start-avd` from the nix dev shell.
|
||||
- matrix-commander account: `@claudetestacc:matrix.org` (credentials in `.mc/credentials.json`, store in `.mc/store/`).
|
||||
- Test room: `!rNfgFvyFMQEWIQnYVV:matrix.org` (shared between `@claudetestacc` and `@ltadeu616`).
|
||||
|
||||
## Files to watch
|
||||
|
||||
- `app/src/main/java/com/ltadeu6/matrix/matrix/MatrixSession.kt`
|
||||
- `app/src/main/java/com/ltadeu6/matrix/matrix/CryptoService.kt`
|
||||
- `app/src/main/java/com/ltadeu6/matrix/matrix/VerificationService.kt`
|
||||
- `app/src/main/java/com/ltadeu6/matrix/rooms/RoomListScreen.kt`
|
||||
- `app/src/main/java/com/ltadeu6/matrix/verification/VerificationScreen.kt`
|
||||
|
||||
## Build / deploy
|
||||
|
||||
```bash
|
||||
nix develop -c gradle --no-daemon assembleDebug
|
||||
adb install -r app/build/outputs/apk/debug/app-debug.apk
|
||||
```
|
||||
|
||||
`adb install -r` preserves user data. Use `logout()` in-app to wipe crypto state.
|
||||
|
||||
To send a test message from matrix-commander:
|
||||
```bash
|
||||
nix develop -c matrix-commander --credentials .mc/credentials.json --store .mc/store \
|
||||
-m "test message" --room '!rNfgFvyFMQEWIQnYVV:matrix.org'
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue