matrix-android/state.md
ltadeu6 3747172488 Fix E2EE encryption surviving app restarts
Six interconnected fixes to make Olm/Megolm state persist correctly
across restarts:

- Fix OlmAccount/OlmSession unpickle argument order (data, key) — was
  reversed, causing INVALID_BASE64 on every restore
- Fix pickle byte encoding: pickle() returns ByteArray, not StringBuffer;
  encode/decode with ISO_8859_1
- Track publishedOtkCount in SharedPreferences to avoid evicting existing
  OTKs on restore (only generate MAX - published new keys)
- Delay startSync() until after uploadKeys() on fresh login to avoid
  concurrent OlmAccount access (race condition causing BAD_MESSAGE_KEY_ID)
- Persist encryptedRooms to SharedPreferences so rooms stay encrypted
  across restarts
- Check m.room.encryption in both state and timeline events (server
  returns it in timeline for rooms where encryption was enabled early)

Also adds SAS device verification UI, DM naming improvements, and
updates state.md with full architecture and debugging notes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 15:21:09 -03:00

5.8 KiB

State - matrix-android

Current snapshot of the project as of 2026-05-17.

What is working

  • NixOS dev shell is available through flake.nix.
  • Android builds work from the CLI with nix develop -c gradle --no-daemon assembleDebug.
  • The app installs and launches over ADB on the test device without wiping user data.
  • Login, room list, chat, send, and receive flows are in place.
  • The UI uses the terminal theme and respects the status bar on the main screens.
  • Direct message naming prefers the other member's display name.
  • Local 16 KB-compatible libolm packaging is integrated.
  • SAS device verification is implemented and can be initiated from the Android side.
  • The app exposes a "sync this device" CTA on the home screen when the current device is unverified.
  • E2EE encryption survives app restarts — messages can be sent and received encrypted after closing and reopening the app.

Encryption state

Encryption works across restarts. All three flows verified:

  • Messages from matrix-commander (CLI) → decrypted on Android after restart ✓
  • Messages from Android → appear encrypted in Element Web ✓
  • Messages from Element Web → decrypted on Android ✓ (after Element Web rotates its Megolm session with fresh OTKs)

Root causes fixed

1. OlmAccount pickle/unpickle argument order unpickle(key, data) was wrong — the Java API is deserialize(data, key). Passing the key bytes as the first argument caused INVALID_BASE64 since the key contains underscores (invalid base64). Fixed to unpickle(data, key) for all three Olm object types (OlmAccount, OlmOutboundGroupSession, OlmSession).

2. Pickle byte encoding pickle() returns ByteArray, not StringBuffer. The original cast as StringBuffer crashed or silently failed. Fixed to String(pickled as ByteArray, Charsets.ISO_8859_1) for persist and stored.toByteArray(Charsets.ISO_8859_1) for restore.

3. OTK eviction on restore uploadOtks() on session restore called generateOneTimeKeys(50) unconditionally. Since the Olm OTK ring buffer has a fixed max (50), this evicted all existing OTKs. Senders who had claimed old OTKs got BAD_MESSAGE_KEY_ID. Fixed by tracking publishedOtkCount in SharedPreferences and calling generateOneTimeKeys(needed) where needed = MAX - publishedOtkCount. This only fills the gap without evicting.

4. Login OTK upload race condition startSync() was called immediately in onLoginSuccess, before uploadKeys() completed. The first sync could report otkCount=0 and trigger uploadOtks() concurrently with uploadKeys() — both calling generateOneTimeKeys on the non-thread-safe OlmAccount. Fixed by moving startSync() inside the upload coroutine for fresh login (so sync only starts after keys are uploaded).

5. m.room.encryption only checked in state events The server returns m.room.encryption in the timeline (not in state) for this room, because the encryption was enabled early in the room's history and the timeline window covers it. The check was roomData.state?.events.any { it.type == "m.room.encryption" } — always false. Fixed to also check timeline events.

6. encryptedRooms not persisted encryptedRooms was in-memory only, reset on every restart. After restart, the room appeared unencrypted and messages were sent as plaintext until the first sync populated it. Fixed by persisting to matrix_session SharedPreferences under key encrypted_room_ids, loaded on init before sync starts.

Remaining known issue

Element Web (e6CqkKos) keeps sending BAD_MESSAGE_KEY_ID PRE_KEY messages. This is because it has cached OTKs from earlier broken sessions (before the fix) that were evicted from the account. Element Web will recover automatically when it rotates its outbound Megolm session (after ~100 messages or 7 days), at which point it re-claims fresh OTKs. New sessions (like matrix-commander) work immediately.

Architecture notes

  • MatrixSession (singleton) owns all runtime state. Always via MatrixSession.getInstance(context).
  • CryptoService holds and persists: OlmAccount, inbound Megolm sessions, outbound Megolm sessions, Olm 1:1 sessions. All in olm_crypto SharedPreferences.
  • uploadKeys() — uploads device keys + OTKs. Called only on fresh login. Causes device_lists.changed.
  • uploadOtks() — uploads only OTKs (no device_keys). Called on session restore and when sync reports OTK count ≤ 9. Does NOT cause device_lists.changed. Only generates MAX - publishedOtkCount new keys to avoid eviction.
  • encryptedRooms — persisted in matrix_session prefs. Populated from both state and timeline events in sync.
  • logout() wipes both matrix_session and olm_crypto prefs.

Test infrastructure

  • Android emulator: Pixel_3a_API_35 (emulator-5554), launched via start-avd from the nix dev shell.
  • matrix-commander account: @claudetestacc:matrix.org (credentials in .mc/credentials.json, store in .mc/store/).
  • Test room: !rNfgFvyFMQEWIQnYVV:matrix.org (shared between @claudetestacc and @ltadeu616).

Files to watch

  • app/src/main/java/com/ltadeu6/matrix/matrix/MatrixSession.kt
  • app/src/main/java/com/ltadeu6/matrix/matrix/CryptoService.kt
  • app/src/main/java/com/ltadeu6/matrix/matrix/VerificationService.kt
  • app/src/main/java/com/ltadeu6/matrix/rooms/RoomListScreen.kt
  • app/src/main/java/com/ltadeu6/matrix/verification/VerificationScreen.kt

Build / deploy

nix develop -c gradle --no-daemon assembleDebug
adb install -r app/build/outputs/apk/debug/app-debug.apk

adb install -r preserves user data. Use logout() in-app to wipe crypto state.

To send a test message from matrix-commander:

nix develop -c matrix-commander --credentials .mc/credentials.json --store .mc/store \
  -m "test message" --room '!rNfgFvyFMQEWIQnYVV:matrix.org'