element-android/docs/_developer_onboarding.md

Developer on boarding

• Introduction
  • Quick introduction to Matrix
    • Matrix data
      • Room
      • Event
    • Sync
      • Glossary about syncs
  • The Android project
  • Matrix SDK
  • Application
    • MvRx
      • Behavior
    • Epoxy
    • Other frameworks
  • Push
  • Dependencies management
  • Test
  • Other points
    • Logging
    • Rageshake
  • Tips
• Happy coding!

Introduction

This doc is a quick introduction about the project and its architecture.

It's aim is to help new developers to understand the overall project and where to start developing.

Other useful documentation:

• all the docs in this folder!
• the contributing doc, that you should also read carefully.

Quick introduction to Matrix

Matrix website: matrix.org, discover page. Note: Matrix.org is also hosting a homeserver (.well-known file). The reference homeserver (this is how Matrix servers are called) implementation is Synapse. But other implementations exist. The Matrix specification is here to ensure that any Matrix client, such as Element Android and its SDK can talk to any Matrix server.

Have a quick look to the client-server API documentation: Client-server documentation. Other network API exist, the list is here: (https://spec.matrix.org/latest/)

Matrix is an open source protocol. Change are possible and are tracked using this GitHub repository. Changes to the protocol are called MSC: Matrix Spec Change. These are PullRequest to this project.

Matrix object are Json data. Unstable prefixes must be used for Json keys when the MSC is not merged (i.e. accepted).

Matrix data

There are many object and data in the Matrix worlds. Let's focus on the most important and used, Room and Event

Room

Room is a place which contains ordered Events. They are identified with their room_id. Nearly all the data are stored in rooms, and shared using homeserver to all the Room Member.

Note: Spaces are also Rooms with a different type.

Event

Events are items of a Room, where data is embedded.

There are 2 types of Room Event:

• Regular Events: contain useful content for the user (message, image, etc.), but are not necessarily displayed as this in the timeline (reaction, message edition, call signaling).
• State Events: contain the state of the Room (name, topic, etc.). They have a non null value for the key state_key.

Also all the Room Member details are in State Events: one State Event per member. In this case, the state_key is the matrixId (= userId).

Important Fields of an Event:

• event_id: unique across the Matrix universe;
• room_id: the room the Event belongs to;
• type: describe what the Event contain, especially in the content section, and how the SDK should handle this Event;
• content: dynamic Event data; depends on the type.

So we have a triple event_id, type, state_key which uniquely defines an Event.

Sync

The Sync is a way for the Matrix client to be up to date regarding the user data hosted by the server. All the Events are coming through the sync response. More details can be found here: spec.matrix.org/v1.3/client-server-api/#syncing When the application is in foreground, this is a looping request. We are using Https requests, which offer the advantage to be compatible with any homeserver. A sync token is used as request parameter, to let the server know what the client knows. The SyncThread is responsible to manage the sync request loop.

When the application is in background, a Push will trigger a sync request.

Glossary about syncs

• initial sync: a sync request without a token. This is the first request a client perform after login or after a clear cache. The server will include in the response all your rooms with the full state (all the room membership Event will not be present), with the latest messages for each room. We are in the process to replace this by version 3: sliding sync. All data are inserted to the Database (currently Realm).
• incremental sync: sync request with a token.
• gappy sync: sync request where all the new Events are not returned for one or several Rooms. Also called limited sync. It can be limited per Room. To get all the missing Events, a Room pagination API has to be called.
• sync token: next_batch value in the previous sync response. Will be provided as the since parameter for the next sync request.

The Android project

The project should compile out of the box.

The project is split into several modules. The main ones are: For the app

• vector-app: application entry point;
• vector: legacy application, but now a library. In the process of being split into several modules;
• vector-config: this is where all the configuration of the application should occurs. Should because we are in the process of migrating all the configuration here;
• library/ui-strings: this is where all the string resources are stored. Please refer to contributing doc to know how to make change on this module;
• library/ui-styles: this is where the Android styles are defined.

For the SDK

• matrix-sdk-android: the main SDK module. The sources are in this project, but are also exported to its own project. All the PRs and issues related to the SDK take place in the Element Android project;
• matrix-sdk-android-flow: contains some wrapper to expose Flow to the application.

Matrix SDK

SDK exposes Services to the client application. Services are public interface, and are defined in this parent package: org.matrix.android.sdk.api. Default implementation are internal to the SDK, in this parent package: org.matrix.android.sdk.internal. Note that you also have to declare the classes as internal when adding classes to the org.matrix.android.sdk.internal package.

Interface allows us to replace the implementation for testing purpose.

A generated documentation of the SDK is available here. Updated after each release. Please ensure that the documentation (KDoc) of all the SDK Services is up to date, and is clear for a SDK user. The SDK generated documentation also contains information about the entry points of the SDK.

Dagger is used to inject all the dependencies to the SDK classes.

SDK is exposing data as LiveData, but we are progressively migrating to Flow. Database is the source of truth.

Example:

• Client send an Event using the SendService;
• At the end a SendEvent task is used;
• Retrofit API is used to send data to the server;
• Goes to the server, which returns only the event_id;
• The Event is coming back from the sync response with eventually extra added data.

Application

This is the UI part of the project.

There are two variants of the application: Gplay and Fdroid.

The main difference is about using Firebase on Gplay variant, to have Push from Google Services. FDroid variant cannot contain closed source dependency.

Fdroid is using background polling to lack the missing of Pushed. Now a solution using UnifiedPush has ben added to the project. See refer to the dedicated documentation for more details.

MvRx

Maverick (or MvRx) is an Android MVI framework that helps to develop Reactive application on Android.

• Activity: holder for Fragment. See the parent VectorBaseActivity;
• Fragment: manage screen of the application. See the parent VectorBaseFragment;
• BottomSheet: see the parent VectorBaseBottomSheetDialogFragment;
• ViewModel: this is where the logic is placed. All our ViewModel has a handle() which takes action as parameter. See the parent VectorViewModel;
• VectorSharedActionViewModel: Specific ViewModel that can be used to communicate between Fragment(s) and the host Activity. See the parent VectorSharedActionViewModel;
• ViewState: this are data class, and this represent the state of the View. Has to be copied and set to be updated. Fragment will update the UI regarding the current state (invalidate() method). Async class from MvRx can be used in the ViewState, especially for asynchronous data loading. Nullability can also be used for optional data. ViewStates have to implement MavericksState;
• ViewEvents: useful when the ViewModel asks the View to trigger a specific action: navigation, show dialog, etc. See the parent VectorViewEvents;
• ViewAction (VectorViewModelAction): useful when the UI (generally the Fragment) asks the ViewModel to do something. See the parent VectorViewModelAction;
• Controller: see the Epoxy section just below.

Behavior

Fragment asks the ViewModel to perform an action (coming from the user, but not necessarily. ViewModel can then talk to the SDK, updates the state once or several times. Fragment update the UI regarding the new state.

When ViewModel is instantiated, it can subscribe using the SDK Services to get live state of the data.

invalidate() has to be used by default, but it's possible to listen to specific member(s) of the ViewState using onEach. TODO Add an example. awaitState() method

Epoxy

Epoxy is an Android library for building complex screens in a RecyclerView. Please read the introduction.

• Controller declares items of the RecyclerView. Controller is injected in the Fragment. Controller extends EpoxyController, or one of its subclass, especially TypedEpoxyController;
• Fragment gives the state to the controller using setData;
• buildModels will be called by the framework;
• Controller will create ordered Items.

Epoxy does the diffing, and handle many other thing for us, like handling item type, etc.

See for instance the controller AccountDataEpoxyController) for a simple example.

Warning: do not use twice the same item id or it will crash.

Other frameworks

• Dependency injection is managed by Dagger (SDK) and Hilt (App);
• Retrofit and OkHttp3: network requests;
• Moshi is used to parse and serialize Json object;

Push

Please see the dedicated documentation for more details.

This is the classical scenario:

• App receives a Push. Note: Push is ignored if app is in foreground;
• App asks the SDK to load Event data (fastlane mode). We have a change to get the data faster and display the notification faster;
• App asks the SDK to perform a sync request.

Dependencies management

All the dependencies are declared in build.gradle files. But some versions are declared in this dedicated file.

When adding a new dependency, you will have to update the file dependencies_groups.gradle to allow the dependency to be downloaded from the artifact repository. Sometimes sub-dependencies need to be added too, until the project can compile.

Dependabot is set up on the project. This tool will automatically create Pull Request to upgrade our dependencies one by one. dependencies_group, gradle files, Dependabot, etc.

Test

Please refer to this dedicated document.

TODO add link to the dedicated screenshot test documentation

Other points

Logging

**Important warning: ** NEVER log private user data, or use the flag LOG_PRIVATE_DATA. Be very careful when logging data class, all the content will be output!

Timber is used to log data to logcat. We do not use directly the Log class. If possible please use a tag, as per

Timber.tag(loggerTag.value).d("my log")

because automatic tag (= class name) will not be available on the release version.

Also generally it is recommended to provide the Throwable to the Timber log functions.

Last point, not that Timber.v function may have no effect on some devices. Prefer using Timber.d and up.

Rageshake

Rageshake is a feature to send bug report directly from the application. Just shake your phone and you will be prompted to send a bug report.

Bug report can contain:

• a screenshot of the current application state
• the application logs from up to 15 application starts
• the logcat logs
• the key share history (crypto data)

The data will be sent to an internal server, which is not publicly accessible. A GitHub issue will also be created to a private GitHub repository.

Rageshake can be very useful to get logs from a release version of the application.

Tips

• Element Android has a developer mode in the Settings/Advanced settings. Other useful options are available here;
• Show hidden Events can also help to debug feature. When developer mode is enabled, it is possible to view the source (= the Json content) of any Events;
• Type /devtools in a Room composer to access a developer menu. There are some other entry points. Developer mode has to be enabled;
• Hidden debug menu: when developer mode is enabled and on debug build, there are some extra screens that can be accessible using the green wheel. In those screens, it will be possible to toggle some feature flags;
• Using logcat, filtering with onResume can help you to understand what screen are currently displayed on your device. Searching for string displayed on the screen can also help to find the running code in the codebase.
• When this is possible, prefer using sealed interface instead of sealed class;
• When writing temporary code, using the string "DO NOT COMMIT" in a comment can help to avoid committing things by mistake. If committed and pushed, the CI will detect this String and will warn the user about it.

Happy coding!

The team is here to support you, feel free to ask anything to other developers.

Also please feel to update this documentation, if incomplete/wrong/obsolete/etc.

Thanks!