Synapse: Matrix homeserver written in Python/Twisted.
Find a file
David Robertson 80d44060c9
Faster joins: omit partial rooms from eager syncs until the resync completes (#14870)
* Allow `AbstractSet` in `StrCollection`

Or else frozensets are excluded. This will be useful in an upcoming
commit where I plan to change a function that accepts `List[str]` to
accept `StrCollection` instead.

* `rooms_to_exclude` -> `rooms_to_exclude_globally`

I am about to make use of this exclusion mechanism to exclude rooms for
a specific user and a specific sync. This rename helps to clarify the
distinction between the global config and the rooms to exclude for a
specific sync.

* Better function names for internal sync methods

* Track a list of excluded rooms on SyncResultBuilder

I plan to feed a list of partially stated rooms for this sync to ignore

* Exclude partial state rooms during eager sync

using the mechanism established in the previous commit

* Track un-partial-state stream in sync tokens

So that we can work out which rooms have become fully-stated during a
given sync period.

* Fix mutation of `@cached` return value

This was fouling up a complement test added alongside this PR.
Excluding a room would mean the set of forgotten rooms in the cache
would be extended. This means that room could be erroneously considered
forgotten in the future.

Introduced in #12310, Synapse 1.57.0. I don't think this had any
user-visible side effects (until now).

* SyncResultBuilder: track rooms to force as newly joined

Similar plan as before. We've omitted rooms from certain sync responses;
now we establish the mechanism to reintroduce them into future syncs.

* Read new field, to present rooms as newly joined

* Force un-partial-stated rooms to be newly-joined

for eager incremental syncs only, provided they're still fully stated

* Notify user stream listeners to wake up long polling syncs

* Changelog

* Typo fix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Unnecessary list cast

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Rephrase comment

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Another comment

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Fixup merge(?)

* Poke notifier when receiving un-partial-stated msg over replication

* Fixup merge whoops

Thanks MV :)

Co-authored-by: Mathieu Velen <mathieuv@matrix.org>

Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-01-23 15:44:39 +00:00
.ci Require poetry>=1.3.2 (#14860) 2023-01-17 18:04:44 +00:00
.github Bump peaceiris/actions-gh-pages from 3.9.1 to 3.9.2 (#14861) 2023-01-17 18:28:04 +00:00
changelog.d Faster joins: omit partial rooms from eager syncs until the resync completes (#14870) 2023-01-23 15:44:39 +00:00
contrib Include x_forwarded in workers example configs (#14667) 2023-01-13 14:06:58 +00:00
debian Require poetry>=1.3.2 (#14860) 2023-01-17 18:04:44 +00:00
demo demo: check if we are in a virtualenv before overriding PYTHONPATH (#12916) 2022-05-30 15:41:24 +00:00
docker Enable Faster Remote Room Joins against worker-mode Synapse. (#14752) 2023-01-22 21:10:11 +00:00
docs Update logging_sample_config.md (#14868) 2023-01-19 11:58:17 +00:00
rust Implement MSC3930: polls push rules (#14787) 2023-01-19 12:47:10 +00:00
scripts-dev Enable Faster Remote Room Joins against worker-mode Synapse. (#14752) 2023-01-22 21:10:11 +00:00
stubs Implement MSC3930: polls push rules (#14787) 2023-01-19 12:47:10 +00:00
synapse Faster joins: omit partial rooms from eager syncs until the resync completes (#14870) 2023-01-23 15:44:39 +00:00
synmark Remove redundant "coding: utf-8" lines (#9786) 2021-04-14 15:34:27 +01:00
tests Faster joins: omit partial rooms from eager syncs until the resync completes (#14870) 2023-01-23 15:44:39 +00:00
.codecov.yml Disable codecov reports to GH comments. 2019-07-31 10:56:02 +01:00
.coveragerc Fix coverage in sytest and use plugins for buildkite (#5922) 2019-08-29 22:19:57 +10:00
.dockerignore Fix pinning Rust deps in docker images (#14129) 2022-10-11 10:53:34 +00:00
.editorconfig Apply correct editorconfig to .pyi files (#14526) 2022-11-22 18:33:28 +00:00
.git-blame-ignore-revs Prevent #3679 from appearing in blame results (#13311) 2022-07-18 14:02:32 +00:00
.gitignore Add poetry.toml to .gitignore (#14807) 2023-01-11 11:44:13 +00:00
.rustfmt.toml Port the push rule classes to Rust. (#13768) 2022-09-20 12:10:31 +01:00
AUTHORS.rst Automatically delete empty groups/communities (#6453) 2019-12-16 12:12:40 +00:00
book.toml Compile and render Synapse's docs into a browsable, mobile-friendly and searchable website (#10086) 2021-06-03 17:20:40 +01:00
build_rust.py Always build Rust extension in release mode (#14009) 2022-10-03 11:20:18 +01:00
Cargo.lock Bump regex from 1.7.0 to 1.7.1 (#14848) 2023-01-16 10:51:55 +00:00
Cargo.toml Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
CHANGES.md 1.75.0 2023-01-17 11:36:22 +00:00
CONTRIBUTING.md Advertise matrix-org.github.io/synapse docs (#10595) 2021-08-31 11:29:27 +01:00
INSTALL.md Fix broken links in INSTALL.md (#10331) 2021-07-08 15:46:13 +01:00
LICENSE Reference Matrix Home Server 2014-08-12 15:10:52 +01:00
mypy.ini Fix type hints for Monthly Active Users tests (#14889) 2023-01-22 10:50:14 +01:00
poetry.lock Bump ruff from 0.0.224 to 0.0.230 (#14897) 2023-01-23 09:32:07 -05:00
pylint.cfg Added pylint config file: ignore missing-docstring messages. 2014-10-24 10:22:09 +01:00
pyproject.toml Bump ruff from 0.0.224 to 0.0.230 (#14897) 2023-01-23 09:32:07 -05:00
README.rst fix grammar 2022-09-03 11:08:28 +01:00
sytest-blacklist Back out implementation of MSC2314 (#12474) 2022-04-19 11:17:29 +00:00
tox.ini Run lints under poetry in CI; remove lint tox jobs (#12434) 2022-04-12 17:35:48 +01:00
UPGRADE.rst fix broken link to upgrade notes (#10631) 2021-08-18 12:38:37 +01:00

Synapse (get support on #synapse:matrix.org) (discuss development on #synapse-dev:matrix.org) (Rendered documentation on GitHub Pages) (check license in LICENSE file) (latest version released on PyPi) (supported python versions)

Synapse is an open-source Matrix homeserver written and maintained by the Matrix.org Foundation. We began rapid development in 2014, reaching v1.0.0 in 2019. Development on Synapse and the Matrix protocol itself continues in earnest today.

Briefly, Matrix is an open standard for communications on the internet, supporting federation, encryption and VoIP. Matrix.org has more to say about the goals of the Matrix project, and the formal specification describes the technical details.

Installing and configuration

The Synapse documentation describes how to install Synapse. We recommend using Docker images or Debian packages from Matrix.org.

Synapse has a variety of config options which can be used to customise its behaviour after installation. There are additional details on how to configure Synapse for federation here.

Using a reverse proxy with Synapse

It is recommended to put a reverse proxy such as nginx, Apache, Caddy, HAProxy or relayd in front of Synapse. One advantage of doing so is that it means that you can expose the default https port (443) to Matrix clients without needing to run Synapse with root privileges. For information on configuring one, see the reverse proxy docs.

Upgrading an existing Synapse

The instructions for upgrading Synapse are in the upgrade notes. Please check these instructions as upgrading may require extra steps for some versions of Synapse.

Platform dependencies

Synapse uses a number of platform dependencies such as Python and PostgreSQL, and aims to follow supported upstream versions. See the deprecation policy for more details.

Security note

Matrix serves raw, user-supplied data in some APIs -- specifically the content repository endpoints.

Whilst we make a reasonable effort to mitigate against XSS attacks (for instance, by using CSP), a Matrix homeserver should not be hosted on a domain hosting other web applications. This especially applies to sharing the domain with Matrix web clients and other sensitive applications like webmail. See https://developer.github.com/changes/2014-04-25-user-content-security for more information.

Ideally, the homeserver should not simply be on a different subdomain, but on a completely different registered domain (also known as top-level site or eTLD+1). This is because some attacks are still possible as long as the two applications share the same registered domain.

To illustrate this with an example, if your Element Web or other sensitive web application is hosted on A.example1.com, you should ideally host Synapse on example2.com. Some amount of protection is offered by hosting on B.example1.com instead, so this is also acceptable in some scenarios. However, you should not host your Synapse on A.example1.com.

Note that all of the above refers exclusively to the domain used in Synapse's public_baseurl setting. In particular, it has no bearing on the domain mentioned in MXIDs hosted on that server.

Following this advice ensures that even if an XSS is found in Synapse, the impact to other applications will be minimal.

Testing a new installation

The easiest way to try out your new Synapse installation is by connecting to it from a web client.

Unless you are running a test instance of Synapse on your local machine, in general, you will need to enable TLS support before you can successfully connect from a client: see TLS certificates.

An easy way to get started is to login or register via Element at https://app.element.io/#/login or https://app.element.io/#/register respectively. You will need to change the server you are logging into from matrix.org and instead specify a Homeserver URL of https://<server_name>:8448 (or just https://<server_name> if you are using a reverse proxy). If you prefer to use another client, refer to our client breakdown.

If all goes well you should at least be able to log in, create a room, and start sending messages.

Registering a new user from a client

By default, registration of new users via Matrix clients is disabled. To enable it:

  1. In the registration config section set enable_registration: true in homeserver.yaml.
  2. Then either:
    1. set up a CAPTCHA, or
    2. set enable_registration_without_verification: true in homeserver.yaml.

We strongly recommend using a CAPTCHA, particularly if your homeserver is exposed to the public internet. Without it, anyone can freely register accounts on your homeserver. This can be exploited by attackers to create spambots targetting the rest of the Matrix federation.

Your new user name will be formed partly from the server_name, and partly from a localpart you specify when you create the account. Your name will take the form of:

@localpart:my.domain.name

(pronounced "at localpart on my dot domain dot name").

As when logging in, you will need to specify a "Custom server". Specify your desired localpart in the 'User name' box.

Troubleshooting and support

The Admin FAQ includes tips on dealing with some common problems. For more details, see Synapse's wider documentation.

For additional support installing or managing Synapse, please ask in the community support room #synapse:matrix.org_ (from a matrix.org account if necessary). We do not use GitHub issues for support requests, only for bug reports and feature requests.

Identity Servers

Identity servers have the job of mapping email addresses and other 3rd Party IDs (3PIDs) to Matrix user IDs, as well as verifying the ownership of 3PIDs before creating that mapping.

They are not where accounts or credentials are stored - these live on home servers. Identity Servers are just for mapping 3rd party IDs to matrix IDs.

This process is very security-sensitive, as there is obvious risk of spam if it is too easy to sign up for Matrix accounts or harvest 3PID data. In the longer term, we hope to create a decentralised system to manage it (matrix-doc #712), but in the meantime, the role of managing trusted identity in the Matrix ecosystem is farmed out to a cluster of known trusted ecosystem partners, who run 'Matrix Identity Servers' such as Sydent, whose role is purely to authenticate and track 3PID logins and publish end-user public keys.

You can host your own copy of Sydent, but this will prevent you reaching other users in the Matrix ecosystem via their email address, and prevent them finding you. We therefore recommend that you use one of the centralised identity servers at https://matrix.org or https://vector.im for now.

To reiterate: the Identity server will only be used if you choose to associate an email address with your account, or send an invite to another user via their email address.

Development

We welcome contributions to Synapse from the community! The best place to get started is our guide for contributors. This is part of our larger documentation, which includes

information for Synapse developers as well as Synapse administrators. Developers might be particularly interested in:

Alongside all that, join our developer community on Matrix: #synapse-dev:matrix.org, featuring real humans!