8289 commits

Author	SHA1	Message	Date
Quentin Gliech	e0ff850cb7	1.111.0rc2	2024-07-10 10:47:35 +02:00
Erik Johnston	22fbc5be54	Fix new media APIs when using synapse.app.media_repository (#17420) ... Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-07-09 18:29:32 +01:00
Andrew Morgan	62d8b0361b	Note the new federated media worker endpoints in the worker docs & upgrade notes (#17421)	2024-07-09 16:41:51 +01:00
Andrew Morgan	4d6f7c0fc9	Route auth'd fed media requests to media repo in Complement tests (#17422)	2024-07-09 15:12:11 +00:00
Quentin Gliech	d48061b7e6	Fix up the changelog	2024-07-09 12:36:24 +02:00
Quentin Gliech	963a60c7e7	Merge remote-tracking branch 'origin/develop' into release-v1.111	2024-07-09 12:36:15 +02:00
Erik Johnston	028b103ae0	Fix exception when failing to talk to remote server (#17411) ... Broke in #17381	2024-07-09 10:04:40 +00:00
Quentin Gliech	abb1384502	1.111.0rc1	2024-07-09 11:51:51 +02:00
Erik Johnston	0ed1c64c83	Fix /versions requests (#17410) ... We need it to work on workers and allow guest access. Broke by #17392	2024-07-09 10:25:44 +01:00
Erik Johnston	8cdd2d214e	Fix bug in sliding sync when using old DB. (#17398) ... We don't necessarily have `instance_name` for old events (before we support multiple event persisters). We treat those as if the `instance_name` was "master". --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>	2024-07-08 20:30:23 +01:00
Eric Eastwood	3fef535ff2	Add rooms.bump_stamp to Sliding Sync /sync for easier client-side sorting (#17395) ... `bump_stamp` corresponds to the `stream_ordering` of the latest `DEFAULT_BUMP_EVENT_TYPES` in the room. This helps clients sort more readily without them needing to pull in a bunch of the timeline to determine the last activity. `bump_event_types` is a thing because for example, we don't want display name changes to mark the room as unread and bump it to the top. For encrypted rooms, we just have to consider any activity as a bump because we can't see the content and the client has to figure it out for themselves. Outside of Synapse, `bump_stamp` is just a free-form counter so other implementations could use `received_ts`or `origin_server_ts` (see the [*Security considerations* section in MSC3575 about the potential pitfalls of using `origin_server_ts`](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#security-considerations)). It doesn't have any guarantee about always going up. In the Synapse case, it could go down if an event was redacted/removed (or purged in cases of retention policies). In the future, we could add `bump_event_types` as [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) mentions if people need to customize the event types. --- In the Sliding Sync proxy, a similar [`timestamp` field was added](https://github.com/matrix-org/sliding-sync/pull/247) for the same purpose but the name is not obvious what it pertains to or what it's for. The `timestamp` field was also added to Ruma in https://github.com/ruma/ruma/pull/1622	2024-07-08 13:17:08 -05:00
Travis Ralston	ccbd619b43	Declare support for Matrix 1.11 (#17403) ... Previous: https://github.com/element-hq/synapse/pull/17082 Fixes https://github.com/element-hq/synapse/issues/17402 See https://github.com/element-hq/synapse/issues/17402 for context **Blocked on https://github.com/element-hq/synapse/pull/17388** (required for spec compliance)	2024-07-08 07:37:28 -06:00
Quentin Gliech	c896030f67	MSC3861: allow overriding the introspection endpoint (#17406) ... This makes it easier to go through an internal endpoint instead of the public facing URL when introspecting tokens, reducing latency.	2024-07-08 14:08:11 +02:00
Shay	cf69f8d59b	Support MSC3916 by adding a federation /thumbnail endpoint and authenticated _matrix/client/v1/media/thumbnail endpoint (#17388) ... [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/pull/3916) added the endpoints `_matrix/federation/v1/media/thumbnail` and the authenticated `_matrix/client/v1/media/thumbnail`. This PR implements those endpoints, along with stabilizing `_matrix/client/v1/media/config` and `_matrix/client/v1/media/preview_url`. Complement tests are at https://github.com/matrix-org/complement/pull/728	2024-07-08 10:11:20 +01:00
dependabot[bot]	20de685a4b	Bump ruff from 0.3.7 to 0.5.0 (#17381)	2024-07-05 12:35:57 +00:00
Erik Johnston	8e9e6f1a0a	Allow enabling sliding sync per-user (#17393) ... Based on #17392	2024-07-05 13:04:27 +01:00
Erik Johnston	57538eb4d9	Finish up work to allow per-user feature flags (#17392) ... Follows on from @H-Shay's great work at https://github.com/matrix-org/synapse/pull/15344 and MSC4026. Also enables its use for MSC3881, mainly as an easy but concrete example of how to use it.	2024-07-05 13:02:35 +01:00
Sandro	45b35f8eae	Fix links to MSC3916 (#17397)	2024-07-05 12:00:19 +01:00
Travis Ralston	2ec257d608	Upon deactivation, forget all of the user's rooms (#17400) ... This can help ensure that the rooms are eventually purged if the other local users also forget them. Synapse already clears some of the room information as part of the `_background_remove_left_rooms` background task, but this doesn't catch `events`, `event_json`, etc.	2024-07-05 11:56:52 +01:00
Hugh Nimmo-Smith	daeaeb2a7b	Fix links in README (#17379) ... I don't think this warrants a changelog? ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [x] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [ ] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))	2024-07-05 10:34:02 +00:00
Quentin Gliech	7786ae7e1c	Make the release script create a release branch for Complement as well (#17318)	2024-07-05 09:07:45 +02:00
Eric Eastwood	22aeb78b77	Add rooms.required_state to Sliding Sync /sync (#17342) ... Also handles excluding rooms with partial state when people are asking for room membership events unless it's `$LAZY` room membership.	2024-07-04 12:25:36 -05:00
Eric Eastwood	a9d2e40ea4	Changelog entries only get merged if they have the same content and extension (#17399) ... Changelog entries only get merged if they have the same content and extension See https://github.com/element-hq/synapse/pull/17301#discussion_r1665387218	2024-07-04 17:21:25 +01:00
Eric Eastwood	7be03d854b	Add room_types/not_room_types filtering to Sliding Sync /sync (#17337) ... Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-02 12:46:27 -05:00
Eric Eastwood	fa91655805	Return some room data in Sliding Sync /sync (#17320) ... - Timeline events - Stripped `invite_state` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-02 11:07:05 -05:00
Devon Hudson	0d2b75cf92	Merge branch 'release-v1.110' into develop	2024-07-02 09:53:32 -06:00
Devon Hudson	ccce858c4a	1.110.0rc3	2024-07-02 08:29:34 -06:00
Erik Johnston	99c107920d	Merge remote-tracking branch 'origin/release-v1.110' into develop	2024-07-02 15:12:36 +01:00
Erik Johnston	1609855ff8	Limit size of presence EDUs (#17371) ... Otherwise they are unbounded. --------- Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-07-02 14:07:11 +01:00
Shay	8f890447b0	Support MSC3916 by adding _matrix/client/v1/media/download endpoint (#17365)	2024-07-02 14:07:04 +01:00
Erik Johnston	b905ae27ca	Fix regression when bounding future tokens (#17391) ... Fix bug added in #17386, where we accidentally used `room_key` for the receipts stream. See first commit. Reviewable commit-by-commit	2024-07-02 14:06:36 +01:00
Erik Johnston	1ce59d7ba0	Fix sync waiting for an invalid token from the "future" (#17386) ... Fixes https://github.com/element-hq/synapse/issues/17274, hopefully. Basically, old versions of Synapse could advance streams without persisting anything in the DB (fixed in #17229). On restart those updates would get lost, and so the position of the stream would revert to an older position. If this happened across an upgrade to a later Synapse version which included #17215, then sync could get blocked indefinitely (until the stream advanced to the position in the token). We fix this by bounding the stream positions we'll wait for to the maximum position of the underlying stream ID generator.	2024-07-02 12:40:03 +01:00
Erik Johnston	b3b793786c	Fix sync waiting for an invalid token from the "future" (#17386) ... Fixes https://github.com/element-hq/synapse/issues/17274, hopefully. Basically, old versions of Synapse could advance streams without persisting anything in the DB (fixed in #17229). On restart those updates would get lost, and so the position of the stream would revert to an older position. If this happened across an upgrade to a later Synapse version which included #17215, then sync could get blocked indefinitely (until the stream advanced to the position in the token). We fix this by bounding the stream positions we'll wait for to the maximum position of the underlying stream ID generator.	2024-07-02 12:39:49 +01:00
Erik Johnston	9c8f1a6d41	Fix building debian packages on non-clean checkouts (#17390) ... If we leave the `.so` in place it causes the tests to fail, as it gets picked up (instead of the newly built .so) and so fails with mismatched GLIBC errors.	2024-07-02 12:39:32 +01:00
Erik Johnston	5b5280e3e5	Fix building debian packages for sid (#17389) ... Sid now defaults to python3.12, and our pinned version of cffi (1.5.1) does not have wheels for 3.12. This installing cffi to fail as we did not have the correct libs installed to build from source.	2024-07-02 12:38:29 +01:00
Erik Johnston	2f5a77efae	Limit size of presence EDUs (#17371) ... Otherwise they are unbounded. --------- Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-07-01 12:48:36 +01:00
davidegirardi	b11f5c984b	Clarify url_preview_url_blacklist is a usability feature (#17356)	2024-06-28 14:39:54 +01:00
Till	27756c9fdf	Use rstcheck to "lint" the README (#17367) ... Follow up to https://github.com/element-hq/synapse/pull/17363, so we can detect issues with the RST file early on. ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [x] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [x] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))	2024-06-27 13:36:08 +02:00
Erik Johnston	cc5e5893fe	Handle multiple rows device inbox (#17362) ... Fix bug where we don't get new to-device from remote if they resent a message we've already persisted and have recorded in the DB twice. `device_federation_inbox` table doesn't have a unique index, and so we can race and store an entry in there twice. If we do so then `simple_select_one_txn` will throw an error due to the query returning more than one row. We should add an unique index, but it doesn't really matter so lets just handle the case of multiple rows correctly for now.	2024-06-27 11:04:31 +01:00
Till Faelligen	7c169f4d2c	Merge branch 'release-v1.110' into develop	2024-06-26 19:00:51 +02:00
Till Faelligen	f75da9cc53	1.110.0rc2	2024-06-26 18:14:57 +02:00
Till	75c19bf57a	Fix uploading packages to PyPi (#17363) ... As per https://github.com/sphinx-doc/sphinx/issues/3921#issuecomment-315581557, we need double underscores. Running `rst2html README.rst > /dev/null` found some more warnings.	2024-06-26 18:14:29 +02:00
Till	b924a8e1a9	Fix uploading packages to PyPi (#17363) ... As per https://github.com/sphinx-doc/sphinx/issues/3921#issuecomment-315581557, we need double underscores. Running `rst2html README.rst > /dev/null` found some more warnings.	2024-06-26 18:01:39 +02:00
Till Faelligen	9f47513458	1.110.0rc1	2024-06-26 14:09:13 +02:00
Andrew Morgan	ef7fbdfebd	Fixes to the table of contents in the README (#17329)	2024-06-25 15:20:59 +00:00
Denis Kasak	9cf0ef9c70	Fix outdated Security Disclosure Policy references (#17341)	2024-06-25 15:58:30 +01:00
Shay	a023538822	Re-introduce federation /download endpoint (#17350)	2024-06-25 14:35:37 +00:00
Erik Johnston	c89fea3fd1	Limit amount of replication we send (#17358) ... Fixes up #17333, where we failed to actually send less data (the `DISTINCT` didn't work due to `stream_id` being different). We fix this by making it so that every device list outbound poke for a given user ID has the same stream ID. We can't change the query to only return e.g. max stream ID as the receivers look up the destinations to send to by doing `SELECT WHERE stream_id = ?`	2024-06-25 11:17:39 +01:00
Erik Johnston	554a92601a	Reintroduce "Reduce device lists replication traffic."" (#17361) ... Reintroduces https://github.com/element-hq/synapse/pull/17333 Turns out the reason for revert was down two master instances running	2024-06-25 10:34:34 +01:00
Erik Johnston	a98cb87bee	Revert "Reduce device lists replication traffic." (#17360) ... Reverts element-hq/synapse#17333 It looks like master was still sending out replication RDATA with the old format... somehow	2024-06-25 09:57:34 +01:00