45 commits

Author	SHA1	Message	Date
Erik Johnston	8bbc98e66d	Use a new token format for sliding sync (#17452) ... This is in preparation for adding per-connection state. --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>	2024-07-24 11:47:25 +01:00
Eric Eastwood	de05a64246	Sliding Sync: Add E2EE extension (MSC3884) (#17454) ... Spec: [MSC3884](https://github.com/matrix-org/matrix-spec-proposals/pull/3884) Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-22 15:40:06 -05:00
Erik Johnston	d221512498	SS: Implement $ME support (#17469) ... `$ME` can be used as a substitute for the requester's user ID.	2024-07-22 17:48:09 +01:00
Eric Eastwood	3fee32ed6b	Order heroes by stream_ordering (as spec'ed) (#17435) ... The spec specifically mentions `stream_ordering` but that's a Synapse specific concept. In any case, the essence of the spec is basically the first 5 members of the room which `stream_ordering` accomplishes. Split off from https://github.com/element-hq/synapse/pull/17419#discussion_r1671342794 ## Spec compliance > This should be the first 5 members of the room, **ordered by stream ordering**, which are joined or invited. The list must never include the client’s own user ID. When no joined or invited members are available, this should consist of the banned and left users. > > *-- https://spec.matrix.org/v1.10/client-server-api/#_matrixclientv3sync_roomsummary* Related to https://github.com/matrix-org/matrix-spec/issues/1334	2024-07-17 13:10:15 -05:00
Eric Eastwood	ab62aa09da	Add room subscriptions to Sliding Sync /sync (#17432) ... Add room subscriptions to Sliding Sync `/sync` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync Currently, you can only subscribe to rooms you have had *any* membership in before. In the future, we will allow `world_readable` rooms to be subscribed to without joining.	2024-07-15 10:37:10 +01:00
Eric Eastwood	fb66e938b2	Add is_dm room field to Sliding Sync /sync (#17429) ... Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-11 18:19:26 -05:00
Eric Eastwood	5a97bbd895	Add heroes and room summary fields to Sliding Sync /sync (#17419) ... Additional room summary fields: `joined_count`, `invited_count` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-11 14:05:38 -05:00
Erik Johnston	4ca13ce0dd	Handle to-device extensions to Sliding Sync (#17416) ... Implements MSC3885 --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>	2024-07-10 11:58:42 +01:00
Eric Eastwood	1cf3ff6b40	Add rooms name and avatar to Sliding Sync /sync (#17418) ... Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-09 12:26:45 -05:00
Eric Eastwood	3fef535ff2	Add rooms.bump_stamp to Sliding Sync /sync for easier client-side sorting (#17395) ... `bump_stamp` corresponds to the `stream_ordering` of the latest `DEFAULT_BUMP_EVENT_TYPES` in the room. This helps clients sort more readily without them needing to pull in a bunch of the timeline to determine the last activity. `bump_event_types` is a thing because for example, we don't want display name changes to mark the room as unread and bump it to the top. For encrypted rooms, we just have to consider any activity as a bump because we can't see the content and the client has to figure it out for themselves. Outside of Synapse, `bump_stamp` is just a free-form counter so other implementations could use `received_ts`or `origin_server_ts` (see the [*Security considerations* section in MSC3575 about the potential pitfalls of using `origin_server_ts`](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#security-considerations)). It doesn't have any guarantee about always going up. In the Synapse case, it could go down if an event was redacted/removed (or purged in cases of retention policies). In the future, we could add `bump_event_types` as [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) mentions if people need to customize the event types. --- In the Sliding Sync proxy, a similar [`timestamp` field was added](https://github.com/matrix-org/sliding-sync/pull/247) for the same purpose but the name is not obvious what it pertains to or what it's for. The `timestamp` field was also added to Ruma in https://github.com/ruma/ruma/pull/1622	2024-07-08 13:17:08 -05:00
Eric Eastwood	22aeb78b77	Add rooms.required_state to Sliding Sync /sync (#17342) ... Also handles excluding rooms with partial state when people are asking for room membership events unless it's `$LAZY` room membership.	2024-07-04 12:25:36 -05:00
Eric Eastwood	fa91655805	Return some room data in Sliding Sync /sync (#17320) ... - Timeline events - Stripped `invite_state` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-07-02 11:07:05 -05:00
Erik Johnston	b3b793786c	Fix sync waiting for an invalid token from the "future" (#17386) ... Fixes https://github.com/element-hq/synapse/issues/17274, hopefully. Basically, old versions of Synapse could advance streams without persisting anything in the DB (fixed in #17229). On restart those updates would get lost, and so the position of the stream would revert to an older position. If this happened across an upgrade to a later Synapse version which included #17215, then sync could get blocked indefinitely (until the stream advanced to the position in the token). We fix this by bounding the stream positions we'll wait for to the maximum position of the underlying stream ID generator.	2024-07-02 12:39:49 +01:00
Eric Eastwood	6e8af83193	Add is_invite filtering to Sliding Sync /sync (#17335) ... Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-06-24 19:07:56 -05:00
Erik Johnston	3239b7459c	Register sliding sync under a different path (#17331) ... As the API is slightly incompatible.	2024-06-19 16:18:45 +00:00
Eric Eastwood	e5b8a3e37f	Add stream_ordering sort to Sliding Sync /sync (#17293) ... Sort is no longer configurable and we always sort rooms by the `stream_ordering` of the last event in the room or the point where the user can see up to in cases of leave/ban/invite/knock.	2024-06-17 11:27:14 -05:00
Eric Eastwood	c12ee0d5ba	Add is_dm filtering to Sliding Sync /sync (#17277) ... Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync	2024-06-13 13:56:58 -05:00
Eric Eastwood	4a7c58642c	Add Sliding Sync /sync endpoint (initial implementation) (#17187) ... Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync This iteration only focuses on returning the list of room IDs in the sliding window API (without sorting/filtering). Rooms appear in the Sliding sync response based on: - `invite`, `join`, `knock`, `ban` membership events - Kicks (`leave` membership events where `sender` is different from the `user_id`/`state_key`) - `newly_left` (rooms that were left during the given token range, > `from_token` and <= `to_token`) - In order for bans/kicks to not show up, you need to `/forget` those rooms. This doesn't modify the event itself though and only adds the `forgotten` flag to `room_memberships` in Synapse. There isn't a way to tell when a room was forgotten at the moment so we can't factor it into the from/to range. ### Example request `POST http://localhost:8008/_matrix/client/unstable/org.matrix.msc3575/sync` ```json { "lists": { "foo-list": { "ranges": [ [0, 99] ], "sort": [ "by_notification_level", "by_recency", "by_name" ], "required_state": [ ["m.room.join_rules", ""], ["m.room.history_visibility", ""], ["m.space.child", "*"] ], "timeline_limit": 100 } } } ``` Response: ```json { "next_pos": "s58_224_0_13_10_1_1_16_0_1", "lists": { "foo-list": { "count": 1, "ops": [ { "op": "SYNC", "range": [0, 99], "room_ids": [ "!MmgikIyFzsuvtnbvVG:my.synapse.linux.server" ] } ] } }, "rooms": {}, "extensions": {} } ```	2024-06-06 14:44:32 -05:00
Eric Eastwood	c97251d5ba	Add Sliding Sync /sync/e2ee endpoint for To-Device messages (#17167) ... This is being introduced as part of Sliding Sync but doesn't have any sliding window component. It's just a way to get E2EE events without having to sit through a big initial sync (`/sync` v2). And we can avoid encryption events being backed up by the main sync response or vice-versa. Part of some Sliding Sync simplification/experimentation. See [this discussion](https://github.com/element-hq/synapse/pull/17167#discussion_r1610495866) for why it may not be as useful as we thought. Based on: - https://github.com/matrix-org/matrix-spec-proposals/pull/3575 - https://github.com/matrix-org/matrix-spec-proposals/pull/3885 - https://github.com/matrix-org/matrix-spec-proposals/pull/3884	2024-05-23 12:06:16 -05:00
Erik Johnston	23740eaa3d	Correctly mention previous copyright (#16820) ... During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.	2024-01-23 11:26:48 +00:00
Patrick Cloke	8e1e62c9e0	Update license headers	2023-11-21 15:29:58 -05:00
Patrick Cloke	f2f2c7c1f0	Use full GitHub links instead of bare issue numbers. (#16637)	2023-11-15 08:02:11 -05:00
Patrick Cloke	63d28a88c1	Additional validation of receipts (#16327) ... Reject invalid receipts with a reasonable error message & expands tests for receipts.	2023-09-18 13:02:12 +00:00
dependabot[bot]	9bb2eac719	Bump black from 22.12.0 to 23.1.0 (#15103)	2023-02-22 15:29:09 -05:00
Andrew Morgan	836c592f15	Fix type hints in knocking tests. (#14887)	2023-01-25 14:38:20 -05:00
David Robertson	80d44060c9	Faster joins: omit partial rooms from eager syncs until the resync completes (#14870) ... * Allow `AbstractSet` in `StrCollection` Or else frozensets are excluded. This will be useful in an upcoming commit where I plan to change a function that accepts `List[str]` to accept `StrCollection` instead. * `rooms_to_exclude` -> `rooms_to_exclude_globally` I am about to make use of this exclusion mechanism to exclude rooms for a specific user and a specific sync. This rename helps to clarify the distinction between the global config and the rooms to exclude for a specific sync. * Better function names for internal sync methods * Track a list of excluded rooms on SyncResultBuilder I plan to feed a list of partially stated rooms for this sync to ignore * Exclude partial state rooms during eager sync using the mechanism established in the previous commit * Track un-partial-state stream in sync tokens So that we can work out which rooms have become fully-stated during a given sync period. * Fix mutation of `@cached` return value This was fouling up a complement test added alongside this PR. Excluding a room would mean the set of forgotten rooms in the cache would be extended. This means that room could be erroneously considered forgotten in the future. Introduced in #12310, Synapse 1.57.0. I don't think this had any user-visible side effects (until now). * SyncResultBuilder: track rooms to force as newly joined Similar plan as before. We've omitted rooms from certain sync responses; now we establish the mechanism to reintroduce them into future syncs. * Read new field, to present rooms as newly joined * Force un-partial-stated rooms to be newly-joined for eager incremental syncs only, provided they're still fully stated * Notify user stream listeners to wake up long polling syncs * Changelog * Typo fix Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Unnecessary list cast Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Rephrase comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Another comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Fixup merge(?) * Poke notifier when receiving un-partial-stated msg over replication * Fixup merge whoops Thanks MV :) Co-authored-by: Mathieu Velen <mathieuv@matrix.org> Co-authored-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2023-01-23 15:44:39 +00:00
Šimon Brandner	0e99f07952	Remove support for unstable private read receipts (#13653) ... Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>	2022-09-01 13:31:54 +01:00
Šimon Brandner	ab18441573	Support stable identifiers for MSC2285: private read receipts. (#13273) ... This adds support for the stable identifiers of MSC2285 while continuing to support the unstable identifiers behind the configuration flag. These will be removed in a future version.	2022-08-05 11:09:33 -04:00
Brendan Abolivier	845732be45	Fix rooms not being properly excluded from incremental sync (#13408)	2022-08-04 11:02:29 +02:00
Dirk Klimpel	efee345b45	Remove unnecessary json.dumps from tests (#13303)	2022-07-17 22:28:45 +01:00
Patrick Cloke	c52abc1cfd	Additional constants for EDU types. (#12884) ... Instead of hard-coding strings in many places.	2022-05-27 07:14:36 -04:00
Patrick Cloke	86a515ccbf	Consolidate logic for parsing relations. (#12693) ... Parse the `m.relates_to` event content field (which describes relations) in a single place, this is used during: * Event persistence. * Validation of the Client-Server API. * Fetching bundled aggregations. * Processing of push rules. Each of these separately implement the logic and each made slightly different assumptions about what was valid. Some had minor / potential bugs.	2022-05-16 12:42:45 +00:00
David Robertson	db10f2c037	No longer permit empty body when sending receipts (#12709)	2022-05-11 15:34:17 +00:00
Šimon Brandner	9ae0253f4e	Use private instead of hidden in MSC2285 related code. (#12635)	2022-05-05 12:31:25 +00:00
Šimon Brandner	116a4c8340	Implement changes to MSC2285 (hidden read receipts) (#12168) ... * Changes hidden read receipts to be a separate receipt type (instead of a field on `m.read`). * Updates the `/receipts` endpoint to accept `m.fully_read`.	2022-05-04 11:59:22 -04:00
Šimon Brandner	0d9eaa19fd	Use constants for receipt types in tests. (#12582)	2022-04-28 13:34:33 -04:00
Patrick Cloke	bebf994ee8	Move MSC2654 support behind an experimental configuration flag. (#12295) ... To match the current thinking on disabling experimental features by default.	2022-03-31 15:05:13 -04:00
Andrew Morgan	19a1d6a42a	Remove references to "msc2403" (#12165)	2022-03-30 11:15:47 +00:00
Brendan Abolivier	437a8ed9ef	Add a configuration to exclude rooms from sync response (#12310)	2022-03-30 09:43:04 +00:00
Patrick Cloke	02d708568b	Replace assertEquals and friends with non-deprecated versions. (#12092)	2022-02-28 07:12:29 -05:00
Dirk Klimpel	64c73c6ac8	Add type hints to tests/rest/client (#12066)	2022-02-23 08:33:19 -05:00
Richard van der Hoff	e24ff8ebe3	Remove HomeServer.get_datastore() (#12031) ... The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733	2022-02-23 11:04:02 +00:00
Andrew Morgan	0c4878caf2	Add a unit test for users receiving their own device list updates (#11909)	2022-02-07 13:21:19 +00:00
rogersheu	820337e6a4	Require body for read receipts with user-agent exceptions (#11157) ... Co-authored-by: reivilibre <olivier@librepush.net>	2021-11-09 10:26:07 +00:00
David Robertson	ecd823d766	Flatten tests/rest/client/{v1,v2_alpha} too (#10667)	2021-08-20 17:50:44 +01:00

Renamed from tests/rest/client/v2_alpha/test_sync.py (Browse further)