Erik Johnston
726006cdf2
Don't invalidate all get_relations_for_event on history purge ( #17083 )
...
This is a tree cache already, so may as well move the room ID to the
front and use that
2024-05-29 12:57:10 +01:00
Erik Johnston
967b6948b0
Change allow_unsafe_locale to also apply on new databases ( #17238 )
...
We relax this as there are use cases where this is safe, though it is
still highly recommended that people avoid using it.
2024-05-29 12:04:13 +01:00
Erik Johnston
d7198dfb95
Ignore attempts to send to-device messages to bad users ( #17240 )
...
Currently sending a to-device message to a user ID with a dodgy
destination is accepted, but then ends up spamming the logs when we try
and send to the destination.
An alternative would be to reject the request, but I'm slightly nervous
that could break things.
2024-05-29 11:52:48 +01:00
Erik Johnston
94ef2f4f5d
Handle duplicate OTK uploads racing ( #17241 )
...
Currently this causes one of then to 500.
2024-05-29 11:16:00 +01:00
Erik Johnston
bb5a692946
Fix slipped logging context when media rejected ( #17239 )
...
When a module rejects a piece of media we end up trying to close the
same logging context twice.
Instead of fixing the existing code we refactor to use an async context
manager, which is easier to write correctly.
2024-05-29 11:14:42 +01:00
Olivier 'reivilibre
ad179b0136
Merge branch 'master' into develop
Build release artifacts / Build wheels on ${{ matrix.os }} for ${{ matrix.arch }} (x86_64, ${{ startsWith(github.ref, 'refs/pull/') }}, ubuntu-20.04) (push) Waiting to run
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / calculate-test-jobs (push) Blocked by required conditions
Tests / lint-newsfile (push) Waiting to run
Tests / lint-pydantic (push) Blocked by required conditions
Tests / lint-clippy (push) Blocked by required conditions
Tests / lint-clippy-nightly (push) Blocked by required conditions
Tests / lint-rustfmt (push) Blocked by required conditions
Tests / linting-done (push) Blocked by required conditions
Tests / changes (push) Waiting to run
Tests / check-sampleconfig (push) Blocked by required conditions
Tests / check-schema-delta (push) Blocked by required conditions
Tests / check-lockfile (push) Waiting to run
Tests / lint (push) Blocked by required conditions
Tests / Typechecking (push) Blocked by required conditions
Tests / lint-crlf (push) Waiting to run
Tests / trial (push) Blocked by required conditions
Tests / trial-olddeps (push) Blocked by required conditions
Tests / trial-pypy (all, pypy-3.8) (push) Blocked by required conditions
Tests / sytest (push) Blocked by required conditions
Tests / export-data (push) Blocked by required conditions
Tests / portdb (11, 3.8) (push) Blocked by required conditions
Tests / portdb (15, 3.11) (push) Blocked by required conditions
Tests / complement (monolith, Postgres) (push) Blocked by required conditions
Tests / complement (monolith, SQLite) (push) Blocked by required conditions
Tests / complement (workers, Postgres) (push) Blocked by required conditions
Tests / cargo-test (push) Blocked by required conditions
Tests / cargo-bench (push) Blocked by required conditions
Tests / tests-done (push) Blocked by required conditions
/ Check locked dependencies have sdists (push) Has been cancelled
2024-05-28 13:34:19 +01:00
dependabot[bot]
5147ce294a
Bump phonenumbers from 8.13.35 to 8.13.37 ( #17235 )
2024-05-28 13:26:37 +01:00
Olivier 'reivilibre
f35bc08d39
1.108.0
Build release artifacts / Build sdist (push) Has been cancelled
Tests / changes (push) Has been cancelled
Tests / lint-crlf (push) Has been cancelled
Tests / lint-newsfile (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / calculate-test-jobs (push) Has been cancelled
Tests / trial (push) Has been cancelled
Tests / cargo-bench (push) Has been cancelled
Tests / tests-done (push) Has been cancelled
Tests / check-schema-delta (push) Has been cancelled
Deploy the documentation / GitHub Pages (push) Has been cancelled
Build release artifacts / Build .deb packages (push) Has been cancelled
Build release artifacts / Attach assets to release (push) Has been cancelled
Tests / check-sampleconfig (push) Has been cancelled
Tests / Typechecking (push) Has been cancelled
Tests / lint-pydantic (push) Has been cancelled
Tests / lint-clippy (push) Has been cancelled
Tests / lint-clippy-nightly (push) Has been cancelled
Tests / lint-rustfmt (push) Has been cancelled
Tests / linting-done (push) Has been cancelled
Tests / trial-olddeps (push) Has been cancelled
Tests / trial-pypy (all, pypy-3.8) (push) Has been cancelled
Tests / sytest (push) Has been cancelled
Tests / export-data (push) Has been cancelled
Tests / portdb (11, 3.8) (push) Has been cancelled
Tests / portdb (15, 3.11) (push) Has been cancelled
Tests / complement (monolith, Postgres) (push) Has been cancelled
Tests / complement (monolith, SQLite) (push) Has been cancelled
Tests / complement (workers, Postgres) (push) Has been cancelled
Tests / cargo-test (push) Has been cancelled
2024-05-28 11:54:28 +01:00
dependabot[bot]
f2616edb73
Bump pyicu from 2.13 to 2.13.1 ( #17236 )
2024-05-28 11:28:58 +01:00
dependabot[bot]
86a2a0258f
Bump pyopenssl from 24.0.0 to 24.1.0 ( #17234 )
2024-05-28 11:28:32 +01:00
dependabot[bot]
0893ee9af8
Bump prometheus-client from 0.19.0 to 0.20.0 ( #17233 )
2024-05-28 11:28:16 +01:00
dependabot[bot]
887f773472
Bump serde from 1.0.202 to 1.0.203 ( #17232 )
2024-05-28 11:27:51 +01:00
Shay
9edb725ebc
Support MSC3916 by adding unstable media endpoints to _matrix/client ( #17213 )
...
Tests / check-lockfile (push) Has been cancelled
Tests / changes (push) Has been cancelled
Tests / lint-crlf (push) Has been cancelled
Tests / lint-newsfile (push) Has been cancelled
Deploy the documentation / GitHub Pages (push) Has been cancelled
Build release artifacts / Build .deb packages (push) Has been cancelled
Build release artifacts / Attach assets to release (push) Has been cancelled
Tests / check-sampleconfig (push) Has been cancelled
Tests / check-schema-delta (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / Typechecking (push) Has been cancelled
Tests / lint-pydantic (push) Has been cancelled
Tests / lint-clippy (push) Has been cancelled
Tests / lint-clippy-nightly (push) Has been cancelled
Tests / lint-rustfmt (push) Has been cancelled
Tests / linting-done (push) Has been cancelled
Tests / calculate-test-jobs (push) Has been cancelled
Tests / trial (push) Has been cancelled
Tests / trial-olddeps (push) Has been cancelled
Tests / trial-pypy (all, pypy-3.8) (push) Has been cancelled
Tests / sytest (push) Has been cancelled
Tests / export-data (push) Has been cancelled
Tests / portdb (11, 3.8) (push) Has been cancelled
Tests / portdb (15, 3.11) (push) Has been cancelled
Tests / complement (monolith, Postgres) (push) Has been cancelled
Tests / complement (monolith, SQLite) (push) Has been cancelled
Tests / complement (workers, Postgres) (push) Has been cancelled
Tests / cargo-test (push) Has been cancelled
Tests / cargo-bench (push) Has been cancelled
Tests / tests-done (push) Has been cancelled
[MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/authentication-for-media/proposals/3916-authentication-for-media.md )
adds new media endpoints under `_matrix/client`. This PR adds the
`/preview_url`, `/config`, and `/thumbnail` endpoints. `/download` will
be added in a follow-up PR once the work for the federation `/download`
endpoint is complete (see
https://github.com/element-hq/synapse/pull/17172 ).
Should be reviewable commit-by-commit.
2024-05-24 09:47:37 +01:00
Eric Eastwood
c97251d5ba
Add Sliding Sync /sync/e2ee endpoint for To-Device messages ( #17167 )
...
Build release artifacts / Build wheels on ${{ matrix.os }} for ${{ matrix.arch }} (x86_64, ${{ startsWith(github.ref, 'refs/pull/') }}, ubuntu-20.04) (push) Waiting to run
Build release artifacts / Build sdist (push) Waiting to run
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / lint-pydantic (push) Blocked by required conditions
Tests / lint-clippy (push) Blocked by required conditions
Tests / lint-clippy-nightly (push) Blocked by required conditions
Tests / lint-rustfmt (push) Blocked by required conditions
Tests / linting-done (push) Blocked by required conditions
Tests / calculate-test-jobs (push) Blocked by required conditions
Tests / trial (push) Blocked by required conditions
Tests / cargo-bench (push) Blocked by required conditions
Tests / tests-done (push) Blocked by required conditions
Tests / changes (push) Waiting to run
Tests / check-sampleconfig (push) Blocked by required conditions
Tests / check-schema-delta (push) Blocked by required conditions
Tests / check-lockfile (push) Waiting to run
Tests / lint (push) Blocked by required conditions
Tests / Typechecking (push) Blocked by required conditions
Tests / lint-crlf (push) Waiting to run
Tests / lint-newsfile (push) Waiting to run
Tests / trial-olddeps (push) Blocked by required conditions
Tests / trial-pypy (all, pypy-3.8) (push) Blocked by required conditions
Tests / sytest (push) Blocked by required conditions
Tests / export-data (push) Blocked by required conditions
Tests / portdb (11, 3.8) (push) Blocked by required conditions
Tests / portdb (15, 3.11) (push) Blocked by required conditions
Tests / complement (monolith, Postgres) (push) Blocked by required conditions
Tests / complement (monolith, SQLite) (push) Blocked by required conditions
Tests / complement (workers, Postgres) (push) Blocked by required conditions
Tests / cargo-test (push) Blocked by required conditions
This is being introduced as part of Sliding Sync but doesn't have any sliding window component. It's just a way to get E2EE events without having to sit through a big initial sync (`/sync` v2). And we can avoid encryption events being backed up by the main sync response or vice-versa.
Part of some Sliding Sync simplification/experimentation. See [this discussion](https://github.com/element-hq/synapse/pull/17167#discussion_r1610495866 ) for why it may not be as useful as we thought.
Based on:
- https://github.com/matrix-org/matrix-spec-proposals/pull/3575
- https://github.com/matrix-org/matrix-spec-proposals/pull/3885
- https://github.com/matrix-org/matrix-spec-proposals/pull/3884
2024-05-23 12:06:16 -05:00
reivilibre
7e2412265d
Log exceptions when failing to auto-join new user according to the auto_join_rooms option. ( #17176 )
...
Would have been useful for tracking down #16878 .
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2024-05-22 14:22:33 +01:00
reivilibre
7ef00b7628
Add logging to tasks managed by the task scheduler, showing CPU and database usage. ( #17219 )
...
The log format is the same as the request log format, except:
- fields that are specific to HTTP requests have been removed
- the task's params are included at the end of the log line.
These log lines are emitted:
- when the task function finishes — both completion and failure (and I
suppose it is possible for a task to become schedulable again?)
- every 5 minutes whilst it is running
Closes #17217 .
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2024-05-22 14:12:58 +01:00
Erik Johnston
b71d277438
Reduce work of calculating outbound device pokes ( #17211 )
2024-05-22 13:55:18 +01:00
Yadd
a547b49773
Update Lemonldap-NG OIDC config ( #17204 )
...
Update OIDC documentation: by default Matrix doesn't query userinfo endpoint, then claims should be put on id_token.
2024-05-22 12:29:31 +00:00
devonh
6a9a641fb8
Bring auto-accept invite logic into Synapse ( #17147 )
...
This PR ports the logic from the
[synapse_auto_accept_invite](https://github.com/matrix-org/synapse-auto-accept-invite )
module into synapse.
I went with the naive approach of injecting the "module" next to where
third party modules are currently loaded. If there is a better/preferred
way to handle this, I'm all ears. It wasn't obvious to me if there was a
better location to add this logic that would cleanly apply to all
incoming invite events.
Relies on https://github.com/element-hq/synapse/pull/17166 to fix linter
errors.
2024-05-21 20:09:17 +00:00
Erik Johnston
b5facbac0f
Improve perf of sync device lists ( #17216 )
...
Re-introduces #17191 , and includes #17197 and #17214
The basic idea is to stop calling `get_rooms_for_user` everywhere, and
instead use the table `device_lists_changes_in_room`.
Commits reviewable one-by-one.
2024-05-21 16:48:20 +01:00
Erik Johnston
b250ca5df2
Merge branch 'release-v1.108' into develop
2024-05-21 12:21:26 +01:00
Erik Johnston
e0d420fbd1
Fixup changelog
2024-05-21 10:57:58 +01:00
dependabot[bot]
9956f35c6a
Bump twine from 5.0.0 to 5.1.0 ( #17225 )
2024-05-21 10:57:31 +01:00
dependabot[bot]
d464ee3602
Bump bcrypt from 4.1.2 to 4.1.3 ( #17224 )
2024-05-21 10:57:08 +01:00
dependabot[bot]
439a095edc
Bump pyasn1 from 0.5.1 to 0.6.0 ( #17223 )
2024-05-21 10:56:40 +01:00
dependabot[bot]
5d040f2066
Bump types-psycopg2 from 2.9.21.20240311 to 2.9.21.20240417 ( #17222 )
2024-05-21 10:56:24 +01:00
dependabot[bot]
f33266232e
Bump serde from 1.0.201 to 1.0.202 ( #17221 )
2024-05-21 10:56:16 +01:00
dependabot[bot]
d43042864a
Bump anyhow from 1.0.83 to 1.0.86 ( #17220 )
2024-05-21 10:56:07 +01:00
Erik Johnston
f4ce030608
Fixup changelog
2024-05-21 10:55:22 +01:00
Erik Johnston
8b43cc89fa
1.108.0rc1
2024-05-21 10:54:19 +01:00
Erik Johnston
52af16c561
Add a short sleep if the request is rate-limited ( #17210 )
...
This helps prevent clients from "tight-looping" retrying their request.
2024-05-18 12:03:30 +01:00
Dominic Schubert
38f03a09ff
Federated Knocking Endpoints added (missing in Docu) ( #17058 )
2024-05-17 09:54:11 +00:00
Eric Eastwood
c856ae4724
Refactor SyncResultBuilder assembly to its own function ( #17202 )
...
We will re-use `get_sync_result_builder(...)` in
https://github.com/element-hq/synapse/pull/17167
Split out from https://github.com/element-hq/synapse/pull/17167
2024-05-16 13:05:31 -05:00
Eric Eastwood
fe07995e69
Fix joined_rooms/joined_room_ids usage ( #17208 )
...
This change was introduced in
https://github.com/element-hq/synapse/pull/17203
But then https://github.com/element-hq/synapse/pull/17207 was reverted
which brought back usage `joined_rooms` that needed to be updated.
Wasn't caught because `develop` wasn't up to date before merging.
2024-05-16 17:27:38 +00:00
Eric Eastwood
52a649580f
Rename to be obvious: joined_rooms -> joined_room_ids ( #17203 )
...
Split out from https://github.com/element-hq/synapse/pull/17167
2024-05-16 11:55:51 -05:00
Eric Eastwood
28a948f04f
Removed request_key from the SyncConfig (moved outside as its own function parameter) ( #17201 )
...
Removed `request_key` from the `SyncConfig` (moved outside as its own function parameter) so it doesn't have to flow into `_generate_sync_entry_for_xxx` methods. This way we can separate the concerns of caching from generating the response and reuse the `_generate_sync_entry_for_xxx` functions as we see fit. Plus caching doesn't really have anything to do with the config of sync.
Split from https://github.com/element-hq/synapse/pull/17167
Spawning from https://github.com/element-hq/synapse/pull/17167#discussion_r1601497279
2024-05-16 11:54:46 -05:00
Andrew Morgan
7cb3f8a979
Route /make_knock and /send_knock to workers in Complement docker image ( #17195 )
2024-05-16 16:53:26 +00:00
Erik Johnston
fd12003441
Revert "Improve perf of sync device lists" ( #17207 )
...
Reverts element-hq/synapse#17191
2024-05-16 16:07:54 +01:00
Erik Johnston
5e892671a7
Fix bug where push rules would be empty in /sync ( #17142 )
...
Fixes #16987
Some old accounts seem to have an entry in global account data table for
push rules, which we should ignore
2024-05-16 15:04:14 +01:00
Eric Eastwood
d2d48cce85
Refactor Sync handler to be able to return different sync responses (SyncVersion) ( #17200 )
...
Refactor Sync handler to be able to be able to return different sync
responses (`SyncVersion`). Preparation to be able support sync v2 and a
new Sliding Sync `/sync/e2ee` endpoint which returns a subset of sync
v2.
Split upon request:
https://github.com/element-hq/synapse/pull/17167#discussion_r1601497279
Split from https://github.com/element-hq/synapse/pull/17167 where we
will add `SyncVersion.E2EE_SYNC` and a new type of sync response.
2024-05-16 11:36:54 +01:00
Andrew Morgan
2359c64dec
Fix request path for federation_whitelist_endpoint_enabled option in documentation ( #17199 )
2024-05-15 17:26:22 +01:00
Andrew Morgan
68dca8076f
Merge branch 'master' into develop
2024-05-14 15:21:42 +01:00
Erik Johnston
284d85dee3
Cache literal sync filter validation ( #17186 )
...
The sliding sync proxy (amongst other things) use literal json blobs as
filters, and repeatedly validating them takes a bunch of CPU.
2024-05-14 15:08:46 +01:00
Erik Johnston
ebe77381b0
Reduce pauses on large device list changes ( #17192 )
...
For large accounts waking up all the relevant notifier streams can cause
pauses of the reactor.
2024-05-14 14:39:11 +01:00
Erik Johnston
0b91ccce47
Improve perf of sync device lists ( #17191 )
...
It's almost always more efficient to query the rooms that have device
list changes, rather than looking at the list of all users whose devices
have changed and then look for shared rooms.
2024-05-14 14:39:04 +01:00
Andrew Morgan
ecf4e0674c
1.107.0
2024-05-14 14:15:41 +01:00
Aurélien Grimpard
7d82987b27
Allows CAS SSO flow to provide user IDs composed of numbers only ( #17098 )
2024-05-14 13:55:32 +01:00
dependabot[bot]
bd8d8865fb
Bump serde_json from 1.0.116 to 1.0.117 ( #17182 )
2024-05-14 09:32:23 +01:00
dependabot[bot]
caf528477e
Bump serde from 1.0.200 to 1.0.201 ( #17183 )
2024-05-14 09:32:14 +01:00
dependabot[bot]
f0c72d8e87
Bump gitpython from 3.1.41 to 3.1.43 ( #17181 )
2024-05-14 09:32:01 +01:00
