mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-11-21 17:15:38 +03:00
Code Issues Projects Releases Packages Wiki Activity
24394 commits 934 branches 728 tags 485 MiB
Commit graph

162 commits

Author SHA1 Message Date
Quentin Gliech
7d52ce7d4b
Format files with Ruff (#17643) 
I thought ruff check would also format, but it doesn't.

This runs ruff format in CI and dev scripts. The first commit is just a
run of `ruff format .` in the root directory.
 2024-09-02 12:39:04 +01:00
Shay
cf69f8d59b
Support MSC3916 by adding a federation /thumbnail endpoint and authenticated _matrix/client/v1/media/thumbnail endpoint (#17388) 
[MSC3916](https://github.com/matrix-org/matrix-spec-proposals/pull/3916)
added the endpoints `_matrix/federation/v1/media/thumbnail` and the
authenticated `_matrix/client/v1/media/thumbnail`.

This PR implements those endpoints, along with stabilizing
`_matrix/client/v1/media/config` and
`_matrix/client/v1/media/preview_url`.

Complement tests are at
https://github.com/matrix-org/complement/pull/728
 2024-07-08 10:11:20 +01:00
Shay
8f890447b0
Support MSC3916 by adding _matrix/client/v1/media/download endpoint (#17365) 2024-07-02 14:07:04 +01:00
Erik Johnston
2f5a77efae
Limit size of presence EDUs (#17371) 
Otherwise they are unbounded.

---------

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2024-07-01 12:48:36 +01:00
Shay
a023538822
Re-introduce federation /download endpoint (#17350) 2024-06-25 14:35:37 +00:00
Andrew Morgan
199223062a
Revert "Support MSC3916 by adding a federation /download endpoint" (#17325)
Some checks are pending
Build release artifacts / Build wheels on ${{ matrix.os }} for ${{ matrix.arch }} (x86_64, ${{ startsWith(github.ref, 'refs/pull/') }}, ubuntu-20.04) (push) Waiting to run
Build release artifacts / Build sdist (push) Waiting to run
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / changes (push) Waiting to run
Tests / check-sampleconfig (push) Blocked by required conditions
Tests / check-schema-delta (push) Blocked by required conditions
Tests / check-lockfile (push) Waiting to run
Tests / lint (push) Blocked by required conditions
Tests / calculate-test-jobs (push) Blocked by required conditions
Tests / Typechecking (push) Blocked by required conditions
Tests / lint-crlf (push) Waiting to run
Tests / lint-newsfile (push) Waiting to run
Tests / lint-pydantic (push) Blocked by required conditions
Tests / lint-clippy (push) Blocked by required conditions
Tests / lint-clippy-nightly (push) Blocked by required conditions
Tests / lint-rustfmt (push) Blocked by required conditions
Tests / linting-done (push) Blocked by required conditions
Tests / trial (push) Blocked by required conditions
Tests / trial-olddeps (push) Blocked by required conditions
Tests / trial-pypy (all, pypy-3.8) (push) Blocked by required conditions
Tests / sytest (push) Blocked by required conditions
Tests / export-data (push) Blocked by required conditions
Tests / portdb (11, 3.8) (push) Blocked by required conditions
Tests / portdb (15, 3.11) (push) Blocked by required conditions
Tests / complement (monolith, Postgres) (push) Blocked by required conditions
Tests / complement (monolith, SQLite) (push) Blocked by required conditions
Tests / complement (workers, Postgres) (push) Blocked by required conditions
Tests / cargo-test (push) Blocked by required conditions
Tests / cargo-bench (push) Blocked by required conditions
Tests / tests-done (push) Blocked by required conditions
 2024-06-18 16:54:19 +01:00
Shay
ab94bce02c
Support MSC3916 by adding a federation /download endpoint (#17172)
Some checks failed
Tests / check-schema-delta (push) Has been cancelled
Build release artifacts / Build wheels on ${{ matrix.os }} for ${{ matrix.arch }} (x86_64, ${{ startsWith(github.ref, 'refs/pull/') }}, macos-11) (push) Has been cancelled
Build release artifacts / Build .deb packages (push) Has been cancelled
Build release artifacts / Build wheels on ${{ matrix.os }} for ${{ matrix.arch }} (x86_64, ${{ startsWith(github.ref, 'refs/pull/') }}, ubuntu-20.04) (push) Has been cancelled
Build release artifacts / Build sdist (push) Has been cancelled
Tests / changes (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / Typechecking (push) Has been cancelled
Tests / lint-pydantic (push) Has been cancelled
Tests / lint-clippy (push) Has been cancelled
Tests / lint-clippy-nightly (push) Has been cancelled
Tests / lint-rustfmt (push) Has been cancelled
Tests / linting-done (push) Has been cancelled
Tests / calculate-test-jobs (push) Has been cancelled
Deploy the documentation / GitHub Pages (push) Has been cancelled
Tests / check-sampleconfig (push) Has been cancelled
Build release artifacts / Attach assets to release (push) Has been cancelled
Tests / trial (push) Has been cancelled
Tests / trial-olddeps (push) Has been cancelled
Tests / trial-pypy (all, pypy-3.8) (push) Has been cancelled
Tests / sytest (push) Has been cancelled
Tests / export-data (push) Has been cancelled
Tests / portdb (11, 3.8) (push) Has been cancelled
Tests / portdb (15, 3.11) (push) Has been cancelled
Tests / complement (monolith, Postgres) (push) Has been cancelled
Tests / complement (monolith, SQLite) (push) Has been cancelled
Tests / complement (workers, Postgres) (push) Has been cancelled
Tests / cargo-test (push) Has been cancelled
Tests / cargo-bench (push) Has been cancelled
Tests / tests-done (push) Has been cancelled
 2024-06-07 13:54:28 +01:00
Timshel
34a8652366
Optional whitespace support in Authorization (#1350) (#17145) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2024-05-08 13:56:16 +00:00
Richard van der Hoff
c897ac63e9
Ensure that incoming to-device messages are not dropped (#17127) 
... when workers are unreachable, etc.

Fixes https://github.com/element-hq/synapse/issues/17117.

The general principle is just to make sure that we propagate any
exceptions to the JsonResource, so that we return an error code to the
sending server. That means that the sending server no longer considers
the message safely sent, so it will retry later.

In the issue, Erik mentions that an alternative solution would be to
persist the to-device messages into a table so that they can be retried.
This might be an improvement for performance, but even if we did that,
we still need this mechanism, since we might be unable to reach the
database. So, if we want to do that, it can be a later follow-up.

---------

Co-authored-by: Erik Johnston <erik@matrix.org>
 2024-04-29 14:11:00 +01:00
Erik Johnston
23740eaa3d
Correctly mention previous copyright (#16820) 
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
 2024-01-23 11:26:48 +00:00
Patrick Cloke
8e1e62c9e0 Update license headers 2023-11-21 15:29:58 -05:00
Patrick Cloke
f2f2c7c1f0
Use full GitHub links instead of bare issue numbers. (#16637) 2023-11-15 08:02:11 -05:00
Patrick Cloke
679c691f6f
Remove more usages of cursor_to_dict. (#16551) 
Mostly to improve type safety.
 2023-10-26 15:12:28 -04:00
Patrick Cloke
fc31b495b3
Stop sending incorrect knock_state_events. (#16403) 
Synapse was incorrectly implemented with a knock_state_events
property on some APIs (instead of knock_room_state). This was
correct in Synapse 1.70.0, but *both* fields were sent to also be
compatible with Synapse versions expecting the wrong field.

Enough time has passed that only the correct field needs to be
included/handled.
 2023-10-06 07:27:35 -04:00
Patrick Cloke
f84da3c32e
Add a cache around server ACL checking (#16360) 
* Pre-compiles the server ACLs onto an object per room and
  invalidates them when new events come in.
* Converts the server ACL checking into Rust.
 2023-09-26 11:57:50 -04:00
Erik Johnston
d35bed8369
Don't wake up destination transaction queue if they're not due for retry. (#16223) 2023-09-04 17:14:09 +01:00
Patrick Cloke
9ec3da06da
Bump mypy-zope & mypy. (#16188) 2023-08-29 10:38:56 -04:00
Patrick Cloke
daf11e26ef
Replace make_awaitable with AsyncMock (#16179) 
Python 3.8 provides a native AsyncMock, we can replace the
homegrown version we have.
 2023-08-24 19:38:46 -04:00
Eric Eastwood
887fa4b66b
Switch from matrix:// to matrix-federation:// scheme for internal Synapse routing of outbound federation traffic (#15806) 
`matrix://` is a registered specced scheme nowadays and doesn't make sense for
our internal to Synapse use case anymore. ([discussion]
(https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679))
 2023-06-20 10:05:31 +01:00
Mathieu Velten
f63d4a3a65
Regularly try to wake up dests instead of waiting for next PDU/EDU (#15743) 2023-06-16 10:15:12 +00:00
Patrick Cloke
1e89976b26
Rename blacklist/whitelist internally. (#15620) 
Avoid renaming configuration settings for now and rename internal code
to use blocklist and allowlist instead.
 2023-05-19 12:25:25 +00:00
Patrick Cloke
07771fa487
Remove experimental configuration flags & unstable values for faster joins (#15625) 
Synapse will no longer send (or respond to) the unstable flags
for faster joins. These were only available behind a configuration
flag and handled in parallel with the stable flags.
 2023-05-19 07:23:09 -04:00
Patrick Cloke
ea5c3ede4f
Finish type hints for federation client HTTP code. (#15465) 2023-04-24 13:12:06 -04:00
David Robertson
c071cd5a0e
Ensure fed-sender catchup does not block for full state (#15248) 
* Reproduce bad scenario in test
* Avoid catchup optimisation for partial state rooms
 2023-03-13 12:31:19 +00:00
dependabot[bot]
9bb2eac719
Bump black from 22.12.0 to 23.1.0 (#15103) 2023-02-22 15:29:09 -05:00
Patrick Cloke
42aea0d8af
Add final type hint to tests.unittest. (#15072) 
Adds a return type to HomeServerTestCase.make_homeserver and deal
with any variables which are no longer Any.
 2023-02-14 14:03:35 -05:00
David Robertson
0f34abed7c
Type hints for tests.federation (#14991) 
* Make tests.federation pass mypy

* Untyped defs in tests.federation.transport

* test methods return None

* Remaining type hints in tests.federation

* Changelog

* Avoid an uncessary type-ignore
 2023-02-06 16:05:06 +00:00
Andrew Morgan
836c592f15
Fix type hints in knocking tests. (#14887) 2023-01-25 14:38:20 -05:00
David Robertson
5b3af1c7d0
Stabilise serving partial join responses (#14839) 
Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true.

Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
 2023-01-17 12:44:15 +00:00
David Robertson
85a7a201fa
Also use stable name in SendJoinResponse struct (#14841) 
* Also use stable name in SendJoinResponse struct

follow-up to #14832

* Changelog

* Fix a rename I missed

* Run black

* Update synapse/federation/federation_client.py

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2023-01-16 12:40:25 +00:00
David Robertson
52ae80dd1a
Use stable identifiers for faster joins (#14832) 
* Use new query param when requesting a partial join

* Read new query param when serving partial join

* Provide new field names when serving partial joins

* Read new field names from partial join response

* Changelog
 2023-01-13 17:58:53 +00:00
realtyem
854a6884d8
Modernize unit tests configuration settings for workers. (#14568) 
Use the newer foo_instances configuration instead of the
deprecated flags to enable specific features (e.g. start_pushers).
 2022-12-01 07:38:27 -05:00
Mathieu Velten
4569eda944
Use servers list approx to send read receipts when in partial state (#14549) 
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-11-30 13:39:47 +01:00
Patrick Cloke
d748bbc8f8
Include thread information when sending receipts over federation. (#14466) 
Include the thread_id field when sending read receipts over
federation. This might result in the same user having multiple
read receipts per-room, meaning multiple EDUs must be sent
to encapsulate those receipts.

This restructures the PerDestinationQueue APIs to support
multiple receipt EDUs, queue_read_receipt now becomes linear
time in the number of queued threaded receipts in the room for
the given user, it is expected this is a small number since receipt
EDUs are sent as filler in transactions.
 2022-11-28 14:40:17 +00:00
David Robertson
1eed795fc5
Include heroes in partial join responses' state (#14442) 
* Pull out hero selection logic

* Include heroes in partial join response's state

* Changelog

* Fixup trial test

* Remove TODO
 2022-11-15 17:35:19 +00:00
David Robertson
5905ba12d0
Run trial tests against Python 3.11 (#13812) 2022-11-01 13:07:54 +00:00
Eric Eastwood
40fa8294e3
Refactor MSC3030 /timestamp_to_event to move away from our snowflake pull from destination pattern (#14096) 
1. `federation_client.timestamp_to_event(...)` now handles all `destination` looping and uses our generic `_try_destination_list(...)` helper.
 2. Consistently handling `NotRetryingDestination` and `FederationDeniedError` across `get_pdu` , backfill, and the generic `_try_destination_list` which is used for many places we use this pattern.
 3. `get_pdu(...)` now returns `PulledPduInfo` so we know which `destination` we ended up pulling the PDU from
 2022-10-26 16:10:55 -05:00
Quentin Gliech
9192d74b0b
Refactor OIDC tests to better mimic an actual OIDC provider. (#13910) 
This implements a fake OIDC server, which intercepts calls to the HTTP client.
Improves accuracy of tests by covering more internal methods.

One particular example was the ID token validation, which previously mocked.

This uncovered an incorrect dependency: Synapse actually requires at least
authlib 0.15.1, not 0.14.0.
 2022-10-25 14:25:02 +00:00
dependabot[bot]
0b7830e457
Bump flake8-bugbear from 21.3.2 to 22.9.23 (#14042) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
Co-authored-by: David Robertson <davidr@element.io>
 2022-10-19 19:38:24 +00:00
David Robertson
cb20b885cb
Always close _all_ ijson coroutines, even if doing so raises Exceptions (#14065) 2022-10-06 18:17:50 +00:00
Eric Eastwood
70a4317692
Track when the pulled event signature fails (#13815) 
Because we're doing the recording in `_check_sigs_and_hash_for_pulled_events_and_fetch` (previously named `_check_sigs_and_hash_and_fetch`), this means we will track signature failures for `backfill`, `get_room_state`, `get_event_auth`, and `get_missing_events` (all pulled event scenarios). And we also record signature failures from `get_pdu`.

Part of https://github.com/matrix-org/synapse/issues/13700

Part of https://github.com/matrix-org/synapse/issues/13676 and https://github.com/matrix-org/synapse/issues/13356

This PR will be especially important for https://github.com/matrix-org/synapse/pull/13816 so we can avoid the costly `_get_state_ids_after_missing_prev_event` down the line when `/messages` calls backfill.
 2022-10-03 14:53:29 -05:00
Patrick Cloke
efd108b45d
Accept & store thread IDs for receipts (implement MSC3771). (#13782) 
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
 2022-09-23 14:33:28 +00:00
reivilibre
7bc110a19e
Generalise the @cancellable annotation so it can be used on functions other than just servlet methods. (#13662) 2022-08-31 11:16:05 +00:00
Eric Eastwood
1a209efdb2
Update get_users_in_room mis-use to get hosts with dedicated get_current_hosts_in_room (#13605) 
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755
 2022-08-24 14:15:37 -05:00
Dirk Klimpel
e2ed1b7155
Use literals in place of HTTPStatus constants in tests (#13463) 2022-08-05 16:59:09 +02:00
Eric Eastwood
0f971ca68e
Update get_pdu to return the original, pristine EventBase (#13320) 
Update `get_pdu` to return the untouched, pristine `EventBase` as it was originally seen over federation (no metadata added). Previously, we returned the same `event` reference that we stored in the cache which downstream code modified in place and added metadata like setting it as an `outlier`  and essentially poisoned our cache. Now we always return a copy of the `event` so the original can stay pristine in our cache and re-used for the next cache call.

Split out from https://github.com/matrix-org/synapse/pull/13205

As discussed at:

 - https://github.com/matrix-org/synapse/pull/13205#discussion_r918365746
 - https://github.com/matrix-org/synapse/pull/13205#discussion_r918366125

Related to https://github.com/matrix-org/synapse/issues/12584. This PR doesn't fix that issue because it hits [`get_event` which exists from the local database before it tries to `get_pdu`](7864f33e28/synapse/federation/federation_client.py (L581-L594)).
 2022-07-20 15:58:51 -05:00
Shay
a1b62af2af
Validate federation destinations and log an error if server name is invalid. (#13318) 2022-07-20 11:17:26 -07:00
David Robertson
b977867358
Rate limit joins per-room (#13276) 2022-07-19 11:45:17 +00:00
Dirk Klimpel
96cf81e312
Use HTTPStatus constants in place of literals in tests. (#13297) 2022-07-15 19:31:27 +00:00
David Robertson
52a0c8f2f7
Rename test case method to add_hashes_and_signatures_from_other_server (#13255) 2022-07-12 18:46:32 +00:00