mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-11-26 11:36:03 +03:00
Code Issues Projects Releases Packages Wiki Activity
21418 commits 935 branches 728 tags 494 MiB
Commit graph

6481 commits

Author SHA1 Message Date
David Robertson
0a38c7ec6d
Snapshot schema 72 (#13873) 
Including another batch of fixes to the schema dump script
 2022-09-26 18:28:32 +01:00
Mathieu Velten
41461fd4d6
typing: check origin server of typing event against room's servers (#13830) 
This is also using the partial state approximation if needed so we do
not block here during a fast join.

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2022-09-26 17:33:32 +02:00
Nick Mills-Barrett
6b4593a80f
Simplify cache invalidation after event persist txn (#13796) 
This moves all the invalidations into a single place and de-duplicates
the code involved in invalidating caches for a given event by using
the base class method.
 2022-09-26 16:26:35 +01:00
enterprisey
f34b0bc262
Update NixOS module URL (#13818) 
* Update NixOS module URL

* Create 13818.doc
 2022-09-26 15:26:59 +01:00
Erik Johnston
c5defa4cba
Correctly add Cargo.lock to sdsit (#13909) 2022-09-26 13:58:10 +00:00
David Robertson
dcdd50e458
Fix mypy errors with latest canonicaljson (#13905) 
* Lockfile: update canonicaljson 1.6.0 -> 1.6.3

* Fix mypy errors with latest canonicaljson

The change to `_encode_json_bytes` definition wasn't sufficient:

```
synapse/http/server.py:751: error: Incompatible types in assignment (expression has type "Callable[[Arg(object, 'json_object')], bytes]", variable has type "Callable[[Arg(object, 'data')], bytes]")  [assignment]
```

Which I think is mypy warning us that the two functions accept different
sets of kwargs. Fair enough!

* Changelog
 2022-09-26 13:30:00 +01:00
Eric Eastwood
ac1a31740b
Only try to backfill event if we haven't tried before recently (#13635) 
Only try to backfill event if we haven't tried before recently (exponential backoff). No need to keep trying the same backfill point that fails over and over.

Fix https://github.com/matrix-org/synapse/issues/13622
Fix https://github.com/matrix-org/synapse/issues/8451

Follow-up to https://github.com/matrix-org/synapse/pull/13589

Part of https://github.com/matrix-org/synapse/issues/13356
 2022-09-23 14:01:29 -05:00
Sean Quah
f49f73c0da
Faster room joins: Avoid blocking /keys/changes (#13888) 
Part of the work for #12993.

Once #12993 is fully resolved, we expect `/keys/changes` to behave
sensibly when joined to a room with partial state.

Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-09-23 17:55:15 +01:00
Eric Eastwood
db868db594
Fix access token leak to logs from proxyagent (#13855) 
This can happen specifically with an application service `/transactions/10722?access_token=leaked` request

Fix https://github.com/matrix-org/synapse/issues/13010

---

Saw an example leak in https://github.com/matrix-org/synapse/issues/13423#issuecomment-1205348482

```
2022-08-04 14:47:57,925 - synapse.http.client - 401 - DEBUG - as-sender-signal-1 - Sending request PUT http://localhost:29328/transactions/10722?access_token=<redacted>
2022-08-04 14:47:57,926 - synapse.http.proxyagent - 223 - DEBUG - as-sender-signal-1 - Requesting b'http://localhost:29328/transactions/10722?access_token=leaked' via <HostnameEndpoint localhost:29328>
```
 2022-09-23 11:49:39 -05:00
Patrick Cloke
e3512a7719
Update Cargo.lock file. (#13889) 2022-09-23 11:59:39 -04:00
Patrick Cloke
efd108b45d
Accept & store thread IDs for receipts (implement MSC3771). (#13782) 
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
 2022-09-23 14:33:28 +00:00
Sean Quah
03c2bfb7f8
Send device list updates out to servers in partially joined rooms (#13874) 
Use the provided list of servers in the room from the `/send_join`
response, since we will not know which users are in the room.  This
isn't sufficient to ensure that all remote servers receive the right
device list updates, since the `/send_join` response may be inaccurate
or we may calculate the membership state of new users in the room
incorrectly.

Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-09-23 13:44:03 +01:00
reivilibre
c06b2b7142
Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823) 2022-09-23 11:47:16 +01:00
reivilibre
ac7e5683d6
Add comments to the Prometheus recording rules to make it clear which set of rules you need for Grafana or Prometheus Console. (#13876) 2022-09-23 11:46:45 +01:00
David Robertson
f7c89c44c5
Raise issue if complement fails with latest deps (#13859) 2022-09-22 17:33:37 +01:00
Patrick Cloke
8c3dcdf1b9
Fix the cross-link from register admin API to config docs. (#13870) 2022-09-22 09:11:50 -04:00
Patrick Cloke
b7272b73aa
Properly paginate forward in the /relations API. (#13840) 
This fixes a bug where the `/relations` API with `dir=f` would
skip the first item of each page (except the first page), causing
incomplete data to be returned to the client.
 2022-09-22 12:47:49 +00:00
David Robertson
1a1abdda42
Last batch of Pydantic for synapse/rest/client/account.py (#13832) 
* Validation for `/add_threepid/msisdn/submit_token`

* Don't validate deprecated endpoint

* Changelog
 2022-09-21 22:23:44 +01:00
Brendan Abolivier
efabf44c76
Add version flag for MSC3881 (#13860) 2022-09-21 18:18:44 +02:00
Brendan Abolivier
ccca14140a
Track device IDs for pushers (#13831) 
Second half of the MSC3881 implementation
 2022-09-21 15:31:53 +00:00
Hugh Nimmo-Smith
0fd2f2d460
Implementation of MSC3882 login token request (#13722) 2022-09-21 15:12:29 +00:00
villepeh
269eddad6f
Add worker_main_http_uri to the contrib bash script (#13772) 
* Add worker_main_http_uri, replace >> with >

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
 2022-09-21 15:58:46 +01:00
Brendan Abolivier
8ae42ab8fa
Support enabling/disabling pushers (from MSC3881) (#13799) 
Partial implementation of MSC3881
 2022-09-21 14:39:01 +00:00
Mathieu Velten
6bd8763804
Add cache invalidation across workers to module API (#13667) 
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-09-21 15:32:01 +02:00
Peter Scheu
16e1a9d9a7
Correct documentation for map_user_attributes of OpenID Mapping Providers (#13836) 
Co-authored-by: David Robertson <davidr@element.io>
 2022-09-21 13:08:16 +00:00
Quentin Gliech
e0804ef898
Improve the synapse.api.auth.Auth mock used in unit tests. (#13809) 
To return the proper type (`Requester`) instead of a `dict`.
 2022-09-21 12:40:34 +00:00
reivilibre
a35842caec
Fix the release script not publishing binary wheels. (#13850) 2022-09-21 09:43:08 +00:00
Olivier Wilkinson (reivilibre)
2b522cceb6 Merge branch 'release-v1.68' into develop 2022-09-20 14:54:05 +01:00
Quentin Gliech
85fc7ea1a1
Remove the complete_sso_login method from the Module API which was deprecated in Synapse 1.13.0. (#13843) 
Signed-off-by: Quentin Gliech <quenting@element.io>
 2022-09-20 15:18:07 +02:00
David Robertson
fff9b955fa
Generate separate snapshots for logical databases (#13792) 
* Generate separate snapshots for sqlite, postgres and common
* Cleanup postgres dbs in the TRAP
* Say which logical DB we're applying updates to
* Run background updates on the state DB
* Add new option for accepting a SCHEMA_NUMBER
 2022-09-20 14:14:12 +01:00
Erik Johnston
42d261c32f
Port the push rule classes to Rust. (#13768) 2022-09-20 12:10:31 +01:00
Olivier Wilkinson (reivilibre)
e124b24ad9 1.68.0rc1 2022-09-20 11:30:48 +01:00
Denis
c802ef1411
Don't include redundant prev_state in new events (#13791) 2022-09-20 09:44:38 +01:00
Eric Eastwood
44be42338e
Add support to purge rows from MSC2716 and other tables when purging a room (#13825) 
`event_failed_pull_attempts` added in https://github.com/matrix-org/synapse/pull/13589

MSC2716 related tables added in:

 - https://github.com/matrix-org/synapse/pull/10245/files#diff-3d42dfb44d02f7de3aada105e0bdc1cc9dd7f953cbf0f36c5d0f50827bf0320aR1
    - Renamed in https://github.com/matrix-org/synapse/pull/10838/files#diff-2730bfbe9e688b55e46f9371aefe67dac2bd2b2b7d9d6b92774eea1fcfae156dR1
 - https://github.com/matrix-org/synapse/pull/10498/files#diff-c52bbfbb5921a3f6f023b24343668479d966fac164f13b7c39d2197ce3afa7a5R1
 2022-09-16 10:56:56 -05:00
Mathieu Velten
d5292b8017
Fix Docker build when Rust .so has been build locally first (#13811) 
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-09-16 15:38:54 +00:00
David Robertson
642c4b253d
Compare ported to unported PG schemas in portdb test job (#13808) 2022-09-16 16:25:54 +01:00
David Robertson
5e84461653
Minor speedups to CI linting (#13827) 2022-09-16 16:18:32 +01:00
Sean Quah
d64e85197a
Remove error spam when users query the keys of departed remote users (#13826) 
The error message introduced in #13749 has turned out to be very spammy.
Remove it for now.
 2022-09-16 16:16:05 +01:00
Mathieu Velten
384dca53d6
complement: init postgres DB directly inside the target image (#13819) 
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-09-16 17:12:45 +02:00
Quentin Gliech
74f60cec92
Add an admin API endpoint to find a user based on its external ID in an auth provider. (#13810) 2022-09-16 12:29:03 +00:00
reivilibre
f7a77ad717
Update request log format documentation to mention the format used when the authenticated user is controlling another user. (#13794) 2022-09-16 11:48:41 +00:00
Sean Quah
b73cbb8215
Avoid putting rejected events in room state (#13723) 
Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-09-16 12:45:04 +01:00
Eric Eastwood
6986bcbf39
Document common fix of Poetry problems by removing egg-info (#13785) 
`matrix_synapse.egg-info/`

Mentioned at https://matrix.to/#/!vcyiEtMVHIhWXcJAfl:sw1v.org/$aKy_IjrKwb70aTVZWeW_6zt0k7OIZ1YkyZpkP9uiRaM?via=matrix.org&via=element.io&via=beeper.com and many other places.
 2022-09-15 16:28:03 -05:00
Eric Eastwood
5093cbf88d
Be able to correlate timeouts in reverse-proxy layer in front of Synapse (pull request ID from header) (#13801) 
Fix https://github.com/matrix-org/synapse/issues/13685

New config:

```diff
  listeners:
    - port: 8008
      tls: false
      type: http
      x_forwarded: true
+     request_id_header: "cf-ray"
      bind_addresses: ['::1', '127.0.0.1', '0.0.0.0']
```
 2022-09-15 15:32:25 -05:00
Eric Eastwood
140af0cdb6
Record any exception when processing a pulled event (#13814) 
Part of https://github.com/matrix-org/synapse/issues/13700 and https://github.com/matrix-org/synapse/issues/13356

Follow-up to https://github.com/matrix-org/synapse/pull/13589
 2022-09-15 14:40:49 -05:00
Patrick Cloke
b2b0c85279
Support providing an index predicate for upserts. (#13822) 
This is useful to upsert against a table which has a unique
partial index while avoiding conflicts.
 2022-09-15 18:28:48 +00:00
David Robertson
742f9f9d78
A third batch of Pydantic validation for rest/client/account.py (#13736) 2022-09-15 18:36:02 +01:00
Andrew Morgan
918c74bfb5
Add a MXCUri class to make working with mxc uri's easier. (#13162) 2022-09-15 12:57:16 +00:00
Eric Eastwood
957e3d74fc
Keep track when we try and fail to process a pulled event (#13589) 
We can follow-up this PR with:

 1. Only try to backfill from an event if we haven't tried recently -> https://github.com/matrix-org/synapse/issues/13622
 1. When we decide to backfill that event again, process it in the background so it doesn't block and make `/messages` slow when we know it will probably fail again -> https://github.com/matrix-org/synapse/issues/13623
 1. Generally track failures everywhere we try and fail to pull an event over federation -> https://github.com/matrix-org/synapse/issues/13700

Fix https://github.com/matrix-org/synapse/issues/13621

Part of https://github.com/matrix-org/synapse/issues/13356

Mentioned in [internal doc](https://docs.google.com/document/d/1lvUoVfYUiy6UaHB6Rb4HicjaJAU40-APue9Q4vzuW3c/edit#bookmark=id.qv7cj51sv9i5)
 2022-09-14 13:57:50 -05:00
Patrick Cloke
666ae87729
Update event push action and receipt tables to support threads. (#13753) 
Adds a `thread_id` column to the `event_push_actions`, `event_push_actions_staging`,
and `event_push_summary` tables. This will notifications to be segmented by the thread
in a future pull request. The `thread_id` column stores the root event ID or the special
value `"main"`.

The `thread_id` column for `event_push_actions` and `event_push_summary` is
backfilled with `"main"` for all existing rows. New entries into `event_push_actions`
and `event_push_actions_staging` will get the proper thread ID.

`receipts_linearized` and `receipts_graph` also gain a `thread_id` column, which is similar,
except `NULL` is a special value meaning the receipt is "unthreaded".

See MSC3771 and MSC3773 for where this data will be useful.
 2022-09-14 17:11:16 +00:00