Commit graph

25154 commits

Author SHA1 Message Date
Erik Johnston
284d85dee3
Cache literal sync filter validation (#17186)
The sliding sync proxy (amongst other things) use literal json blobs as
filters, and repeatedly validating them takes a bunch of CPU.
2024-05-14 15:08:46 +01:00
Erik Johnston
ebe77381b0
Reduce pauses on large device list changes (#17192)
For large accounts waking up all the relevant notifier streams can cause
pauses of the reactor.
2024-05-14 14:39:11 +01:00
Erik Johnston
0b91ccce47
Improve perf of sync device lists (#17191)
It's almost always more efficient to query the rooms that have device
list changes, rather than looking at the list of all users whose devices
have changed and then look for shared rooms.
2024-05-14 14:39:04 +01:00
Andrew Morgan
ecf4e0674c 1.107.0 2024-05-14 14:15:41 +01:00
Aurélien Grimpard
7d82987b27
Allows CAS SSO flow to provide user IDs composed of numbers only (#17098) 2024-05-14 13:55:32 +01:00
dependabot[bot]
bd8d8865fb
Bump serde_json from 1.0.116 to 1.0.117 (#17182) 2024-05-14 09:32:23 +01:00
dependabot[bot]
caf528477e
Bump serde from 1.0.200 to 1.0.201 (#17183) 2024-05-14 09:32:14 +01:00
dependabot[bot]
f0c72d8e87
Bump gitpython from 3.1.41 to 3.1.43 (#17181) 2024-05-14 09:32:01 +01:00
dependabot[bot]
03a342b049
Bump cryptography from 42.0.5 to 42.0.7 (#17180) 2024-05-14 09:31:46 +01:00
dependabot[bot]
aa6345cb3b
Bump immutabledict from 4.1.0 to 4.2.0 (#17179) 2024-05-14 09:31:35 +01:00
dependabot[bot]
2b438df9b3
Bump sentry-sdk from 1.40.3 to 2.1.1 (#17178) 2024-05-14 09:31:25 +01:00
Erik Johnston
038b9ec59a
An federation whitelist query endpoint extension (#16848)
This is to allow clients to query the configured federation whitelist.
Disabled by default.

---------

Co-authored-by: Devon Hudson <devonhudson@librem.one>
Co-authored-by: devonh <devon.dmytro@gmail.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-05-13 19:38:45 +00:00
Erik Johnston
cef3519e96 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2024-05-13 13:11:28 +01:00
Erik Johnston
59ac541310
Actually fix public rooms (#17184)
See #17177.

I'm an idiot and moved them to the wrong store 🤦
2024-05-13 13:11:07 +01:00
Erik Johnston
784c048aa4 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2024-05-13 12:12:54 +01:00
Erik Johnston
a2e6f43f11
Fix bug with creating public rooms on workers (#17177)
If room publication is disabled then creating public rooms on workers
would not work.

Introduced in #16811.
2024-05-13 12:12:26 +01:00
ll-SKY-ll
4cf4a8281b
Update docs to bump libjemalloc version on latest debian; correct "push_rules" stream name (#17171) 2024-05-09 10:50:05 +00:00
Andrew Morgan
ef7e040e54
Bump black from 24.2.0 to 24.4.2 (#17170) 2024-05-08 17:02:09 +01:00
devonh
393429d692
Fix undiscovered linter errors (#17166)
Linter errors are showing up in #17147 that are unrelated to that PR.
The errors do not currently show up on develop.

This PR aims to resolve the linter errors separately from #17147.
2024-05-08 14:57:32 +00:00
Timshel
34a8652366
Optional whitespace support in Authorization (#1350) (#17145)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-05-08 13:56:16 +00:00
Erik Johnston
414ddcd457
Update PyO3 to 0.21 (#17162)
This version change requires a migration to a new API. See
https://pyo3.rs/v0.21.2/migration#from-020-to-021

This will fix the annoying warnings added when using the recent rust
nightly:

> warning: non-local `impl` definition, they should be avoided as they
go against expectation
2024-05-08 14:30:06 +01:00
Andrew Morgan
4d408cb4dd
Note preset behaviour in autocreate_auto_join_room_preset docs (#17150) 2024-05-08 13:05:10 +01:00
Hugh Nimmo-Smith
212f150208
Add note about MSC3886 being closed (#17151) 2024-05-08 12:49:32 +01:00
Olivier 'reivilibre
4c6e78fa14 Merge branch 'release-v1.107' into develop 2024-05-07 18:52:15 +01:00
Jacob Sánchez
1b155362ca
Add note about external_ids for User Admin API in documentation (#17139) 2024-05-07 16:38:29 +00:00
Olivier 'reivilibre
522a40c4de Tweak changelog 2024-05-07 17:25:47 +01:00
Olivier 'reivilibre
1629429538 Merge remote-tracking branch 'origin/release-v1.107' into matrix-org-hotfixes 2024-05-07 16:35:57 +01:00
Olivier 'reivilibre
dcd03d3b15 1.107.0rc1 2024-05-07 16:30:07 +01:00
Erik Johnston
fa68816fb8 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2024-05-07 11:48:08 +01:00
dependabot[bot]
438bc23560
Bump serde from 1.0.199 to 1.0.200 (#17161) 2024-05-07 10:35:37 +01:00
dependabot[bot]
cf30cfe5d1
Bump pydantic from 2.7.0 to 2.7.1 (#17160) 2024-05-07 10:35:24 +01:00
dependabot[bot]
1726b49457
Bump types-pillow from 10.2.0.20240415 to 10.2.0.20240423 (#17159) 2024-05-07 10:34:56 +01:00
dependabot[bot]
792cfe7ba6
Bump lxml from 5.1.0 to 5.2.1 (#17158) 2024-05-07 10:34:46 +01:00
dependabot[bot]
c3682ff668
Bump jsonschema from 4.21.1 to 4.22.0 (#17157)
Bumps [jsonschema](https://github.com/python-jsonschema/jsonschema) from
4.21.1 to 4.22.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/python-jsonschema/jsonschema/releases">jsonschema's
releases</a>.</em></p>
<blockquote>
<h2>v4.22.0</h2>
<!-- raw HTML omitted -->
<h2>What's Changed</h2>
<ul>
<li>Improve <code>best_match</code> (and thereby error messages from
<code>jsonschema.validate</code>) in cases where there are multiple
<em>sibling</em> errors from applying <code>anyOf</code> /
<code>allOf</code> -- i.e. when multiple elements of a JSON array have
errors, we now do prefer showing errors from earlier elements rather
than simply showing an error for the full array (<a
href="https://redirect.github.com/python-jsonschema/jsonschema/issues/1250">#1250</a>).</li>
<li>(Micro-)optimize equality checks when comparing for JSON Schema
equality by first checking for object identity, as <code>==</code>
would.</li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/shinnar"><code>@​shinnar</code></a> made
their first contribution in <a
href="https://redirect.github.com/python-jsonschema/jsonschema/pull/1224">python-jsonschema/jsonschema#1224</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/python-jsonschema/jsonschema/compare/v4.21.1...v4.22.0">https://github.com/python-jsonschema/jsonschema/compare/v4.21.1...v4.22.0</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/python-jsonschema/jsonschema/blob/main/CHANGELOG.rst">jsonschema's
changelog</a>.</em></p>
<blockquote>
<h1>v4.22.0</h1>
<ul>
<li>Improve <code>best_match</code> (and thereby error messages from
<code>jsonschema.validate</code>) in cases where there are multiple
<em>sibling</em> errors from applying <code>anyOf</code> /
<code>allOf</code> -- i.e. when multiple elements of a JSON array have
errors, we now do prefer showing errors from earlier elements rather
than simply showing an error for the full array (<a
href="https://redirect.github.com/python-jsonschema/jsonschema/issues/1250">#1250</a>).</li>
<li>(Micro-)optimize equality checks when comparing for JSON Schema
equality by first checking for object identity, as <code>==</code>
would.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9882dbeb1a"><code>9882dbe</code></a>
Add / ignore the new specification test suite property.</li>
<li><a
href="ebc90bb2df"><code>ebc90bb</code></a>
Merge commit '8fcfc3a674a7188a4fcc822b7a91efb3e0422a20'</li>
<li><a
href="8fcfc3a674"><code>8fcfc3a</code></a>
Squashed 'json/' changes from b41167c74..54f3784a8</li>
<li><a
href="30b7537944"><code>30b7537</code></a>
Pin pyenchant to pre from below until <a
href="https://redirect.github.com/pyenchant/pyenchant/issues/302">pyenchant/pyenchant#302</a>
is released.</li>
<li><a
href="c3729db732"><code>c3729db</code></a>
Enable doctests for the rest of the referencing page.</li>
<li><a
href="70a994ceab"><code>70a994c</code></a>
Remove a now-unneeded noqa since apparently this is fixed in new
ruff.</li>
<li><a
href="e6d0ef1cff"><code>e6d0ef1</code></a>
Fix a minor typo in the referencing example docs.</li>
<li><a
href="bceaf41a7d"><code>bceaf41</code></a>
Another placeholder benchmark for future optimization.</li>
<li><a
href="b20234e86c"><code>b20234e</code></a>
Consider errors from earlier indices (in instances) to be better
matches</li>
<li><a
href="41b49c68e5"><code>41b49c6</code></a>
Minor improvement to test failure message when a best match test
fails.</li>
<li>Additional commits viewable in <a
href="https://github.com/python-jsonschema/jsonschema/compare/v4.21.1...v4.22.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jsonschema&package-manager=pip&previous-version=4.21.1&new-version=4.22.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-07 10:34:30 +01:00
Erik Johnston
3e6ee8ff88
Add optimisation to StreamChangeCache (#17130)
When there have been lots of changes compared with the number of
entities, we can do a fast(er) path.

Locally I ran some benchmarking, and the comparison seems to give the
best determination of which method we use.
2024-05-06 12:56:52 +01:00
Erik Johnston
7c9ac01eb5
Fix bug where StreamChangeCache would not respect cache factors (#17152)
Annoyingly mypy didn't pick up this typo.
2024-05-03 18:00:08 +01:00
Erik Johnston
3818597751
Fix lint.sh script (#17148)
Broke in #17073
2024-05-03 17:12:03 +01:00
Andrew Morgan
3aadf43122
Bump pillow from 10.2.0 to 10.3.0 (#17146) 2024-05-03 10:55:59 +01:00
jahway603
5b6a75935e
upgrade.md: Bump minimum Rust version to 1.66.0 (element-hq#17079) (#17140)
upgrade.md: Bump minimum Rust version to 1.66.0 (element-hq#17079)
2024-05-02 14:57:29 +01:00
Benjamin Bouvier
c0ea2bf800
synapse complement image: hardcode enabling msc3266 (#17105)
This is an alternative to
https://github.com/matrix-org/matrix-rust-sdk/issues/3340 where we don't
need to change our CI setup too much in the Rust SDK repository, and
still can test MSC3266.
2024-05-02 11:48:27 +01:00
Shay
37558d5e4c
Add support for MSC3823 - Account Suspension (#17051) 2024-05-01 17:45:17 +01:00
Erik Johnston
0b358f8643
Drop sphinx docs (#17073)
It is broken, and we only seemed to have been building it for the
federation sender.

Closes https://github.com/element-hq/synapse/issues/16804
2024-05-01 16:01:50 +00:00
Ben Banfield-Zanin
7254015665
Correct licensing metadata on the Docker image (#17141)
### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-05-01 16:23:42 +01:00
Andrew Morgan
e84a493f41 Merge branch 'master' into develop 2024-04-30 14:42:45 +01:00
Richard van der Hoff
07232e27a8
Enable complement tests for MSC4115 support (#17137)
Follow-up to #17137 and
https://github.com/matrix-org/complement/pull/722
2024-04-30 13:57:20 +01:00
Andrew Morgan
e26673fe97 1.106.0 2024-04-30 11:51:50 +01:00
devonh
7ab0f630da
Apply user email & picture during OIDC registration if present & selected (#17120)
This change will apply the `email` & `picture` provided by OIDC to the
new user account when registering a new user via OIDC. If the user is
directed to the account details form, this change makes sure they have
been selected before applying them, otherwise they are omitted. In
particular, this change ensures the values are carried through when
Synapse has consent configured, and the redirect to the consent form/s
are followed.

I have tested everything manually. Including: 
- with/without consent configured
- allowing/not allowing the use of email/avatar (via
`sso_auth_account_details.html`)
- with/without automatic account detail population (by un/commenting the
`localpart_template` option in synapse config).

### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [X] Pull request is based on the develop branch
* [X] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-04-29 15:23:05 +00:00
Richard van der Hoff
b548f7803a
Add support for MSC4115 (#17104)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-29 15:22:13 +01:00
Andrew Morgan
758aec6b34
Update tornado 6.2 -> 6.4 (#17131) 2024-04-29 14:33:25 +01:00
Richard van der Hoff
c897ac63e9
Ensure that incoming to-device messages are not dropped (#17127)
... when workers are unreachable, etc.

Fixes https://github.com/element-hq/synapse/issues/17117.

The general principle is just to make sure that we propagate any
exceptions to the JsonResource, so that we return an error code to the
sending server. That means that the sending server no longer considers
the message safely sent, so it will retry later.

In the issue, Erik mentions that an alternative solution would be to
persist the to-device messages into a table so that they can be retried.
This might be an improvement for performance, but even if we did that,
we still need this mechanism, since we might be unable to reach the
database. So, if we want to do that, it can be a later follow-up.

---------

Co-authored-by: Erik Johnston <erik@matrix.org>
2024-04-29 14:11:00 +01:00