mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-11-24 10:35:46 +03:00
Code Issues Projects Releases Packages Wiki Activity
23834 commits 935 branches 728 tags 489 MiB
Commit graph

290 commits

Author SHA1 Message Date
Dirk Klimpel
8d5325ec0c
Drop unused table presence (#14825) 2023-01-13 14:17:03 +00:00
Andrew Morgan
f5ea9f2b1d
Add rust linting commands to scripts-dev/lint.sh (#14822) 2023-01-12 16:20:34 +00:00
Andrew Morgan
c4456114e1
Add experimental support for MSC3391: deleting account data (#14714) 2023-01-01 03:40:46 +00:00
reivilibre
46993770e5
Suppress the update check in the ruff linter. (#14741) 
* Suppress update check in ruff

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
 2022-12-28 17:23:19 +00:00
Patrick Cloke
7010a3d015
Switch to ruff instead of flake8. (#14633) 
ruff is a flake8-compatible Python linter written in Rust.
It supports the flake8 plugins that we use and is significantly
faster in testing.
 2022-12-21 13:05:21 -05:00
David Robertson
3d87847ecc
Enable --warn-redundant-casts option in mypy (#14671) 
* Enable `--warn-redundant-casts` option in mypy

Doesn't do much but helps me sleep better at night.

* Changelog

* Fix name of the ignore

* Fix one more missed cast

Not sure why I didn't see this one locally, maybe I needed a poetry update

* Remove old comment

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2022-12-12 21:25:07 +00:00
reivilibre
be3a8a85e3
Add --editable flag to complement.sh which uses an editable install of Synapse for faster turn-around times whilst developing iteratively. (#14548) 
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
 2022-12-07 15:45:31 +00:00
Eric Eastwood
8f10c8b054
Move MSC3030 /timestamp_to_event endpoint to stable v1 location (#14471) 
Fix https://github.com/matrix-org/synapse/issues/14390

 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
 - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`

Complement test changes: https://github.com/matrix-org/complement/pull/559
 2022-11-28 15:54:18 -06:00
Richard van der Hoff
8d133a8464
Fixes to federation_client dev script (#14479) 
* Attempt to fix federation-client devscript handling of .well-known

The script was setting the wrong value in the Host header

* Fix TLS verification

Turns out that actually doing TLS verification isn't that hard. Let's enable
it.
 2022-11-20 17:41:17 +00:00
David Robertson
2cc592584a
Remove unused type-ignores (#14433) 
* Remove unused type-ignores

Oversights in #14427 and #14429.

* Changelog
 2022-11-14 13:46:29 +00:00
Sean Quah
0cf48f2d5f
Build Debian packages for Ubuntu 22.10 Kinetic Kudu (#14396) 
Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-11-09 10:33:13 +00:00
realtyem
69814eb282
Allow override for requesting specific worker types for Complement on command line. (#14324) 
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.

* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.

* Update docs to reflect this new ability.

* Changelog

* Don't rely on soft wrapping to format long strings

Good idea dklimpel. Thanks for catching that.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>

* Small nits just noticed in docs.

* Fixup new line in docs.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
 2022-11-08 12:34:09 +00:00
Patrick Cloke
59ca73006c
Enable testing MSC3874 in complement. (#14339) 2022-11-01 13:26:28 -04:00
Patrick Cloke
c604d2c218
Mark /relations endpoint as usable on workers. (#14028) 
Co-authored-by: Eric Eastwood <erice@element.io>
 2022-10-12 06:46:13 -04:00
David Robertson
285d72556b
Update mypy and mypy-zope, attempt 3 (#13993) 
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2022-09-30 17:36:28 +01:00
David Robertson
8e52cb0bce
Revert "Update mypy and mypy-zope (#13925)" 
This reverts commit 6d543d6d9f.
 2022-09-30 16:37:48 +01:00
David Robertson
6d543d6d9f
Update mypy and mypy-zope (#13925) 
* Update mypy and mypy-zope

* Unignore assigning to LogRecord attributes

Presumably https://github.com/python/typeshed/pull/8064 makes this ok

Cherry-picked from #13521

* Remove unused ignores due to mypy ParamSpec fixes

https://github.com/python/mypy/pull/12668

Cherry-picked from #13521

* Remove additional unused ignores

* Fix new mypy complaints related to `assertGreater`

Presumably due to https://github.com/python/typeshed/pull/8077

* Changelog

* Reword changelog

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2022-09-30 16:34:47 +01:00
David Robertson
0a38c7ec6d
Snapshot schema 72 (#13873) 
Including another batch of fixes to the schema dump script
 2022-09-26 18:28:32 +01:00
Mathieu Velten
6bd8763804
Add cache invalidation across workers to module API (#13667) 
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-09-21 15:32:01 +02:00
reivilibre
a35842caec
Fix the release script not publishing binary wheels. (#13850) 2022-09-21 09:43:08 +00:00
David Robertson
fff9b955fa
Generate separate snapshots for logical databases (#13792) 
* Generate separate snapshots for sqlite, postgres and common
* Cleanup postgres dbs in the TRAP
* Say which logical DB we're applying updates to
* Run background updates on the state DB
* Add new option for accepting a SCHEMA_NUMBER
 2022-09-20 14:14:12 +01:00
David Robertson
b60d47ab2c
Updates to the schema dump script (#13770) 2022-09-13 10:53:11 +01:00
reivilibre
c7b18d9d44
Extend the release script to wait for GitHub Actions to finish and to be usable as a guide for the whole process. (#13483) 2022-09-05 12:16:59 +01:00
David Robertson
ba8938b090
Reject non-strict types in Pydantic models (#13502) 2022-08-17 11:17:04 +01:00
reivilibre
1c910e2216
Add a merge-back command to the release script, which automates merging the correct branches after a release. (#13393) 2022-08-02 15:56:28 +00:00
reivilibre
05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. (#13213) 
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2022-08-01 10:55:31 +00:00
reivilibre
543dc9c93e
Extend the release script to automatically push a new SyTest branch, rather than having that be a manual process. (#12978) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2022-07-26 18:08:14 +01:00
Patrick Cloke
1efe6b8c41
Stop building Ubuntu 21.10 (Impish Indri) which is end of life. (#13326) 2022-07-19 09:08:46 -04:00
reivilibre
0c95313a44
Add --build-only option to complement.sh to prevent actually running Complement. (#13158) 2022-07-07 14:18:38 +00:00
reivilibre
65e675504f
Add the ability to set the log level using the SYNAPSE_TEST_LOG_LEVEL environment when using complement.sh. (#13152) 2022-07-05 09:46:20 +00:00
reivilibre
c04e25789e
Enable Complement testing in the 'Twisted Trunk' CI runs. (#13079) 2022-07-01 15:42:49 +00:00
Richard van der Hoff
fe910fb10e
complement.sh: Permit skipping docker build (#13143) 
Add a `-f` argument to `complement.sh` to skip the docker build
 2022-07-01 12:33:59 +00:00
Erik Johnston
de334ac183
Add a CI job to check that schema deltas are in the correct folder. (#13063) 2022-06-15 16:27:18 +01:00
reivilibre
538044ac01
Collapse Docker build commands in Complement CI runs to make the logs easier to read. (#13058) 2022-06-15 14:42:27 +00:00
Erik Johnston
aef398457f
Up complement time outs (#13048) 2022-06-14 17:59:06 +01:00
Patrick Cloke
1df22e16df
Fix example of running complement.sh. (#12990) 2022-06-09 07:21:15 -04:00
reivilibre
67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. (#12881) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2022-06-08 09:57:05 +00:00
David Robertson
81d9f2a8e9
Fixes to MSC3787 implementation (#12858) 2022-05-24 16:50:50 +00:00
reivilibre
7a68203cde
Disable 'faster room join' Complement tests when testing against Synapse with workers. (#12842) 2022-05-23 17:27:05 +01:00
David Robertson
5331fb5b47
allow on_invalidate=None in @cached methods (#12769) 2022-05-17 16:06:45 +00:00
Richard van der Hoff
34e84fee68
Tweaks to workers-under-complement (#12637) 
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
 2022-05-09 22:41:06 +01:00
Andrew Morgan
c4514b97db
Add missing space before 'docker' link in release announcement script (#12612) 2022-05-03 14:46:42 +00:00
David Robertson
bf2fea8f7d
Add sanity checks to the release script (#12556) 
Check we're on the right branch before tagging, and on the right tag before uploading

* Abort if we're on the wrong branch
* Check we have the right tag checked out
* Clarify that `publish` only releases to GitHub
 2022-05-03 10:50:03 +00:00
David Robertson
30c8e7e408
Make scripts-dev pass mypy --disallow-untyped-defs (#12356) 
Not enforced in config yet. One day.
 2022-04-27 13:10:31 +00:00
David Robertson
6b9e95015b
Lint the release script 2022-04-26 11:53:37 +01:00
David Robertson
730fcda546
Update release script to be poetry-aware 
Poetry now manages the project version in pyproject.toml.
 2022-04-26 11:14:27 +01:00
Shay
99ab45423a
build debian package for jammy jellyfish (#12543) 2022-04-26 10:34:59 +01:00
Richard van der Hoff
f5668f0b4a
Await un-partial-stating after a partial-state join (#12399) 
When we join a room via the faster-joins mechanism, we end up with "partial
state" at some points on the event DAG. Many parts of the codebase need to
wait for the full state to load. So, we implement a mechanism to keep track of
which events have partial state, and wait for them to be fully-populated.
 2022-04-21 07:42:03 +01:00
David Robertson
09b4f6e46d
Remove leftover references to setup.py (#12514) 
* Remove leftover references to setup.py

Missed in #12478.

* Changelog
 2022-04-20 18:16:49 +00:00
Richard van der Hoff
798deb3a10
Fix typo in deb changelogs from release script (#12497) 
The release script used to incorrectly write `New synapse release 1.57.0~rc1.`
instead of `New synapse release 1.57.0rc1.`
 2022-04-19 16:41:52 +01:00
Jan Christian Grünhage
a1f87f57ff
Implement MSC3383: include destination in X-Matrix auth header (#11398) 
Co-authored-by: Jan Christian Grünhage <jan.christian@gruenhage.xyz>
Co-authored-by: Marcus Hoffmann <bubu@bubu1.eu>
 2022-04-19 16:23:53 +01:00
David Robertson
535a689cfc
Reintroduce the lint targets in the linter script (#12455) 2022-04-14 11:33:06 +01:00
Erik Johnston
ba1588461b
Fix typos in release script docs (#12450) 2022-04-12 15:12:57 +00:00
Richard van der Hoff
3cdf5a1386
Fix up healthcheck generation for workers docker image (#12405) 
This wasn't quite generating the right thing.
 2022-04-11 13:38:58 +00:00
Richard van der Hoff
5f72ea1bde
Move complement setup stuff into the Synapse repo (#12404) 
Fixes matrix-org/complement#330 (or it will, once we remove the old files).

It's not quite a lift-and-shift: I've also taken the opportunity to get rid of the custom CA that we used to use to sign the TLS certs, which has been superceded by the CA exposed by Complement.
 2022-04-11 11:39:28 +01:00
David Robertson
dd5cc37aa4
Stop maintaining a list of lint targets in lint.sh (#12420) 2022-04-08 14:36:09 +01:00
David Robertson
80839a44f1
Remove more dead/broken dev scripts (#12355) 2022-04-03 21:21:08 +01:00
David Robertson
336bff1104
Burn check_signature dev script. (#12351) 2022-04-01 13:41:42 +00:00
David Robertson
21351820e0
Remove list_url_patterns dev script (#12349) 2022-04-01 13:05:21 +00:00
David Robertson
15cdcf8f30
Remove dockerfile-pgtests (#12336) 2022-03-31 11:30:07 +01:00
David Robertson
a7fb66e800
Update dead links in check-newsfragment.sh (#12331) 2022-03-30 11:44:28 +00:00
Andrew Morgan
19a1d6a42a
Remove references to "msc2403" (#12165) 2022-03-30 11:15:47 +00:00
David Robertson
12d1f82db2
Generate announcement links in release script (#12242) 2022-03-17 13:46:05 +00:00
Richard van der Hoff
00a67f831a Merge remote-tracking branch 'origin/release-v1.54' into develop 2022-03-04 22:40:51 +00:00
David Robertson
4aeb00ca20
Move synctl into synapse._scripts and expose as an entrypoint (#12140) 2022-03-04 11:58:49 +00:00
Eric Eastwood
a511a890d7
Enable MSC2716 Complement tests in Synapse (#12145) 
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2022-03-03 11:19:20 +00:00
Eric Eastwood
31b125ccec
Enable MSC3030 Complement tests in Synapse (#12144) 
The Complement tests for MSC3030 are now merged, https://github.com/matrix-org/complement/pull/178

Synapse implmentation: https://github.com/matrix-org/synapse/pull/9445
 2022-03-03 11:45:23 +01:00
David Robertson
1fbe0316a9
Add suffices to scripts in scripts-dev (#12137) 
* Rename scripts-dev to have suffices

* Update references to `scripts-dev`

* Changelog

* These scripts don't pass mypy
 2022-03-02 18:00:26 +00:00
Erik Johnston
6d282a9c89
Make release script write correct no-op changelog (#12127) 
As we want to include the previous version in the "No new changes..."
string.
 2022-03-02 14:28:18 +00:00
David Robertson
f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points (#12118) 
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2022-03-02 13:00:16 +00:00
David Robertson
5f62a094de
Detox, part 1 of N (#12119) 
* Don't use `tox` for `check-sampleconfig`
* Don't use `tox` for check-newsfragment
 2022-03-01 19:47:02 +00:00
Nicolas Werner
c1ac2a8135
Rename default branch of complement.sh to main (#12063) 
The complement.sh script relies on the name of the ref matching the name
of the unpacked folder. The branch redirect from renaming the default
branch breaks that assumption.

Signed-off-by: Nicolas Werner <n.werner@famedly.com>
 2022-02-23 10:06:18 +00:00
Shay
1e12efa1b2
Drop support for EOL Ubuntu 21.04 (#11961) 2022-02-10 13:59:42 -08:00
Nicolas Werner
fa583c2198
Allow overriding the complement ref. (#11766) 
Updates complement.sh to read the ref from an environment
variable (defaulting to master) when downloading a complement
bundle for testing.
 2022-01-20 13:04:58 +00:00
Richard van der Hoff
d41c4654db
Use buildkit's cache feature to speed up docker builds (#11691) 
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
 2022-01-12 10:37:57 +00:00
Michael Kaye
338e70c617
Complement environment variable name change and update .gitignore. (#11718) 2022-01-10 23:18:56 +00:00
Richard van der Hoff
b38bdae3a2
Fix AssertionErrors after purging events (#11642) 
* Fix AssertionErrors after purging events

If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.

This fixes the situation by rewinding the stream processors.

* `check-newsfragment`: ignore deleted newsfiles
 2022-01-04 16:36:33 +00:00
Shay
13c974ed35
Drop Bionic from Debian builds (#11633) 
* update Trove classifiers to remove py36

* stop building bionic

* update dh-virtualenv

* newsfragment

* fix newsfragment

* update version refs

* another try at correct tag

* Update changelog
 2022-01-03 11:17:16 -08:00
Richard van der Hoff
f61462e1be
scripts-dev/sign_json: support for signing events (#11486) 2021-12-02 15:18:40 +00:00
Patrick Cloke
a4521ce0a8
Support the stable /hierarchy endpoint from MSC2946 (#11329) 
This also makes additional updates where the implementation
had drifted from the approved MSC.

Unstable endpoints will be removed at a later data.
 2021-11-29 14:32:20 -05:00
Patrick Cloke
9d1971a5c4
Return the stable event field from /send_join per MSC3083. (#11413) 
This does not remove the unstable field and still parses both.
Handling of the unstable field will need to be removed in the
future.
 2021-11-29 15:43:20 +00:00
Patrick Cloke
5505da2109
Remove msc2716 from the list of tests for complement. (#11389) 
As the tests are currently failing and not run in CI.
 2021-11-19 07:06:16 -05:00
Patrick Cloke
60ecb6b4d4
Fix running complement.sh script. (#11368) 
By reverting changes from #11166 in this script. Specifically commit
13f084eb58.
 2021-11-17 09:04:50 -05:00
Dan Callahan
d7141e0b8b
Fix Shellcheck SC2006: Use $(...) notation 
Use $(...) notation instead of legacy backticked `...`.

https://github.com/koalaman/shellcheck/wiki/SC2006

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:55 +01:00
Dan Callahan
13f084eb58
Fix Shellcheck SC2086: Quote to prevent splitting 
Double quote to prevent globbing and word splitting.

https://github.com/koalaman/shellcheck/wiki/SC2086

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:54 +01:00
Dan Callahan
6a9d84a676
Fix Shellcheck SC2166: test -a is not well defined 
Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.

https://github.com/koalaman/shellcheck/wiki/SC2166

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:54 +01:00
Dan Callahan
64adbb7b54
Fix Shellcheck SC2046: Quote to prevent word split 
Quote this to prevent word splitting

https://www.shellcheck.net/wiki/SC2046

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:53 +01:00
Dan Callahan
12d79ff1b6
Fix Shellcheck SC2164: exit in case cd fails. 
Use `cd ... || exit` in case cd fails.

https://github.com/koalaman/shellcheck/wiki/SC2164

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:53 +01:00
Richard van der Hoff
57501d9194
Update sign_json to support inline key config (#11139) 
It's been possible to configure a key inline in the homeserver.yaml since
13bc1e0746. Update `sign_json` to work with this.
 2021-10-20 18:49:20 +01:00
Hillery Shay
5dcacdf6d1
Add support for ubuntu 21.10 "Impish Indri" (#11024) 
* support ubuntu 21.10 indri

* add changelog

* update to correct codename

Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>

Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
 2021-10-12 11:29:39 -07:00
reivilibre
b742cb2e4a
Release script improvements (#10966) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-10-11 14:48:38 +01:00
Nick Barrett
c80878d22a
Add --run-background-updates option to update_database script. (#10954) 
Signed-off-by: Nick Barrett <nick@beeper.com>
 2021-10-06 11:26:18 +01:00
Erik Johnston
d37841787a
Sign the git tag in release script (#10925) 2021-09-27 15:39:49 +01:00
David Robertson
ea01d4c2de
Update postgresql testing script (#10906) 
- Use sytest:bionic. Sytest:latest is two years old (do we want
  CI to push out latest at all?) and comes with Python 3.5, which we
  explictly no longer support. The script now runs under PostgreSQL 10
  as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root

A few changes to make it speedier to re-run the tests:

- Create blank DB in the container, not the script, so we don't have to
  `initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
  fetch and install a bunch of packages from PyPI each time

Co-authored-by: reivilibre <olivier@librepush.net>
 2021-09-24 14:27:09 +00:00
Brendan Abolivier
5154afc00d
Add a script to update the debian changelog for non-Debian systems (#10778) 
When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.
 2021-09-08 17:03:25 +02:00
Patrick Cloke
882539e423
Ensure the base Docker image is rebuilt when running complement with workers. (#10686) 
We now always rebuild the matrixdotorg/synapse image, then
build the matrixdotorg/synapse-workers image on top of it.
 2021-08-25 10:18:23 -04:00
Richard van der Hoff
c8132f4a31
Build debs for bookworm (#10612) 2021-08-17 13:48:59 +01:00
David Robertson
d2ad397d3c
Stop building a debian package for Groovy Gorilla (#10588) 2021-08-12 16:50:18 +01:00
David Robertson
314a739160 Also rename in lint.sh 2021-08-12 10:40:44 +01:00
David Robertson
c0ebdfc77e Kill off the .buildkite dir completely 2021-08-10 16:38:13 +01:00