mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-11-26 03:25:53 +03:00
Code Issues Projects Releases Packages Wiki Activity
20238 commits 936 branches 728 tags 490 MiB
Commit graph

1199 commits

Author SHA1 Message Date
Andy Balaam
5ff5f17377
Mention python3-venv and libpq-dev dependencies in contribution guide (#11740) 2022-01-13 16:33:37 +00:00
Jason Robinson
2560b1b6b2
Allow tracking puppeted users for MAU (#11561) 
Currently when puppeting another user, the user doing the puppeting is
tracked for client IPs and MAU (if configured).

When tracking MAU is important, it becomes necessary to be possible to
also track the client IPs and MAU of puppeted users. As an example a
client that manages user creation and creation of tokens via the Synapse
admin API, passing those tokens for the client to use.

This PR adds optional configuration to enable tracking of puppeted users
into monthly active users. The default behaviour stays the same.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2022-01-12 16:09:36 +00:00
haslersn
99ba5ae7b7
Fix documentation of supported PostgreSQL version (#11725) 
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
 2022-01-12 14:43:48 +00:00
Andrew Morgan
7c3408d1a8
Document the SYNAPSE_TEST_PERSIST_SQLITE_DB unit test env var (#11715) 2022-01-10 17:06:42 +00:00
Olivier Wilkinson (reivilibre)
ffd71029ab Add support removal notice to the upgrade notes 2022-01-05 14:14:16 +00:00
Philipp Matthias Schäfer
eedb4527f1
Fix link from generated configuration file to documentation (#11678) 
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: reivilibre <oliverw@matrix.org>
 2022-01-05 12:16:52 +00:00
Dirk Klimpel
7a1cefc6e3
Add admin API to get users' account data (#11664) 
Co-authored-by: reivilibre <olivier@librepush.net>
 2022-01-05 11:49:06 +00:00
reivilibre
84d790a32e
Clarify SSO mapping provider documentation by writing def or async def before the names of methods, as appropriate. (#11681) 2022-01-05 11:25:32 +00:00
Donny Johnson
0715e77b06
Correct Synapse install command for FreeBSD. (#11267) 
Co-authored-by: reivilibre <olivier@librepush.net>
 2022-01-05 10:38:51 +00:00
Richard van der Hoff
79f6d3550a
update ngnix reverse-proxy example (#11680) 
this should not be a case-insensitive match.
 2022-01-04 22:31:45 +00:00
Patrick Cloke
8422a7f7f6
Include the topic event in the prejoin state, per MSC3173. (#11666) 
Invites and knocks will now include the topic in the stripped state
send to clients before joining the room.
 2022-01-04 11:08:08 -05:00
AndrewFerr
c500bf37d6
Add details for how to set up TURN behind NAT (#11553) 
Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
 2021-12-23 10:42:50 +00:00
Richard van der Hoff
144b9ede89 Merge branch 'master' into develop 2021-12-21 18:13:05 +00:00
Richard van der Hoff
57ca8ab10f Add notes about dropping support for Python 3.6 and Postgres 9.6. 2021-12-21 12:06:31 +00:00
reivilibre
83a74d9350
Document the usage of refresh tokens. (#11427) 
Co-authored-by: David Robertson <davidr@element.io>
 2021-12-08 15:31:17 +00:00
reivilibre
2f053f3f82
Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435) 2021-12-06 19:11:43 +00:00
Dirk Klimpel
8b4b153c9e
Add admin API to get some information about federation status (#11407) 2021-12-06 16:59:50 +00:00
Eric Eastwood
4eb77965cd
Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' prev_events (#11469) 
Spawning from https://github.com/matrix-org/synapse/pull/9445#discussion_r758958181

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-12-03 18:25:04 -06:00
Shay
49e1356ee3
Minor cleanup on recently ported doc pages (#11466) 
* move wiki pages to synapse/docs and add a few titles where necessary

* update SUMMARY.md with added pages

* add changelog

* move incorrectly located newsfragment

* update changelog number

* snake case added files and update summary.md accordingly

* update issue/pr links

* update relative links to docs

* update changelog to indicate that we moved wiki pages to the docs and state reasoning

* requested changes to admin_faq.md

* requested changes to database_maintenance_tools.md

* requested changes to understanding_synapse_through_graphana_graphs.md

* add changelog

* fix leftover merge errata

* fix unwanted changes from merge

* use two spaces between entries

* outdent code blocks
 2021-12-02 09:46:20 -08:00
Shay
84dc50e160
Port wiki pages to documentation website (#11402) 
* move wiki pages to synapse/docs and add a few titles where necessary

* update SUMMARY.md with added pages

* add changelog

* move incorrectly located newsfragment

* update changelog number

* snake case added files and update summary.md accordingly

* update issue/pr links

* update relative links to docs

* update changelog to indicate that we moved wiki pages to the docs and state reasoning

* revert unintentional change to CHANGES.md

* add link

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update CHANGES.md

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-12-01 10:12:19 -08:00
Brendan Abolivier
153194c771
Link background update controller docs to summary (#11475) 2021-12-01 14:13:01 +00:00
Richard van der Hoff
b9fef1a7cd
Update openid.md 
fix header level
 2021-12-01 10:57:55 +00:00
Etienne Dysli Metref
b0eb64ff7b
Remove mention of OIDC certification from Dex (#11470) 
Dex isn't yet [1,2] a certified OpenID Provider implementation. As of today, it's not on the list maintained by the OpenID Foundation. [3]

[1] https://github.com/dexidp/dex/issues/42
[2] https://github.com/dexidp/dex/issues/262
[3] https://openid.net/certification/
 2021-12-01 09:40:51 +00:00
Shay
f1795463bf
Add a note about huge pages to our Postgres doc (#11467) 
* Add note to postgres doc about hugepages

* Newsfragment
 2021-11-30 19:05:20 -08:00
Marcus
a9481223d1
Improved push typing (#11409) 
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2021-11-30 11:49:20 +00:00
Patrick Cloke
a4521ce0a8
Support the stable /hierarchy endpoint from MSC2946 (#11329) 
This also makes additional updates where the implementation
had drifted from the approved MSC.

Unstable endpoints will be removed at a later data.
 2021-11-29 14:32:20 -05:00
Erik Johnston
d08ef6f155
Make background updates controllable via a plugin (#11306) 
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-11-29 17:57:06 +01:00
Sean Quah
7564b8e118
Update the media repository documentation (#11415) 2021-11-29 15:37:56 +00:00
Kostas
1035663833
Add config for customizing the claim used for JWT logins. (#11361) 
Allows specifying a different claim (from the default "sub") to use
when calculating the localpart of the Matrix ID used during the
JWT login.
 2021-11-22 13:01:03 -05:00
Shay
d9e9771d6b
Update README.md 2021-11-19 14:01:55 -08:00
Dirk Klimpel
ea20937084
Add an admin API to run background jobs. (#11352) 
Instead of having admins poke into the database directly.

Can currently run jobs to populate stats and to populate
the user directory.
 2021-11-19 19:39:46 +00:00
Dirk Klimpel
81b18fe5c0
Add dedicated admin API for blocking a room (#11324) 2021-11-18 17:43:49 +00:00
Nicolai Søborg
e2dabec996
Docs: Quote wildcard federation_certificate_verification_whitelist (#11381) 
Otherwise I get this beautiful stacktrace:

```
python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/synapse/synapse/app/homeserver.py", line 455, in <module>
    main()
  File "/root/synapse/synapse/app/homeserver.py", line 445, in main
    hs = setup(sys.argv[1:])
  File "/root/synapse/synapse/app/homeserver.py", line 345, in setup
    config = HomeServerConfig.load_or_generate_config(
  File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config
    config_dict = read_config_files(config_files)
  File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files
    yaml_config = yaml.safe_load(file_stream)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node
    while not self.check_event(SequenceEndEvent):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry
    return self.parse_block_sequence_entry()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry
    if not self.check_token(BlockEntryToken, BlockEndToken):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens
    return self.fetch_alias()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias
    self.tokens.append(self.scan_anchor(AliasToken))
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor
    raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
  in "/etc/matrix/homeserver.yaml", line 614, column 5
expected alphabetic or numeric character, but found '.'
  in "/etc/matrix/homeserver.yaml", line 614, column 6
```

Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
 2021-11-18 12:24:40 +00:00
Aaron R
d993c3bb1e
Add support for /_matrix/media/v3 APIs (#11371) 
* Add support for `/_matrix/media/v3` APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Update `workers.md` to use v3 client and media APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2021-11-17 15:30:24 +00:00
David Robertson
b76337fdf8
Merge branch 'master' into develop 2021-11-17 14:19:56 +00:00
David Robertson
077b74929f
Merge remote-tracking branch 'origin/release-v1.47' 2021-11-17 14:19:27 +00:00
Patrick Cloke
24b61f379a
Add ability to un-shadow-ban via the admin API. (#11347) 2021-11-16 12:43:53 +00:00
Ashwin Nair
e72135b9d3
change 'Home Server' to one word 'homeserver' (#11320) 
Signed-off-by: Ashwin S. Nair <58840757+Ashwin-exe@users.noreply.github.com>
 2021-11-16 10:21:01 +00:00
reivilibre
2fffcb24d8
Suggest using /etc/matrix-synapse/conf.d/ for configuration with Debian packages (#11281) 2021-11-12 13:17:06 +00:00
reivilibre
4ad5ee9996
Correct target of link to the modules page from the Password Auth Providers page (#11309) 2021-11-12 12:58:39 +00:00
Dirk Klimpel
8840a7b7f1
Convert delete room admin API to async endpoint (#11223) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-11-12 12:35:31 +00:00
Andrew Morgan
5f277ffe89
Add documentation page stubs for Single Sign-On, SAML and CAS pages (#11298) 2021-11-10 17:54:56 +00:00
Stanislav Motylkov
b09d90cac9
Fix typos in the username_available admin API documentation. (#11286) 2021-11-09 21:11:05 +00:00
David Robertson
b6f4d122ef
Allow admins to proactively block rooms (#11228) 
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-11-09 13:11:47 +00:00
Erik Johnston
4ee71b9637
Add some background update admin APIs (#11263) 
Fixes #11259
 2021-11-08 16:08:02 +00:00
Richard van der Hoff
86a497efaa
Default value for public_baseurl (#11210) 
We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.
 2021-11-08 14:13:10 +00:00
jmcparland
02742fd058
Wrong DTLS port in "Troubleshooting" (#11268) 
Port 5349, not 5479.
 2021-11-08 10:34:39 +00:00
Dan Callahan
9799c569bb
Minor cleanup to Debian packaging (#11269) 
* Remove unused Vagrant scripts

* Change package Architecture to any

* Preinstall the wheel package when building venvs.

Addresses the following warnings during Debian builds:

    Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.

* Allow /etc/default/matrix-synapse to be missing

Per the systemd.exec manpage, prefixing an EnvironmentFile with "-":

> indicates that if the file does not exist, it will not be read and no
> error or warning message is logged.

Signed-off-by: Dan Callahan <danc@element.io>
 2021-11-07 21:18:33 +00:00
Julian
09cb441a04
Add doc to integrate synapse with LemonLDAP OIDC (#11257) 
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Julian Vanden Broeck <julian.vandenbroeck@dalibo.com>
 2021-11-05 12:08:02 +00:00
reivilibre
499c44d696
Make minor correction to type of auth_checkers callbacks (#11253) 2021-11-04 17:10:11 +00:00
Andrew Morgan
d688a6dee5 fix a small typo in the delete room api docs 2021-11-03 11:09:00 +00:00
Dirk Klimpel
7537201840
Add search by room ID and room alias to List Room admin API (#11099) 
Fixes: #10874
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-11-02 10:01:13 +00:00
reivilibre
69ab3dddbc
Make check_event_allowed module API callback not fail open (accept events) when an exception is raised (#11033) 2021-11-01 15:45:56 +00:00
Dirk Klimpel
66bdca3e31
Remove deprecated delete room admin API (#11213) 
Remove deprecated delete room admin API,
`POST /_synapse/admin/v1/rooms/<room_id>/delete`
 2021-11-01 15:11:24 +00:00
Richard van der Hoff
71f9966f27
Support for serving server well-known files (#11211) 
Fixes https://github.com/matrix-org/synapse/issues/8308
 2021-11-01 15:10:16 +00:00
Brett Bethke
2014098d01
Add domain specific matching for haproxy config (#11128) 2021-11-01 14:16:02 +00:00
Sumner Evans
ece84f2c45
Improve code formatting and fix a few typos in docs (#11221) 
* Labeled a lot more code blocks with the appropriate type
* Fixed a couple of minor typos (missing/extraneous commas)

Signed-off-by: Sumner Evans <me@sumnerevans.com>
 2021-11-01 11:35:55 +00:00
Brendan Abolivier
e320f5dba3
Deprecate user_may_create_room_with_invites (#11206) 2021-11-01 10:46:08 +01:00
Sean Quah
3ed17ff651
Clarify lack of Windows support in documentation (#11198) 2021-10-29 14:03:58 +01:00
Skyler Mäntysaari
a19bf32a03
docs/openid: Add Authentik documentation. (#11151) 2021-10-28 10:31:22 +00:00
Brendan Abolivier
c7a5e49664
Implement an on_new_event callback (#11126) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-10-26 15:17:36 +02:00
Jason Robinson
63cbdd8af0
Enable changing user type via users admin API (#11174) 
Users admin API can now also modify user
type in addition to allowing it to be
set on user creation.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-10-26 09:01:06 +00:00
Dirk Klimpel
b0f03aeb6a
Add more information what happens when a user is deactivated (#11083) 2021-10-22 09:00:28 +01:00
Aaron R
2c61a318cc
Show error when timestamp in seconds is provided to the /purge_media_cache API (#11101) 2021-10-20 14:41:48 +00:00
Travis Ralston
106d99b8cd
Remove false warning about copying the log config to a homeserver.yaml (#11092) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-10-20 11:48:15 +00:00
Brendan Abolivier
78d5896d19
Document the version of Synapse each module callback was introduced in (#11132) 
* Mention callbacks introduced in v1.37.0

According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062

* Mention callbacks introduced in v1.39.0

According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884

* Mention callbacks introduced in v1.42.0

According to https://github.com/matrix-org/synapse/pull/10524

* Mention callbacks introduced in v1.44.0 and v1.45.0

As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894

* Mention callbacks introduced in v1.46.0

According to https://github.com/matrix-org/synapse/pull/10548
 2021-10-20 11:04:27 +00:00
Andrew Morgan
5e0e683541
Fix instances of [example]{.title-ref} in the upgrade notes (#11118) 2021-10-19 14:13:56 +01:00
Brendan Abolivier
73743b8ad1
Document Synapse's behaviour when dealing with multiple modules (#11096) 
Document Synapse's behaviour when multiple modules register the same
callback/web resource/etc.

Co-authored-by: reivilibre <oliverw@matrix.org>
 2021-10-18 18:26:52 +02:00
Patrick Cloke
5573133348
Move experimental & retention config out of the server module. (#11070) 2021-10-15 14:30:48 +00:00
Dirk Klimpel
013e0f9cae
Update doc of the allowed characters for registration tokens (#11093) 
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-10-15 09:56:39 +00:00
David Robertson
404444260a
Fix upgrade dead links (#11069) 2021-10-13 14:32:43 +01:00
Azrenbeth
cdd308845b
Port the Password Auth Providers module interface to the new generic interface (#10548) 
Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-10-13 11:21:52 +00:00
Brendan Abolivier
9e13cd98af
Update upgrade notes 2021-10-12 11:23:51 +01:00
reivilibre
5e29d417fc
Include the requirements for [mypy,lint] in [dev] (#11034) 2021-10-11 16:34:31 +01:00
Olivier Wilkinson (reivilibre)
593eeac19e Revert accidental push to develop. 2021-10-08 17:17:22 +01:00
Olivier Wilkinson (reivilibre)
9f23ff78da Update contributing guide to use [all,dev] 2021-10-08 17:11:32 +01:00
Andrew Morgan
e564bdd127
Add content to the Synapse documentation intro page (#10990) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-10-06 17:09:35 +00:00
Brendan Abolivier
4e51621064
Add a spamchecker method to allow or deny 3pid invites (#10894) 
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.

Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.
 2021-10-06 17:18:13 +02:00
Brendan Abolivier
829f2a82b0
Add a spamchecker callback to allow or deny room joins (#10910) 
Co-authored-by: Erik Johnston <erik@matrix.org>
 2021-10-06 14:32:16 +00:00
Max Kratz
38b7db5885
Updated development doc on samling environment for testing. (#10973) 2021-10-06 07:20:41 -04:00
Max Kratz
6744273f0b
Remove "reference" wording according Synapse homeserver (#10971) 2021-10-06 10:05:07 +00:00
Dirk Klimpel
7036a7a60a
Update links to MSCs in documentation (#10991) 
Based on matrix-doc switching from master -> main and
MSCs being merged.
 2021-10-05 11:35:19 +00:00
Richard van der Hoff
e32b9f44ee
Update installation instructions (#10919) 
Various updates to the install docs.
 2021-09-29 10:57:53 +00:00
Sean Quah
f7768f62cb
Avoid storing URL cache files in storage providers (#10911) 
URL cache files are short-lived and it does not make sense to offload
them (eg. to the cloud) or back them up.
 2021-09-27 12:55:27 +01:00
Brendan Abolivier
b10257e879
Add a spamchecker callback to allow or deny room creation based on invites (#10898) 
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.

This adds a callback that's fairly similar to user_may_create_room except it also allows processing based on the invites sent at room creation.
 2021-09-24 16:38:23 +02:00
David Robertson
ea01d4c2de
Update postgresql testing script (#10906) 
- Use sytest:bionic. Sytest:latest is two years old (do we want
  CI to push out latest at all?) and comes with Python 3.5, which we
  explictly no longer support. The script now runs under PostgreSQL 10
  as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root

A few changes to make it speedier to re-run the tests:

- Create blank DB in the container, not the script, so we don't have to
  `initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
  fetch and install a bunch of packages from PyPI each time

Co-authored-by: reivilibre <olivier@librepush.net>
 2021-09-24 14:27:09 +00:00
Patrick Cloke
52913d56a5
Add documentation for experimental feature flags. (#10865) 2021-09-22 13:41:42 +00:00
Patrick Cloke
ba7a91aea5
Refactor oEmbed previews (#10814) 
The major change is moving the decision of whether to use oEmbed
further up the call-stack. This reverts the _download_url method to
being a "dumb" functionwhich takes a single URL and downloads it
(as it was before #7920).

This also makes more minor refactorings:

* Renames internal variables for clarity.
* Factors out shared code between the HTML and rich oEmbed
  previews.
* Fixes tests to preview an oEmbed image.
 2021-09-21 16:09:57 +00:00
David Robertson
60453315bd
Always add local users to the user directory (#10796) 
It's a simplification, but one that'll help make the user directory logic easier
to follow with the other changes upcoming. It's not strictly required for those
changes, but this will help simplify the resulting logic that listens for
`m.room.member` events and generally make the logic easier to follow.

This means the config option `search_all_users` ends up controlling the
search query only, and not the data we store. The cost of doing so is an
extra row in the `user_directory` and `user_directory_search` tables for
each local user which

- belongs to no public rooms
- belongs to no private rooms of size ≥ 2

I think the cost of this will be marginal (since they'll already have entries
 in `users` and `profiles` anyway).

As a small upside, a homeserver whose directory was built with this
change can toggle `search_all_users` without having to rebuild their
directory.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-09-21 12:02:34 +00:00
Charles Wright
6b6bb81b23
Fix #10837 by adding JSON encoding/decoding to the Module API example… (#10845) 2021-09-17 18:04:37 +01:00
BramvdnHeuvel
9f111075e8
Fix copy-paste error in the password section of the sample-config. (#10804) 2021-09-13 08:58:34 -04:00
reivilibre
ceab5a4bfa
Fix 2 typos in docs/log_contexts.md (#10795) 2021-09-10 16:33:36 +01:00
David Robertson
318162f5de
Easy refactors of the user directory (#10789) 
No functional changes here. This came out as I was working to tackle #5677
 2021-09-10 10:54:38 +01:00
Brendan Abolivier
03caba6577
Improve the modules doc (#10758) 
* Split up the documentation in several files rather than one huge one
* Add examples for each callback category
* Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632
* Add titles to callbacks so they're easier to find in the navigation panels and link to
 2021-09-08 17:14:54 +00:00
Patrick Cloke
1fdf2cf8e8
Document that /spaces & /hierarchy can be routed to workers. (#10648) 
Also refactors some of the registration of endpoints on workers.
 2021-09-08 11:02:31 -04:00
Patrick Cloke
89ba834818
Use attrs internally for the URL preview code & add documentation. (#10753) 2021-09-07 13:10:34 +00:00
Azrenbeth
6e895366ea
Add config option to use non-default manhole password and keys (#10643) 2021-09-06 16:08:03 +01:00
David Teller
e1641b46d1
Doc: Clarifying undoing room shutdowns (#10480) 2021-09-06 14:24:31 +01:00
cuttingedge1109
00640ee71a
Fix documentation of directory name for remote thumbnails (#10556) 2021-09-02 14:07:53 +01:00
Olivier Wilkinson (reivilibre)
e059094119 Synapse 1.42.0rc1 (2021-09-01) 
==============================
 
 Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
 
 Features
 --------
 
 - Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
 - Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
 - Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
 - Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))
 
 Bugfixes
 --------
 
 - Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
 - Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
 - Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
 - Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
 - Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
 - Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
 - Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
 - Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
 - Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
 - Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
 - Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))
 
 Improved Documentation
 ----------------------
 
 - Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
 - Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
 - Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
 - Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
 - Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))
 
 Deprecations and Removals
 -------------------------
 
 - Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))
 
 Internal Changes
 ----------------
 
 - Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
 - Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
 - Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
 - Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
 - Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
 - Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
 - Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
 - Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
 - Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
 - Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
 - Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
 - Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
 - Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
 - Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
 - Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
 - Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmEvd3cACgkQLS76LzL7
 4EcF0hAAvOjDeH2hn0eI4YX03/BTCesJ5pCTbEJeC9Hsui3SbWamyYarH0Ae0gM6
 BP9x5/XkCdsZhKT2OJedtZQUQ0cUws5FdJP87NnE1m7KBW9djbTxmkue3JKaHNTT
 ajvEoLe8UjOFbjvyMc+188uWVgtKQ6KLcetwllTpYY3FJ3+yk9CQP1ydYzwfHcHe
 hXW3b20WoRdAmH0B27mj1oQzCeXclbj1kCthcBcxH5RytA8bBc1itBtpAP2FFaeS
 Ml6paoXSs0wZaXXL5T1MhQtcVOOG5Yh7EPsemdyfqkwhIk9QtV3nmGfC7K5iEeq0
 DldTgBNE0wLO+ycdgEZ2oCvitCLT4WDoKkVqKPA91wpuvCMeK5lnDdqbFUSN3hfd
 l8Oz+ydhjsCx7+pBBopnsyGrPIHNaN/vySYGF3lPa3ANKS2J0IVlAaJ59bZqzg2C
 vzdyMBCnn0fhnXMkADPLnIYqtebvC09JlPBRGo5BKmwmlM8Ea1dLQ+Lcy5vXalcw
 +GaYw3U805c03EvsWQxWchjwq6IwMeP06YpK/1ruMjlrCWYJp3V7f/4hXjlTdYm5
 OEWc/kHOnlLfH2cA4aS0sV5WsnEuftotQIzBXuXWw5hSMdFmH7eAviPg0vXgfeA/
 i0FOkOgEYR8O3RMh4Cip8+h44t8SPNjYUFJK9ip3DmNateeUCH0=
 =XxMc
 -----END PGP SIGNATURE-----

Merge tag 'v1.42.0rc1' into develop

Synapse 1.42.0rc1 (2021-09-01)
==============================

Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).

Features
--------

- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))

Bugfixes
--------

- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))

Improved Documentation
----------------------

- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))

Deprecations and Removals
-------------------------

- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))

Internal Changes
----------------

- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 2021-09-01 14:58:14 +01:00