mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-12-21 12:14:29 +03:00
Code Issues Projects Releases Packages Wiki Activity
15604 commits 936 branches 734 tags 467 MiB
Commit graph

2147 commits

Author SHA1 Message Date
Erik Johnston
6b72508d15 Newsfile 2019-10-08 16:18:43 +01:00
Anshul Angaria
474abf1eb6 add M_TOO_LARGE error code for uploading a too large file (#6151) 
Fixes #6109
 2019-10-08 13:55:16 +01:00
Andrew Morgan
ea7d938bca Remove unused public room list timeout param (#6179) 
* Remove unused public room list timeout param

* Add changelog
 2019-10-08 13:51:25 +01:00
Richard van der Hoff
b94a401852 Fix /federation/v1/state for recent room versions (#6170) 
* Fix /federation/v1/state for recent room versions

Turns out this endpoint was completely broken for v3 rooms. Hopefully this
re-signing code is irrelevant nowadays anyway.
 2019-10-08 09:35:37 +01:00
Brendan Abolivier
21b5d8b107
Changelog 2019-10-07 18:00:31 +01:00
Richard van der Hoff
cc2e19ad4b fix changelog 2019-10-07 17:37:55 +01:00
Erik Johnston
2cb7466993
Merge pull request #6161 from matrix-org/erikj/dont_regen_user_id_on_failure 
Don't regenerate numeric user ID if registration fails.
 2019-10-07 16:56:23 +01:00
Michael Kaye
dc795ba709 Log responder we are using. (#6139) 
This prevents us logging "Responding to media request with responder %s".
 2019-10-07 15:41:25 +01:00
Brendan Abolivier
1992f21a9f
Fix changelog for PR #6175 2019-10-07 14:54:36 +01:00
Richard van der Hoff
276ae5c63e
add some logging to the rooms stats updates, to try to track down a flaky test (#6167) 2019-10-07 14:41:39 +01:00
Brendan Abolivier
97e2722723
Merge pull request #6175 from matrix-org/babolivier/fix_unique_user_filter_index 
Fix unique_user_filter_index schema update
 2019-10-07 13:48:43 +01:00
Brendan Abolivier
aa7a003074
Changelog 2019-10-07 13:16:54 +01:00
Erik Johnston
86f4705866
Merge pull request #6159 from matrix-org/erikj/cache_memberships 
Cache room membership lookups in _get_joined_users_from_context
 2019-10-07 13:15:00 +01:00
Brendan Abolivier
ae0b78cb1f
Merge pull request #6147 from matrix-org/babolivier/3pid-invite-revoked 
Don't 500 when trying to exchange a revoked 3PID invite
 2019-10-04 12:09:05 +01:00
Alexander Maznev
13c4345c84 Update user_filters table to have a unique index, and non-null columns (#1172) 2019-10-04 10:34:16 +01:00
Robert Swain
39b40d6d99 media/thumbnailer: Better quality for 1-bit / 8-bit color palette images (#2142) 
Pillow will use nearest neighbour as the resampling algorithm if the
source image is either 1-bit or a color palette using 8 bits. If we
convert to RGB before scaling, we'll probably get a better result.
 2019-10-04 09:34:52 +01:00
Erik Johnston
6511071837 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cache_memberships 2019-10-03 17:51:06 +01:00
Richard van der Hoff
66537e10ce
add some metrics on the federation sender (#6160) 2019-10-03 17:47:20 +01:00
Erik Johnston
0186ec9df7 Fixup newsfile 2019-10-03 17:46:43 +01:00
Erik Johnston
4fc60f12de Newsfile 2019-10-03 17:35:50 +01:00
Erik Johnston
0ccf0ffc85 Newsfile 2019-10-03 17:12:24 +01:00
Andrew Morgan
6018bec919 Merge branch 'master' into develop 2019-10-03 13:48:45 +01:00
Erik Johnston
dca7e32d3d
Merge pull request #6154 from matrix-org/erikj/fix_appservice_pagination 
Fix appservice room list pagination
 2019-10-02 16:50:32 +01:00
Erik Johnston
22a6ffdf91
Merge pull request #6153 from matrix-org/erikj/fix_room_list_non_federatable 
Fix not showing non-federatable rooms to remote room list queries
 2019-10-02 16:09:15 +01:00
Erik Johnston
5d8ffdfe61
Merge pull request #6148 from matrix-org/erikj/find_next_generated 
Bound find_next_generated_user_id DB query.
 2019-10-02 16:09:02 +01:00
Erik Johnston
5be4083306 Newsfile 2019-10-02 15:48:36 +01:00
Erik Johnston
ed73f04bef Newsfile 2019-10-02 15:24:33 +01:00
Erik Johnston
8e32240e6b Newsfile 2019-10-02 15:12:17 +01:00
Erik Johnston
a5166e4d5f
Land improved room list based on room stats (#6019) 
Use room_stats and room_state for room directory search
 2019-10-02 14:08:35 +01:00
Amber Brown
864f144543
Fix up some typechecking (#6150) 
* type checking fixes

* changelog
 2019-10-02 05:29:01 -07:00
Brendan Abolivier
24efea338d
Changelog 2019-10-02 12:20:03 +01:00
Andrew Morgan
2a1470cd05
Fix yields and copy instead of move push rules on room upgrade (#6144) 
Copy push rules during a room upgrade from the old room to the new room, instead of deleting them from the old room.

For instance, we've defined upgrading of a room multiple times to be possible, and push rules won't be transferred on the second upgrade if they're deleted during the first.

Also fix some missing yields that probably broke things quite a bit.
 2019-10-02 12:04:22 +01:00
Erik Johnston
de1823b521 Newsfile 2019-10-02 11:45:31 +01:00
Erik Johnston
ecd254bc49 Merge branch 'release-v1.4.0' of github.com:matrix-org/synapse into develop 2019-10-02 11:08:07 +01:00
Erik Johnston
b4fe7e19c0
Merge pull request #6146 from matrix-org/erikj/fix_destination_retry_timings 
Fix errors storing large retry intervals.
 2019-10-02 11:05:22 +01:00
Erik Johnston
35f392bb29
Merge pull request #6145 from matrix-org/erikj/fix_censored_redactions 
Fix fetching censored redactions from DB
 2019-10-02 10:57:30 +01:00
Erik Johnston
2bc027ab71 Newsfile 2019-10-02 10:41:29 +01:00
Erik Johnston
33d4ebdf78 Newsfile 2019-10-02 10:18:17 +01:00
Erik Johnston
2b8352e638 Newsfile 2019-10-01 13:47:06 +01:00
Erik Johnston
1d349fb159 Merge branch 'erikj/fixup_devices_last_seen_query' of github.com:matrix-org/synapse into develop 2019-10-01 10:17:24 +01:00
Erik Johnston
129343cd8a Newsfile 2019-09-30 12:00:34 +01:00
Richard van der Hoff
16cb9a71b8
Drop unused tables (#6115) 
These tables are unused since #5893 (as amended by #6047), so we can now drop
them.

Fixes #6048.
 2019-09-30 09:38:41 +01:00
Andrew Morgan
f3451118a6
Edit SimpleHttpClient to reference that header keys can be passed as str or bytes (#6077) 2019-09-27 17:59:18 +01:00
Brendan Abolivier
25a0a36ad9
Update changelog.d/6125.feature 
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2019-09-27 16:10:24 +01:00
Brendan Abolivier
0804a27c8c
Changelog 2019-09-27 15:14:34 +01:00
Andrew Morgan
8c27bc8b60
Move lookup-related functions from RoomMemberHandler to IdentityHandler (#5978) 
Just to have all the methods that make calls to identity services in one place.
 2019-09-27 10:36:20 +01:00
Erik Johnston
67ee18daea
Merge pull request #6108 from matrix-org/erikj/remove_get_user_by_req-span 
Add some helpful opentracing tags and remove get_user_by_req span
 2019-09-27 10:02:57 +01:00
Erik Johnston
9d99eade7c Newsfile 2019-09-26 16:46:21 +01:00
Richard van der Hoff
54569c787b
Kill off half-implemented password-reset via sms (#6101) 
Doing a password reset via SMS has never worked, and in any case is a silly
idea because msisdn recycling is a thing.

See also matrix-org/matrix-doc#2303.
 2019-09-26 15:38:25 +01:00
Richard van der Hoff
3fbca80a8d changelog 2019-09-26 13:23:48 +01:00
Neil Johnson
8b8f8c7b3c Explicitly log when a homeserver does not have a trusted key server configured (#6090) 2019-09-26 12:57:01 +01:00
Neil Johnson
1b23f991ab
Clarify upgrade notes ahead of 1.4.0 release 2019-09-26 12:30:10 +01:00
Neil Johnson
034db2ba21 Fix dummy event insertion consent bug (#6053) 
Fixes #5905
 2019-09-26 11:47:53 +01:00
Erik Johnston
4fb3c129aa Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cleanup_user_ips_2 2019-09-25 17:53:13 +01:00
Erik Johnston
9614d3c9d1
Merge pull request #6089 from matrix-org/erikj/cleanup_user_ips 
Move last seen info into devices table
 2019-09-25 17:42:39 +01:00
Neil Johnson
a4f3ca48b5 Enable cleaning up extremities with dummy events by default to prevent undue build up of forward extremities. (#5884) 2019-09-25 17:27:35 +01:00
Erik Johnston
d2bd0bc6b1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cleanup_user_ips 2019-09-25 17:16:28 +01:00
Neil Johnson
77dc7093a7 Threepid validity checks on msisdns should not be dependent on 'threepid_behaviour_email'. (#6104) 
Fixes #6103
 2019-09-25 12:29:35 +01:00
Richard van der Hoff
990928abde
Stop advertising unsupported flows for registration (#6107) 
If email or msisdn verification aren't supported, let's stop advertising them
for registration.

Fixes #6100.
 2019-09-25 12:10:26 +01:00
Erik Johnston
5c1af6d1b8 Newsfile 2019-09-25 11:42:00 +01:00
Richard van der Hoff
2cd98812ba
Refactor the user-interactive auth handling (#6105) 
Pull the checkers out to their own classes, rather than having them lost in a
massive 1000-line class which does everything.

This is also preparation for some more intelligent advertising of flows, as per #6100
 2019-09-25 11:33:03 +01:00
Richard van der Hoff
8004d6ca2f
Refactor code for calculating registration flows (#6106) 
because, frankly, it looked like it was written by an axe-murderer.

This should be a non-functional change, except that where `m.login.dummy` was
previously advertised *before* `m.login.terms`, it will now be advertised
afterwards. AFAICT that should have no effect, and will be more consistent with
the flows that involve passing a 3pid.
 2019-09-25 11:32:05 +01:00
Brendan Abolivier
f99a9c9cb0
Merge pull request #6092 from matrix-org/babolivier/background_update_deactivated_return 
Fix the return value in the users_set_deactivated_flag background job
 2019-09-24 17:11:12 +01:00
Richard van der Hoff
9b7c4f4078 Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work 2019-09-24 17:03:50 +01:00
Richard van der Hoff
566ac40939
remove unused parameter to get_user_id_by_threepid (#6099) 
Added in #5377, apparently in error
 2019-09-24 17:01:09 +01:00
Erik Johnston
f8b02c5420 Newsfile 2019-09-24 15:59:43 +01:00
Richard van der Hoff
bb82be9851
Merge pull request #6069 from matrix-org/rav/fix_attribute_mapping 
Fix a bug with saml attribute maps.
 2019-09-24 15:07:26 +01:00
J. Ryan Stinnett
40fb00f5b7
Add sid to next_link for email validation (#6097) 2019-09-24 14:39:50 +01:00
Richard van der Hoff
ed8b92f0d2 Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work 2019-09-24 12:57:32 +01:00
Andrew Morgan
50776261e1 Add submit_url response parameter to msisdn /requestToken (#6079) 
Second part of solving #6076
Fixes #6076

We return a submit_url parameter on calls to POST */msisdn/requestToken so that clients know where to submit token information to.
 2019-09-23 21:21:03 +01:00
Andrew Morgan
2b071a2ff1 Add an unstable feature flag for separate add/bind 3pid APIs (#6044) 
Add a m.separate_add_and_bind flag set to True. See MSC2290's Backward Compatibility section for details.
 2019-09-23 20:46:34 +01:00
Richard van der Hoff
78e8ec368e
Merge pull request #6064 from matrix-org/rav/saml_config_cleanup 
Make the sample saml config closer to our standards
 2019-09-23 20:36:51 +01:00
Andrew Morgan
e08ea43463 Use the federation blacklist for requests to untrusted Identity Servers (#6000) 
Uses a SimpleHttpClient instance equipped with the federation_ip_range_blacklist list for requests to identity servers provided by user input. Does not use a blacklist when contacting identity servers specified by account_threepid_delegates. The homeserver trusts the latter and we don't want to prevent homeserver admins from specifying delegates that are on internal IP addresses.

Fixes #5935
 2019-09-23 20:23:20 +01:00
Andrew Morgan
1ea3ed7620 Add m.id_access_token to /versions unstable_features (MSC2264) (#5974) 
Adds a flag to /versions' unstable_features section indicating that this Synapse understands what an id_access_token is, as per MSC2264.

Fixes #5927
 2019-09-23 20:19:25 +01:00
Andrew Morgan
b38aa82b83 Add m.require_identity_server to /versions unstable_flags (#5972) 
As MSC2263 states, m.require_identity_server must be set to false when it does not require an identity server to be provided by the client for the purposes of email registration or password reset.

Adds an m.require_identity_server flag to /versionss unstable_flags section. This will advertise that Synapse no longer needs id_server as a parameter.
 2019-09-23 18:52:43 +01:00
Andrew Morgan
2c99c63453 Add POST submit_token endpoint for MSISDN (#6078) 
First part of solving #6076
 2019-09-23 17:49:00 +01:00
Brendan Abolivier
a2a09d42dd
Changelog 2019-09-23 17:22:55 +01:00
Erik Johnston
691a70190b Newsfile 2019-09-23 16:59:45 +01:00
Andrew Morgan
30af161af2 Implement MSC2290 (#6043) 
Implements MSC2290. This PR adds two new endpoints, /unstable/account/3pid/add and /unstable/account/3pid/bind. Depending on the progress of that MSC the unstable prefix may go away.

This PR also removes the blacklist on some 3PID tests which occurs in #6042, as the corresponding Sytest PR changes them to use the new endpoints.

Finally, it also modifies the account deactivation code such that it doesn't just try to deactivate 3PIDs that were bound to the user's account, but any 3PIDs that were bound through the homeserver on that user's account.
 2019-09-23 16:50:27 +01:00
Andrew Morgan
1b519e0272 Disable /register/available if registration is disabled (#6082) 
Fixes #6066

This register endpoint should be disabled if registration is disabled, otherwise we're giving anyone the ability to check if a username exists on a server when we don't need to be.

Error code is 403 (Forbidden) as that's the same returned by /register when registration is disabled.
 2019-09-23 15:38:38 +01:00
Andrew Morgan
1c9feadf4b Generalize email sending logging (#6075) 
In ancient times Synapse would only send emails when it was notifying a user about a message they received...

Now it can do all sorts of neat things!

Change the logging so it's not just about notifications.
 2019-09-23 14:38:19 +01:00
Andrew Morgan
885a4726b7 Return timeout error to user for identity server calls (#6073) 2019-09-23 14:37:23 +01:00
Richard van der Hoff
0660f8e367
Merge pull request #6072 from matrix-org/rav/fix_retry_reset 
Fix exception when resetting retry timings
 2019-09-23 14:31:47 +01:00
Andrew Morgan
df3401a71d
Allow HS to send emails when adding an email to the HS (#6042) 2019-09-20 15:21:30 +01:00
Andrew Morgan
7763dd3e95
Remove trailing slash ability from password reset's submit_token endpoint (#6074) 
Remove trailing slash ability from the password reset submit_token endpoint. Since we provide the link in an email, and have never sent it with a trailing slash, there's no point for us to accept them on the endpoint.
 2019-09-20 14:58:37 +01:00
Richard van der Hoff
9d94313209 Fix exception when resetting retry timings 
Fixes:
> TypeError: set_destination_retry_timings() missing 1 required positional
argument: 'retry_interval'

Introduced in #6016.
 2019-09-20 12:09:39 +01:00
Andrew Morgan
aeb40f355c
Ensure email validation link parameters are URL-encoded (#6063) 
The validation links sent via email had their query parameters inserted without any URL-encoding. Surprisingly this didn't seem to cause any issues, but if a user were to put a `/` in their client_secret it could lead to problems.
 2019-09-20 10:46:59 +01:00
Andrew Morgan
3ac614eb6c
Drop support for bind param on POST /account/3pid (MSC2290) (#6067) 
As per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290/files#diff-05cde9463e9209b701312b3baf2fb2ebR151), we're dropping the bind parameter from `/account/3pid`. This endpoint can now only be used for adding threepid's to the user's account on the homeserver.
 2019-09-20 10:46:34 +01:00
Michael Kaye
2def5ea0da Docker: support SYNAPSE_WORKER envvar (#6058) 
* Allow passing SYNAPSE_WORKER envvar

* changelog.d

* Document SYNAPSE_WORKER.

Attempting to imply that you don't need to change this default
unless you're in worker mode.

Also aware that there's a bigger problem of attempting to document
a complete working configuration of workers using docker, as we
currently only document to use `synctl` for worker mode, and synctl
doesn't work that way in docker.
 2019-09-19 22:29:47 +01:00
J. Ryan Stinnett
36015d68ef Use unstable prefix for 3PID unbind API (#6062) 2019-09-19 22:28:29 +01:00
Richard van der Hoff
284e1cb027 Merge branch 'develop' into rav/fix_attribute_mapping 2019-09-19 20:32:25 +01:00
Richard van der Hoff
b74606ea22 Fix a bug with saml attribute maps. 
Fixes a bug where the default attribute maps were prioritised over
user-specified ones, resulting in incorrect mappings.

The problem is that if you call SPConfig.load() multiple times, it adds new
attribute mappers to a list. So by calling it with the default config first,
and then the user-specified config, we would always get the default mappers
before the user-specified mappers.

To solve this, let's merge the config dicts first, and then pass them to
SPConfig.
 2019-09-19 20:32:14 +01:00
Richard van der Hoff
599f786e4e
Update 6037.feature 2019-09-19 18:52:17 +01:00
Richard van der Hoff
6db5adca0c Merge branch 'rav/saml_config_cleanup' into rav/saml_mapping_work 2019-09-19 18:20:23 +01:00
Richard van der Hoff
fe349b497e
Update the upgrade notes (#6050) 
* make it clear that if you installed from a package manager, you should use
   that to upgrade

 * Document the new way of getting the server version (cf #4878)

 * Write some words about downgrading.
 2019-09-19 18:20:01 +01:00
Richard van der Hoff
b65327ff66 Merge branch 'develop' into rav/saml_mapping_work 2019-09-19 18:13:31 +01:00
Richard van der Hoff
35ce3bda7a
Add some notes on rolling back to v1.3.1. (#6049) 2019-09-19 15:06:48 +01:00
Richard van der Hoff
bcd9132869
Undo the deletion of some tables (#6047) 
This is a partial revert of #5893. The problem is that if we drop these tables
in the same release as removing the code that writes to them, it prevents users
users from being able to roll back to a previous release.

So let's leave the tables in place for now, and remember to drop them in a
subsequent release.

(Note that these tables haven't been *read* for *years*, so any missing rows
resulting from a temporary upgrade to vNext won't cause a problem.)
 2019-09-19 15:06:27 +01:00
Richard van der Hoff
b789c7eb03 Merge branch 'develop' into rav/saml_config_cleanup 2019-09-19 15:05:31 +01:00
Richard van der Hoff
84a2743e2e Add changelog 2019-09-19 15:04:02 +01:00
Andrew Morgan
62e3ff92fd
Remove POST method from password reset submit_token endpoint (#6056) 
Removes the POST method from `/password_reset/<medium>/submit_token/` as it's only used by phone number verification which Synapse does not support yet.
 2019-09-19 10:53:14 +01:00
Jorik Schellekens
38fd1f8e3f Fix typo in account_threepid_delegates config (#6028) 2019-09-18 22:30:44 +01:00
Jorik Schellekens
d58cad635e Give appropriate exit codes when synctl fails (#5992) 2019-09-18 22:27:59 +01:00
J. Ryan Stinnett
a86a290850 Fix logcontext spam on non-Linux platforms (#6059) 
This checks whether the current platform supports thread resource usage tracking
before logging a warning to avoid log spam.

Fixes https://github.com/matrix-org/synapse/issues/6055
 2019-09-18 21:55:37 +01:00
Andrew Morgan
6670bd4072
v2 3PID Invites (part of MSC2140) (#5979) 
3PID invites require making a request to an identity server to check that the invited 3PID has an Matrix ID linked, and if so, what it is.

These requests are being made on behalf of a user. The user will supply an identity server and an access token for that identity server. The homeserver will then forward this request with the access token (using an `Authorization` header) and, if the given identity server doesn't support v2 endpoints, will fall back to v1 (which doesn't require any access tokens).

Requires: ~~#5976~~
 2019-09-17 18:05:13 +01:00
dstipp
379d2a8c39 (#5849) Convert rst to markdown (#6040) 
Converting some of the rst documentation to markdown.  Attempted to
preserve whitespace and line breaks to minimize cosmetic change.
 2019-09-17 12:55:29 +01:00
Erik Johnston
70c52821ce Fix race condition in room stats. (#6029) 
Broke in #5971

Basically the bug is that if get_current_state_deltas returns no new updates and we then take the max pos, its possible that we miss an update that happens in between the two calls. (e.g. get_current_state_deltas looks up to stream pos 5, then an event persists and so getting the max stream pos returns 6, meaning that next time we check for things with a stream pos bigger than 6)
 2019-09-17 12:41:23 +01:00
Richard van der Hoff
1e19ce00bf
Add 'failure_ts' column to 'destinations' table (#6016) 
Track the time that a server started failing at, for general analysis purposes.
 2019-09-17 11:41:54 +01:00
Amber Brown
850dcfd2d3
Fix well-known lookups with the federation certificate whitelist (#5997) 2019-09-14 04:58:38 +10:00
Richard van der Hoff
b9d57502da changelog 2019-09-13 16:06:03 +01:00
Travis Ralston
c755955f33 Add developer docs for using SAML without a server (#6032) 2019-09-13 08:58:18 +01:00
Amber Brown
b617864cd9
Fix for structured logging tests stomping on logs (#6023) 2019-09-13 02:29:55 +10:00
Richard van der Hoff
3d882a7ba5
Remove the cap on federation retry interval. (#6026) 
Essentially the intention here is to end up blacklisting servers which never
respond to federation requests.

Fixes https://github.com/matrix-org/synapse/issues/5113.
 2019-09-12 13:00:13 +01:00
Richard van der Hoff
0388beafe4
Fix bug in calculating the federation retry backoff period (#6025) 
This was intended to introduce an element of jitter; instead it gave you a
30/60 chance of resetting to zero.
 2019-09-12 12:59:43 +01:00
David Baker
6db22e4702 changelog 2019-09-12 11:46:37 +01:00
Sorunome
dd2e5b0038 add report_stats_endpoint config option (#6012) 
This PR adds the optional `report_stats_endpoint` to configure where stats are reported to, if enabled.
 2019-09-12 11:24:57 +01:00
Jorik Schellekens
a8251da10f
Blow up config if opentracing is missing (#5985) 
* Blow up config if opentracing is missing
 2019-09-12 10:57:37 +01:00
Jason Robinson
6d847d8ce6 Ensure support users can be registered even if MAU limit is reached 
This allows support users to be created even on MAU limits via
the admin API. Support users are excluded from MAU after creation,
so it makes sense to exclude them in creation - except if the
whole host is in disabled state.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2019-09-11 20:48:31 +03:00
Andrew Morgan
9fc71dc5ee
Use the v2 Identity Service API for lookups (MSC2134 + MSC2140) (#5976) 
This is a redo of https://github.com/matrix-org/synapse/pull/5897 but with `id_access_token` accepted.

Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) plus Identity Service v2 authentication ala [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140).

Identity lookup-related functions were also moved from `RoomMemberHandler` to `IdentityHandler`.
 2019-09-11 16:02:42 +01:00
Erik Johnston
cbcbfe64a2
Merge pull request #6015 from matrix-org/erikj/ratelimit_admin_redaction 
Allow use of different ratelimits for admin redactions.
 2019-09-11 15:39:38 +01:00
Richard van der Hoff
7902bf1e1d
Clean up some code in the retry logic (#6017) 
* remove some unused code
* make things which were constants into constants for efficiency and clarity
 2019-09-11 15:14:56 +01:00
Andrew Morgan
9c555f37e3
Add note about extra arg to send_membership_event, remove arg in remote_reject_invite (#6009) 
Some small fixes to `room_member.py` found while doing other PRs.

1. Add requester to the base `_remote_reject_invite` method.
2. `send_membership_event`'s docstring was out of date and took in a `remote_room_hosts` arg that was not used and no calling function provided.
 2019-09-11 14:23:24 +01:00
Jorik Schellekens
6604b64fae
Check dependencies on setup in the nicer way. (#5989) 2019-09-11 14:00:37 +01:00
Andrew Morgan
3505ffcda7
Fix existing v2 identity server calls (MSC2140) (#6013) 
Two things I missed while implementing [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140/files#diff-c03a26de5ac40fb532de19cb7fc2aaf7R80).

1. Access tokens should be provided to the identity server as `access_token`, not `id_access_token`, even though the homeserver may accept the tokens as `id_access_token`.
2. Access tokens must be sent to the identity server in a query parameter, the JSON body is not allowed.

We now send the access token as part of an `Authorization: ...` header, which fixes both things.

The breaking code was added in https://github.com/matrix-org/synapse/pull/5892

Sytest PR: https://github.com/matrix-org/sytest/pull/697
 2019-09-11 11:59:45 +01:00
Erik Johnston
2434c0084b Newsfile 2019-09-11 10:48:52 +01:00
Andrew Morgan
cd17a2085e
Remove origin parameter from add_display_name_to_third_party_invite and add params to docstring (#6010) 
Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument.

Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
 2019-09-11 10:37:17 +01:00
Erik Johnston
5e9b05d7da
Merge pull request #6011 from matrix-org/anoa/fix_3pid_validation 
Use account_threepid_delegate for 3pid validation
 2019-09-10 18:15:07 +01:00
Andrew Morgan
b5833a2abf Add changelog 2019-09-10 17:56:10 +01:00
Jason Robinson
63f9317b8e
Merge pull request #6004 from matrix-org/jaywink/autojoin-create-real-users 
Only count real users when checking for auto-creation of auto-join room
 2019-09-09 17:37:52 +03:00
Erik Johnston
470dc621ae
Merge pull request #5934 from matrix-org/erikj/censor_redactions 
Censor redactions in DB after a month
 2019-09-09 15:29:39 +01:00
Amber Brown
aeb9b2179e
Add a build info metric to Prometheus (#6005) 2019-09-10 00:14:58 +10:00
Jason Robinson
be618e0551 Only count real users when checking for auto-creation of auto-join room 
Previously if the first registered user was a "support" or "bot" user,
when the first real user registers, the auto-join rooms were not
created.

Fix to exclude non-real (ie users with a special user type) users
when counting how many users there are to determine whether we should
auto-create a room.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2019-09-09 14:48:08 +03:00
Erik Johnston
a852e93408 Newsfile 2019-09-09 10:24:14 +01:00
Hubert Chathi
8e86f5b65c Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-09-07 13:20:34 -04:00
Amber Brown
55d5b3af88
Servers-known-about statistic (#5981) 2019-09-07 01:45:51 +10:00
Andrew Morgan
78801e7f9e
Ensure a sid parameter is passed to bind_threepid (#5995) 
`sid` is required to be part of `three_pid_creds`. We were 500'ing if it wasn't provided instead of returning `M_MISSING_PARAM`.
 2019-09-06 15:36:50 +01:00
Erik Johnston
a2a695b7ec
Merge pull request #5998 from matrix-org/erikj/fixup_federate_flag 
Correctly handle non-bool m.federate flag
 2019-09-06 15:32:43 +01:00
Erik Johnston
85275c89d7 Newsfile 2019-09-06 14:21:14 +01:00
Erik Johnston
30b67e0f63
Merge pull request #5993 from matrix-org/anoa/worker_store_reg 
Move get_threepid_validation_session and delete_threepid_session into RegistrationWorkerStore
 2019-09-06 14:10:02 +01:00
Andrew Morgan
5d833f0923 Add changelog 2019-09-06 13:27:55 +01:00
Andrew Morgan
5a7e9fdd84 Change changelog 2019-09-06 13:18:03 +01:00
Andrew Morgan
1ab1479a92 Add changelog 2019-09-06 13:08:52 +01:00
Erik Johnston
146af7b47f
Merge pull request #5991 from matrix-org/erikj/fix_tracing_funcs 
Don't assume there is a 'self' arg in @trace decorator
 2019-09-06 11:42:45 +01:00
Andrew Morgan
0c0b82b6d1
Allow Synapse to send registration emails + choose Synapse or an external server to handle 3pid validation (#5987) 
This is a combination of a few different PRs, finally all being merged into `develop`:

* #5875 
* #5876 
* #5868 (This one added the `/versions` flag but the flag itself was actually [backed out](891afb57cb (diff-e591d42d30690ffb79f63bb726200891)) in #5969. What's left is just giving /versions access to the config file, which could be useful in the future)
* #5835 
* #5969 
* #5940

Clients should not actually use the new registration functionality until https://github.com/matrix-org/synapse/pull/5972 is merged.

UPGRADE.rst, changelog entries and config file changes should all be reviewed closely before this PR is merged.
 2019-09-06 11:35:28 +01:00
Erik Johnston
e5baf80237 Update changelog 2019-09-06 10:53:05 +01:00
Erik Johnston
4bc6b7130d Newsfile 2019-09-06 10:13:10 +01:00
Jorik Schellekens
f7c873a643
Trace how long it takes for the send trasaction to complete, including retrys (#5986) 2019-09-05 17:44:55 +01:00
Jorik Schellekens
bc604e7f94
Gracefully handle log context slips and missing opentracing import errors. (#5988) 2019-09-05 17:33:29 +01:00
Erik Johnston
591d82f06b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/censor_redactions 2019-09-05 17:27:46 +01:00
Erik Johnston
1a6ae33309
Merge pull request #5984 from matrix-org/joriks/opentracing_link_send_to_edu_contexts 
Link the send loop with the edus contexts
 2019-09-05 15:22:24 +01:00
Jorik Schellekens
7093790fbc
Bugfix phrasing 
Co-Authored-By: Erik Johnston <erik@matrix.org>
 2019-09-05 15:07:00 +01:00
Jorik Schellekens
909827b422
Add opentracing to all client servlets (#5983) 2019-09-05 14:46:04 +01:00
Jorik Schellekens
93bc9d73bf newsfile 2019-09-05 14:45:07 +01:00
Andrew Morgan
a0d294c306
Switch to using v2 Identity Service APIs other than lookup (MSC 2140) (#5892) 2019-09-05 14:31:22 +01:00
Jorik Schellekens
b9cfd3c375
Fix opentracing contexts missing from outbound replication requests (#5982) 2019-09-05 14:22:15 +01:00
Andrew Morgan
90d17a3d28
Add POST /_matrix/client/r0/account/3pid/unbind (MSC2140) (#5980) 
Implements `POST /_matrix/client/r0/account/3pid/unbind` from [MSC2140](https://github.com/matrix-org/matrix-doc/blob/dbkr/tos_2/proposals/2140-terms-of-service-2.md#post-_matrixclientr0account3pidunbind).
 2019-09-05 14:00:30 +01:00
Hubert Chathi
faf72a4c40 Merge branch 'develop' into cross-signing_keys 2019-09-04 19:12:29 -04:00
Hubert Chathi
c75ad8a6d0 Merge branch 'develop' into uhoreg/e2e_backup_hash 2019-09-04 18:45:09 -04:00
Andrew Morgan
b736c6cd3a
Remove bind_email and bind_msisdn (#5964) 
Removes the `bind_email` and `bind_msisdn` parameters from the `/register` C/S API endpoint as per [MSC2140: Terms of Service for ISes and IMs](https://github.com/matrix-org/matrix-doc/pull/2140/files#diff-c03a26de5ac40fb532de19cb7fc2aaf7R107).
 2019-09-04 18:24:23 +01:00
Andrew Morgan
b09d443632
Cleanup event auth type initialisation (#5975) 
Very small code cleanup.
 2019-09-04 16:16:56 +01:00
Erik Johnston
6e834e94fc
Fix and refactor room and user stats (#5971) 
Previously the stats were not being correctly populated.
 2019-09-04 13:04:27 +01:00
Travis Ralston
2f416fc997
Ensure the list media admin API is always available (#5966) 
* Ensure the list media admin API is always available

This API is required for some external media repo implementations to operate (mostly for doing quarantine operations on a room).

* changelog
 2019-09-03 13:35:20 -06:00
Michael Kaye
894c1a5759
Docker packaging should not su-exec or chmod if already running as UID/GID (#5970) 
Adjust su-exec to only be used if needed.

If UID == getuid() and GID == getgid() then we do not need to su-exec, and chmod will not work.
 2019-09-03 16:36:01 +01:00
Travis Ralston
0eac7077c9
Ensure an auth instance is available to ListMediaInRoom (#5967) 
* Ensure an auth instance is available to ListMediaInRoom

Fixes https://github.com/matrix-org/synapse/issues/5737

* Changelog
 2019-09-03 09:01:30 -06:00
Andrew Morgan
2a44782666
Remove double return statements (#5962) 
Remove all the "double return" statements which were a result of us removing all the instances of

```
defer.returnValue(...)
return
```

statements when we switched to python3 fully.
 2019-09-03 11:42:45 +01:00
Jorik Schellekens
a90d16dabc
Opentrace device lists (#5853) 
Trace device list changes.
 2019-09-03 10:21:30 +01:00
Andrew Morgan
36f34e6f3d
Remove unused methods from c/s api v1 in register.py (#5963) 
These methods were part of the v1 C/S API. Remove them as they are no longer used by any code paths.
 2019-09-02 18:29:21 +01:00
L0ric0
ce7803b8b0 fix thumbnail storage location (#5915) 
* fix thumbnail storage location

Signed-off-by: Lorenz Steinert <lorenz@steinerts.de>

* Add changelog file.

Signed-off-by: Lorenz Steinert <lorenz@steinerts.de>

* Update Changelog

Signed-off-by: Lorenz Steinert <lorenz@steinerts.de>
 2019-09-02 12:18:41 +01:00
Aaron Raimist
cee00a3584 Update INSTALL.md to say that Python 2 is no longer supported (#5953) 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-09-02 11:27:39 +01:00
Hubert Chathi
e9d587d0d8
Apply suggestions from code review 
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2019-08-31 15:46:49 -07:00
Andrew Morgan
2a012e8a04
Revert "Add m.id_access_token flag (#5930)" (#5945) 
This reverts commit 4765f0cfd9.
 2019-08-30 17:13:37 +01:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements (#5931) 
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
 2019-08-30 16:28:26 +01:00
Amber Brown
4fca313389
Move buildkite config to the pipelines repo (#5943) 2019-08-31 01:01:57 +10:00
Andrew Morgan
4765f0cfd9
Add m.id_access_token flag (#5930) 
Adds a flag to `/versions`' `unstable_features` section indicating that this Synapse understands what an `id_access_token` is, as per https://github.com/matrix-org/synapse/issues/5927#issuecomment-523566043

Fixes #5927
 2019-08-30 15:22:51 +01:00
Amber Brown
d19505a8c1
Removed unused jenkins/ folder and script (#5938) 2019-08-30 23:13:16 +10:00
Andrew Morgan
3057095a5d Revert "Use the v2 lookup API for 3PID invites (#5897)" (#5937) 
This reverts commit 71fc04069a.

This broke 3PID invites as #5892 was required for it to work correctly.
 2019-08-30 12:00:20 +01:00
Erik Johnston
549f974897 Newsfile 2019-08-30 11:29:17 +01:00
Amber Brown
e7011280c7
Fix coverage in sytest and use plugins for buildkite (#5922) 2019-08-29 22:19:57 +10:00
Hubert Chathi
e3d3fbf63f Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys 2019-08-28 17:36:46 -07:00
Hubert Chathi
72d296a7f3 Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-08-28 17:33:44 -07:00
Jorik Schellekens
92c1550f4a Add a link to python's logging config schema (#5926) 2019-08-28 19:08:32 +01:00
Will Hunt
c8fa620d7a
Merge pull request #5902 from matrix-org/hs/exempt-support-users-from-consent 
Exempt support users from consent
 2019-08-28 16:31:40 +01:00
Jorik Schellekens
deca277d09
Let synctl use a config directory. (#5904) 
* Let synctl use a config directory.
 2019-08-28 15:55:58 +01:00
Will Hunt
5798a134c0
Removing entry for 5903 2019-08-28 14:25:05 +01:00
Andrew Morgan
71fc04069a
Use the v2 lookup API for 3PID invites (#5897) 
Fixes https://github.com/matrix-org/synapse/issues/5861

Adds support for the v2 lookup API as defined in [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134). Currently this is only used for 3PID invites.

Sytest PR: https://github.com/matrix-org/sytest/pull/679
 2019-08-28 14:59:26 +02:00
Jorik Schellekens
6d97843793
Config templating (#5900) 
Template config files

* Imagine a system composed entirely of x, y, z etc and the basic operations..

Wait George, why XOR? Why not just neq?

George: Eh, I didn't think of that..

Co-Authored-By: Erik Johnston <erik@matrix.org>
 2019-08-28 13:12:22 +01:00
Amber Brown
7dc398586c
Implement a structured logging output system. (#5680) 2019-08-28 21:18:53 +10:00
Richard van der Hoff
49ef8ec399
Fix a cache-invalidation bug for worker-based deployments (#5920) 
Some of the caches on worker processes were not being correctly invalidated
when a room's state was changed in a way that did not affect the membership
list of the room.

We need to make sure we send out cache invalidations even when no memberships
are changing.
 2019-08-28 10:18:16 +01:00
reivilibre
a3f0635686
Merge pull request #5914 from matrix-org/rei/admin_getadmin 
Add GET method to admin API /users/@user:dom/admin
 2019-08-28 09:44:22 +01:00
reivilibre
7ccc251415
Merge pull request #5859 from matrix-org/rei/msc2197 
MSC2197 Search Filters over Federation
 2019-08-28 09:00:21 +01:00
Erik Johnston
dfd10f5133
Merge pull request #5864 from matrix-org/erikj/reliable_lookups 
Refactor MatrixFederationAgent to retry SRV.
 2019-08-27 16:54:06 +01:00
Olivier Wilkinson (reivilibre)
c88a119259 Add GET method to admin API /users/@user:dom/admin 
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
 2019-08-27 13:12:27 +01:00
reivilibre
322ccac33f
Allow schema deltas to be engine-specific (#5911) 
* Allow schema deltas to be engine-specific

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Code style (Black)

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
 2019-08-27 11:53:21 +01:00
Richard van der Hoff
ccb15a5bbe
Merge pull request #5906 from matrix-org/neilj/increase_display_name_limit 
Increase profile display name limit
 2019-08-27 11:52:59 +01:00
Erik Johnston
f5b50d0871
Merge pull request #5895 from matrix-org/erikj/notary_key 
Add config option to sign remote key query responses with a separate key.
 2019-08-27 11:51:37 +01:00
Richard van der Hoff
e7577427c9
Update 5909.misc 2019-08-27 11:50:52 +01:00
Richard van der Hoff
7837a5f2ea
Merge pull request #5909 from aaronraimist/public_base_url 
public_base_url is actually public_baseurl
 2019-08-27 11:49:59 +01:00
reivilibre
1a7e6eb633
Add Admin API capability to set adminship of a user (#5878) 
Admin API: Set adminship of a user
 2019-08-27 10:14:00 +01:00
Olivier Wilkinson (reivilibre)
62a1639287 Newsfile 
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
 2019-08-27 09:36:12 +01:00
Aaron Raimist
c25137a99f
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-08-26 21:06:10 -05:00