mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-12-22 20:50:23 +03:00
Code Issues Projects Releases Packages Wiki Activity
13901 commits 936 branches 734 tags 467 MiB
Commit graph

1222 commits

Author SHA1 Message Date
Richard van der Hoff
b7c0218812
Check jinja version for consent resource (#4327) 
* Raise a ConfigError if an invalid resource is specified

* Require Jinja 2.9 for the consent resource

* changelog
 2019-01-07 10:14:31 +00:00
Michael Telatynski
a27e501b09 fix the check for whether is_url to match all the other ones in codebase (#3405) 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
 2019-01-06 21:43:25 +00:00
Richard van der Hoff
29f20a8a1a
Update debian Conflicts specifications (#4349) 
...  to allow installation alongside our matrix-synapse transitional package.
 2019-01-04 17:24:13 +00:00
Matthew Hodgson
27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Aaron Raimist
37f8bdc1d5 Update README to not lie about required restart (#4343) 
* Update README to not lie about required restart

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-02 16:32:37 +00:00
Neil Johnson
84b6fae1f5 Ensure synchrotrons can access is_support_user in the storage layer 2019-01-02 10:19:59 +00:00
Aaron Raimist
252c0c81fa Update PR template to use absolute links (#4341) 
* Update PR template to use absolute links

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-02 07:33:13 +00:00
Richard van der Hoff
e9cdfedff3
Avoid packaging _trial_temp directory (#4326) 
Make sure we don't put the _trial_temp directory in the package target
directory.

Fixes https://github.com/matrix-org/synapse/issues/4322
 2019-01-02 07:30:31 +00:00
Richard van der Hoff
7134832c01
Install the optional dependencies into the debian package (#4325) 
since #4298, the optional dependencies are no longer installed with a simple
`pip install .`, which meant that they were not being included in the debian
package.

The easy fix to that is dh_virtualenv --extras, but that needs dh_virtualenv
1.1...
 2019-01-02 07:17:39 +00:00
Aaron Raimist
7975d39cbd Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-01 15:40:55 -06:00
Amber Brown
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00
Krithin Sitaram
98df67a8de Remove mention of lt-cred-mech in the sample coturn config. (#4333) 
* Remove mention of lt-cred-mech in the sample coturn config.

See https://github.com/coturn/coturn/pull/262 for more context.
Also clean up some minor formatting issues while I'm here.

* Add changelog.

Signed-off-by: Krithin Sitaram <krithin@gmail.com>
 2018-12-28 23:31:49 +00:00
Will Hunt
0708f437cc Log roomid along with Unknown room (#4297) 2018-12-24 10:49:42 +01:00
Richard van der Hoff
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms 
Add v2 room version
 2018-12-24 10:48:59 +01:00
Richard van der Hoff
f85676cc93
Return well_known in /login response (#4319) 
... as per MSC1730.
 2018-12-24 10:44:33 +01:00
Richard van der Hoff
3355c1a9ec
Merge pull request #4317 from de-vri-es/test-metric-prometheus-0.5 
Fix test_metrics.py compatibility prometheus_client 0.5
 2018-12-23 00:14:47 +01:00
Richard van der Hoff
597dafb5e8
Merge pull request #4316 from matrix-org/rav/fix_docker_upload 
Fix circleci config for synapse:latest docker upload
 2018-12-21 17:33:34 +01:00
Richard van der Hoff
9c2af7b2c5 Add a script to generate a clean config file (#4315) 2018-12-22 02:04:57 +11:00
Richard van der Hoff
f3561f8d86 Fix indentation in default config (#4313) 
These settings are not supposed to be under 'listeners'.
 2018-12-22 02:01:43 +11:00
Amber Brown
ea6abf6724
Fix IP URL previews on Python 3 (#4215) 2018-12-22 01:56:13 +11:00
Amber Brown
c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools (#4298) 2018-12-22 01:37:26 +11:00
Maarten de Vries
48b7ff7a35 Fix test_metrics.py compatibility prometheus_client 0.5 
prometheus_client 0.5 has a named-tuple Sample type with more member
than the old plain tuple had. This commit makes sure the unit test
detects this and changes the way it reads the sample.

Signed-off-by: Maarten de Vries <maarten@de-vri.es>
 2018-12-21 01:53:57 +01:00
Richard van der Hoff
d9aaf26539 Fix circleci config for synapse:latest docker upload 
Give the image the right tag, so that we can push it.
 2018-12-21 00:22:31 +00:00
Nathan Pennie
da8628ba2c
Create 4309.bugfix 2018-12-19 14:42:49 -05:00
Erik Johnston
1dc7492ce5 Newsfile 2018-12-18 18:11:56 +00:00
Neil Johnson
7e22cd90f5
ensure can report mau stats when hs.config.mau_stats_only is set (#4305) 
* ensure can report mau stats when hs.config.mau_stats_only is set
 2018-12-18 14:36:11 +00:00
Neil Johnson
d2f7c4e6b1
create support user (#4141) 
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
 2018-12-14 18:20:59 +00:00
Amber Brown
e93a0ebf50
Settings Fix deleting e2e room keys on xenial (#4295) 2018-12-14 05:10:31 +11:00
Nad Chishtie
cef8ae272a Improved welcome page (#4294) 
* Improved static/index.html look, feel and content.

* Added accessibility tags to in-line SVG.

* Grammar.
 2018-12-14 02:42:51 +11:00
Richard van der Hoff
a83826ae99
Merge pull request #4274 from matrix-org/michaelkaye/update_kernel_dco_link 
Update link to kernel.org DCO usage
 2018-12-11 20:55:43 +01:00
Richard van der Hoff
f208f608cb Merge branch 'release-v0.34.0' into develop 2018-12-11 15:43:20 +00:00
Richard van der Hoff
eb835bf65b 0.34.0rc2 2018-12-11 15:35:47 +00:00
Richard van der Hoff
03116da984 remove changelog files 2018-12-11 15:35:10 +00:00
Richard van der Hoff
188945713e
Merge pull request #4290 from matrix-org/rav/remove_webclient 
Stop installing Matrix Console by default
 2018-12-11 16:24:15 +01:00
Richard van der Hoff
bd52978fd7 Merge branch 'rav/disable_pager_in_ci' into rav/welcome_page 2018-12-11 14:13:02 +00:00
Richard van der Hoff
2755a0d48a Disable pager for ci script 
... otherwise it hangs
 2018-12-11 14:12:22 +00:00
Richard van der Hoff
f537432ef9 Add a welcome page to the static resources 
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.

The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.

It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.

It's based on the work done by @krombel for #2601.
 2018-12-11 13:26:22 +00:00
Richard van der Hoff
df96177ca7 Stop installing Matrix Console by default 
This is based on the work done by @krombel in #2601.
 2018-12-11 13:20:33 +00:00
David Baker
89ac2a5bdb Add 'sandbox' to CSP for media repo (#4284) 
* Add 'sandbox' to the CSP for media repo

* Changelog
 2018-12-11 04:05:02 +11:00
Will Hunt
989f1167af
Merge pull request #4279 from matrix-org/hs/fix-config-cors 
Make /config more CORS-y
 2018-12-10 15:24:03 +00:00
Erik Johnston
74c3606c53 Newsfile 2018-12-10 14:15:43 +00:00
Will Hunt
91206e09f2 changelog & isort 2018-12-09 17:39:44 +00:00
Michael Kaye
912a843294
changelog 2018-12-07 17:50:52 +00:00
Richard van der Hoff
30da50a5b8
Initialise user displayname from SAML2 data (#4272) 
When we register a new user from SAML2 data, initialise their displayname
correctly.
 2018-12-07 14:44:46 +01:00
Richard van der Hoff
35e13477cf
Update the example systemd config to use a virtualenv (#4273) 
If you're installing as a system package, the system package should have set up
the systemd config, so it's more useful to give an example of running in a
virtualenv here.
 2018-12-07 14:43:41 +01:00
Richard van der Hoff
c7401a697f
Implement SAML2 authentication (#4267) 
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.

What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
 2018-12-07 13:11:11 +01:00
Richard van der Hoff
c588b9b9e4
Factor SSO success handling out of CAS login (#4264) 
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
 2018-12-07 13:10:07 +01:00
rkfg
ae19a7db8c Prevent crash on pagination. 2018-12-06 13:32:05 +03:00
Richard van der Hoff
b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265) 
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
 2018-12-06 19:44:38 +11:00
Richard van der Hoff
9a3e24a13d drop undocumented dependency on dateutil (#4266) 
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
 2018-12-06 04:52:42 +11:00
Richard van der Hoff
e8d98466b0
Implement .well-known handling (#4262) 
Sometimes it's useful for synapse to generate its own .well-known file.
 2018-12-05 14:38:58 +01:00
Richard van der Hoff
a2ed0f287e
Merge pull request #4260 from matrix-org/rav/python3 
Notes on upgrading to python3, and README updates.
 2018-12-04 14:46:31 +01:00
Richard van der Hoff
75937e9033 Remove obsolete settings from docker homeserver.yaml 
These aren't used, because we have a `log_config` setting.
 2018-12-04 12:31:00 +00:00
Richard van der Hoff
4acd1a3549 Notes on upgrading to python3, and README updates. 2018-12-04 12:28:24 +00:00
Travis Ralston
1737753a62 Add an option to enable recording IPs for appservice users (#3831) 2018-12-04 12:44:41 +01:00
Amber Brown
fd96dd75a3 Fix non-ASCII pushrules (#4248) 2018-12-04 12:44:02 +01:00
Travis Ralston
158ffb92f1 Add an option to disable search for homeservers which may not be interested in it (#4230) 
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
 2018-12-04 12:01:02 +01:00
Aaron Raimist
512e94d230 Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers (#4224) 
* Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-04 11:59:09 +01:00
Ben Parsons
b5ac0ffa0a add more detail to logging regarding "More than one row matched" error (#4234) 2018-12-04 11:57:39 +01:00
Richard van der Hoff
ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english (#4253) 
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
 2018-12-04 11:55:52 +01:00
Richard van der Hoff
f144c0a210
Merge pull request #4244 from aaronraimist/drop-sent-txt 
Drop sent_transactions
 2018-12-04 11:41:28 +01:00
Richard van der Hoff
48972ce9d1
Patch defer.inlineCallbacks to check logcontexts in tests (#4205) 2018-12-04 11:30:32 +01:00
Richard van der Hoff
a077e710a3
Merge pull request #4250 from matrix-org/hawkowl/pusher-remove-py3 
Fix removing pushers on python 3
 2018-12-04 11:22:46 +01:00
Richard van der Hoff
a484735bb0
Merge pull request #4257 from aaronraimist/add-editorconfig 
Add a basic .editorconfig
 2018-12-04 11:10:02 +01:00
Richard van der Hoff
52e87fbfbe
Run the AS senders as background processes (#4189) 
This should fix some "Starting db connection from sentinel context" warnings,
and will mean we get metrics for these processes.
 2018-12-04 10:53:49 +01:00
Aaron Raimist
3518c28aa8
Add a basic .editorconfig 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-03 22:38:47 -06:00
Amber Brown
998ba41493 changelog 2018-12-03 22:28:12 +11:00
Richard van der Hoff
c03324294d Workaround for non-ascii event ids (#4241) 
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.

Fixes #4226
 2018-12-03 21:47:48 +11:00
Aaron Raimist
44dc4c365b
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-01 23:10:21 -06:00
Neil Johnson
7039ece8fb Neilj/fix autojoin (#4223) 
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
 2018-11-28 22:24:57 +11:00
Amber Brown
8ca53fb53e
Report combined coverage to codecov (#4225) 2018-11-28 20:59:31 +11:00
Neil Johnson
f9b136a886 Neilj/fix mau initial reserved users (#4211) 
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail

* towncrier

* white space
 2018-11-28 20:33:41 +11:00
Richard van der Hoff
944d524f18 Support m.login.sso (#4220) 
* Clean up the CSS for the fallback login form

I was finding this hard to work with, so simplify a bunch of things. Each
flow is now a form inside a div of class login_flow.

The login_flow class now has a fixed width, as that looks much better than each
flow having a differnt width.

* Support m.login.sso

MSC1721 renames m.login.cas to m.login.sso. This implements the change
(retaining support for m.login.cas for older clients).

* changelog
 2018-11-27 18:51:52 +11:00
Richard van der Hoff
a44c0a096f Check logcontexts before and after each test (#4190) 
* Add better diagnostics to flakey keyring test

* fix interpolation fail

* Check logcontexts before and after each test

* update changelog

* update changelog
 2018-11-27 13:47:18 +11:00
Richard van der Hoff
80527b568d Fix more logcontext leaks in tests (#4209) 2018-11-27 13:01:04 +11:00
Richard van der Hoff
de8772a655 Do a GC after each test to fix logcontext leaks (#4227) 
* Some words about garbage collections and logcontexts

* Do a GC after each test to fix logcontext leaks

This feels like an awful hack, but...

* changelog
 2018-11-27 13:00:33 +11:00
Richard van der Hoff
6c18cc4b50 Ignore __pycache__ directories in schema delta dir 
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
 2018-11-20 22:52:34 +00:00
Neil Johnson
78ba0e7ab8 Remove riot.im from the list of trusted Identity Servers in the default configuration (#4207) 2018-11-20 12:29:25 +01:00
Richard van der Hoff
416c671474
Merge pull request #4204 from matrix-org/rav/logcontext_leak_fixes 
Fix some logcontext leaks
 2018-11-20 12:19:19 +01:00
Amber Brown
31425d82a3 Merge remote-tracking branch 'origin/master' into develop 2018-11-19 12:55:25 -06:00
Amber Brown
80cac86b2c
Fix fallback auth on Python 3 (#4197) 2018-11-19 12:27:33 -06:00
Richard van der Hoff
0c05da2e2e changelog 2018-11-19 17:07:42 +00:00
Richard van der Hoff
10cdf519aa
Merge pull request #4182 from aaronraimist/update-issue-template 
Add a pull request template and add multiple issue templates
 2018-11-19 14:24:30 +01:00
Aaron Raimist
cc2cf2da97
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-11-18 12:42:08 -06:00
Amber Brown
4285c818ec
Merge pull request #4193 from kivikakk/add-openbsd-prereq 
add jpeg to OpenBSD prereq list
 2018-11-17 14:27:53 -06:00
Ashe Connor
ceca3b2f30
add changelog.d entry 2018-11-17 15:01:02 +11:00
Travis Ralston
d75db3df59 Changelog 2018-11-15 20:44:57 -07:00
Amber Brown
8b1affe7d5
Fix Content-Disposition in media repository (#4176) 2018-11-15 15:55:58 -06:00
Travis Ralston
835779f7fb Add option to track MAU stats (but not limit people) (#3830) 2018-11-15 18:08:27 +00:00
Amber Brown
df758e155d
Use <meta> tags to discover the per-page encoding of html previews (#4183) 2018-11-15 11:05:08 -06:00
Amber Brown
a51288e5d6
Add a coveragerc (#4180) 2018-11-15 10:50:08 -06:00
Richard van der Hoff
4f8bb633c7 Update README for #1491 fix 2018-11-15 10:03:36 +00:00
Richard van der Hoff
0c4dc6fd76 changelog 2018-11-14 10:48:08 +00:00
David Baker
0869566ad3
Merge pull request #4113 from matrix-org/dbkr/e2e_backup_versions_are_numbers 
Make e2e backup versions numeric in the DB
 2018-11-14 07:55:48 +00:00
Aaron Raimist
9ca1215582
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-11-13 21:46:48 -06:00
David Baker
bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
Erik Johnston
db5a1c059a
Merge pull request #4166 from matrix-org/erikj/drop_unknown_events 
Drop incoming events from federation for unknown rooms
 2018-11-09 17:59:34 +00:00
Erik Johnston
7b22421a7b
Merge pull request #4164 from matrix-org/erikj/fix_device_comparison 
Fix noop checks when updating device keys
 2018-11-08 14:37:20 +00:00
Erik Johnston
94896d7ffe Newsfile 2018-11-08 12:30:25 +00:00
Erik Johnston
06c3d8050f Newsfile 2018-11-08 12:18:41 +00:00
Richard van der Hoff
de6223836e changelog 2018-11-08 11:06:28 +00:00
Amber Brown
264cb14402
Port hash_password to Python 3 (#4161) 
* port hash_password

* changelog
 2018-11-08 04:57:28 +11:00
Amber Brown
b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157) 2018-11-08 01:37:43 +11:00
Richard van der Hoff
c8ba79327b
Merge pull request #4155 from rubo77/purge-api 
add purge_history.sh and purge_remote_media.sh scripts
 2018-11-07 14:06:41 +00:00
rubo77
2904d133f3 add purge_history.sh and purge_remote_media.sh scripts to contrib/purge_api/ 
Signed-off-by: Ruben Barkow <github@r.z11.de>
 2018-11-07 14:02:41 +01:00
Amber Brown
e62f7f17b3
Remove some boilerplate in tests (#4156) 2018-11-07 03:00:00 +11:00
Travis Ralston
0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142) 
So people can still collect consent the old way if they want to.
 2018-11-06 10:32:34 +00:00
Hubert Chathi
f1087106cf
handle empty backups according to latest spec proposal (#4123) 
fixes #4056
 2018-11-05 17:59:29 -05:00
Amber Brown
efdcbbe46b
Tests for user consent resource (#4140) 2018-11-06 05:53:44 +11:00
Amber Brown
5a63589e80
Add some tests for the HTTP pusher (#4149) 2018-11-06 05:53:24 +11:00
Erik Johnston
bc80b3f454 Add helpers for getting prev and auth events (#4139) 
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
 2018-11-06 00:35:15 +11:00
Amber Brown
0467384d2f
Set the encoding to UTF8 in the default logconfig (#4138) 2018-11-03 02:28:07 +11:00
Erik Johnston
90d713b8c6
Merge pull request #4137 from matrix-org/erikj/clean_up_events 
Clean up event accesses and tests
 2018-11-02 14:12:49 +00:00
Erik Johnston
76cd7de108 Newsfile 2018-11-02 13:45:56 +00:00
Amber Brown
cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Richard van der Hoff
efb9343c8c
Merge pull request #4132 from matrix-org/rav/fix_device_list_locking 
Fix locked upsert on device_lists_remote_cache
 2018-11-02 10:50:53 +00:00
Richard van der Hoff
00f12e00f8
Merge pull request #4133 from matrix-org/travis/fix-terms-auth 
Fix logic error that prevented guests from seeing the privacy policy
 2018-11-02 10:50:43 +00:00
Erik Johnston
b199534518
Merge pull request #4135 from matrix-org/erikj/fix_state_res_none 
Fix None exception in state res v2
 2018-11-02 10:45:57 +00:00
Richard van der Hoff
1cc6671ec4 changelog 2018-11-02 10:36:13 +00:00
Erik Johnston
f05d97e283 Newsfile 2018-11-02 10:32:06 +00:00
Travis Ralston
552f090f62 Changelog 2018-11-01 16:51:11 -06:00
Richard van der Hoff
3149d55b7d
Merge pull request #3778 from z3ntu/patch-1 
Fix build of Docker image with docker-compose
 2018-11-01 17:34:56 +00:00
Travis Ralston
c68aab1536
Merge pull request #4004 from matrix-org/travis/login-terms 
Add m.login.terms to the registration flow
 2018-11-01 11:03:38 -06:00
Erik Johnston
1b21e771d0
Merge pull request #4128 from matrix-org/erikj/state_res_v2_version 
Add STATE_V2_TEST room version
 2018-11-01 13:17:57 +00:00
Erik Johnston
62d683161e Newsfile 2018-11-01 11:44:44 +00:00
Amber Brown
073d400b84 Merge branch 'master' into develop 2018-11-01 21:32:12 +11:00
Travis Ralston
d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Richard van der Hoff
1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades 
Attempt to move room aliases on room upgrades
 2018-10-31 17:52:18 +00:00
Richard van der Hoff
94c7fadc98 Attempt to move room aliases on room upgrades 2018-10-31 15:43:57 +00:00
Richard van der Hoff
9b827c40ca
Log some bits about event creation (#4121) 
I found these helpful in debugging my room upgrade tests.
 2018-10-31 15:42:23 +00:00
Richard van der Hoff
60f128a401
Merge pull request #4124 from matrix-org/rav/fix_tox 
Attempt to fix tox installs
 2018-10-31 15:41:55 +00:00
Amber Brown
916efc8249
Remove fetching keys via the deprecated v1 kex method (#4120) 2018-10-31 23:14:39 +11:00
Amber Brown
f79f454485
Remove deprecated v1 key exchange endpoint (#4119) 2018-10-31 22:29:02 +11:00
Richard van der Hoff
a2d8bff0dc changelog 2018-10-30 21:21:05 +00:00
Amber Brown
e615e95590 changelog 2018-10-31 06:28:11 +11:00
Amber Brown
086e1a8f3e Fix search 500ing (#4122) 2018-10-31 06:26:46 +11:00
Amber Brown
3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown
2e223a8c22
Remove the unused /pull federation API (#4118) 2018-10-31 04:24:59 +11:00
Erik Johnston
0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups 
Delete unreferenced state groups during purge
 2018-10-30 16:58:22 +00:00
Amber Brown
0dce9e1379
Write some tests for the email pusher (#4095) 2018-10-30 23:55:43 +11:00
David Baker
4eacf0f200 news fragment 2018-10-30 10:05:51 +00:00
Erik Johnston
169851b412
Merge pull request #4109 from matrix-org/erikj/repl_devices 
A couple of replication fixes for device lists
 2018-10-29 18:16:48 +00:00
Erik Johnston
4f0fa7a120 Newsfile 2018-10-29 18:15:42 +00:00
Erik Johnston
39f419868f Newsfile 2018-10-29 17:38:09 +00:00
Amber Brown
4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff
7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar 
Remember to copy the avatar on room upgrades
 2018-10-29 12:49:21 +00:00
Richard van der Hoff
56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls 
Better handling of odd PLs during room upgrades
 2018-10-29 12:48:51 +00:00
Richard van der Hoff
bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades 
Room version upgrade support
 2018-10-29 12:47:20 +00:00
Amber Brown
c4b3698a80
Make the replication logger quieter (#4108) 2018-10-29 22:59:44 +11:00
Amber Brown
3b0a85fc8e changelog 2018-10-29 21:54:01 +11:00
Richard van der Hoff
db24d7f15e Better handling of odd PLs during room upgrades 
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
 2018-10-27 00:54:26 +01:00
Richard van der Hoff
5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00
Erik Johnston
03e634dad4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-26 16:22:45 +01:00
Amber Brown
77d70a7646
Port register_new_matrix_user to Python 3 and add tests (#4085) 2018-10-26 22:05:22 +11:00
Richard van der Hoff
6cb2e2448a
Merge pull request #4089 from dekonnection/master 
Make Docker image listen on ipv6 as well as ipv4
 2018-10-25 19:23:37 +01:00
Richard van der Hoff
68c0ce62d8 changelog 2018-10-25 19:18:25 +01:00
Cédric Laudrel
379376e5e6 Make Docker image listening on ipv6 as well as ipv4 
Signed-off-by: Cédric Laudrel <dek@iono.me>
 2018-10-25 20:03:47 +02:00
Erik Johnston
cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011) 
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
 2018-10-25 17:49:55 +01:00
Richard van der Hoff
e5da60d75d Merge remote-tracking branch 'origin/master' into develop 2018-10-25 17:08:09 +01:00
Erik Johnston
c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list 
Add config option to control alias creation
 2018-10-25 17:04:59 +01:00
Neil Johnson
95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init 
fix race condiftion in calling initialise_reserved_users
 2018-10-25 16:33:40 +01:00
Erik Johnston
b94a43d5b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list 2018-10-25 15:25:31 +01:00
Neil Johnson
f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Travis Ralston
54def42c19 Merge branch 'develop' into travis/login-terms 2018-10-24 13:22:59 -06:00
Neil Johnson
ea69a84bbb fix style inconsistencies 2018-10-24 17:18:08 +01:00
Neil Johnson
07126e43a4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init 2018-10-24 16:25:39 +01:00
Richard van der Hoff
9f72c209ee
Update changelog.d/3975.feature 
Co-Authored-By: neilisfragile <neil@matrix.org>
 2018-10-24 14:37:36 +01:00
Richard van der Hoff
3ad359e5be Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation 2018-10-24 11:23:49 +01:00
Richard van der Hoff
7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8 
Fix a number of flake8 errors
 2018-10-24 11:23:35 +01:00
Erik Johnston
3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase 
Add v2 state resolution algorithm
 2018-10-24 11:12:12 +01:00
Richard van der Hoff
7e07d25ed6 Allow backslashes in event field filters 
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.

The intention here is to allow single-backslashes, but disallow
double-backslashes.
 2018-10-24 11:11:24 +01:00
Richard van der Hoff
ef771cc4c2 Fix a number of flake8 errors 
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
 2018-10-24 10:39:03 +01:00
Richard van der Hoff
e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts 
Clean up the way logcontexts and threads work in the pushers
 2018-10-24 09:53:57 +01:00
Erik Johnston
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation 
Clean up room alias creation
 2018-10-24 09:40:27 +01:00
Neil Johnson
a67d8ace9b remove errant exception and style 2018-10-23 17:44:39 +01:00
Travis Ralston
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests 
Handle HttpResponseException more safely for federated groups
 2018-10-23 10:41:04 -06:00
Neil Johnson
329d18b39c remove white space 2018-10-23 15:27:20 +01:00
Neil Johnson
6105c6101f fix race condiftion in calling initialise_reserved_users 2018-10-23 15:24:58 +01:00
Richard van der Hoff
b3f6dddad2
Give some more things logcontexts (#4077) 2018-10-23 14:29:17 +01:00
Richard van der Hoff
5c445114d3
Correctly account for cpu usage by background threads (#4074) 
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
 2018-10-23 13:12:32 +01:00
Richard van der Hoff
1fe6bbb555
Merge pull request #3698 from spantaleev/add-matrix-docker-ansible-deploy 
Add information about the matrix-docker-ansible-deploy playbook
 2018-10-23 10:11:58 +01:00
Richard van der Hoff
abd9914683 Changelog 2018-10-22 16:12:11 +01:00
Richard van der Hoff
7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts 
Run MAU queries as background processes
 2018-10-22 14:46:59 +01:00
Richard van der Hoff
911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil 
Make psutil an explicit dependency
 2018-10-22 12:33:21 +01:00
Richard van der Hoff
a6f421e812 Run MAU queries as background processes 
Fixes #3820
 2018-10-20 02:14:35 +01:00
Amber Brown
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
steamport
b85fe45f46
Add CL 2018-10-19 21:55:38 +00:00
Richard van der Hoff
e5b52d0f94 Make psutil an explicit dependency 
As of #4027, we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992.

Fixes #4062.
 2018-10-19 21:51:15 +01:00
Richard van der Hoff
f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065 
Calculate absolute path for email templates
 2018-10-19 16:33:54 +01:00
Erik Johnston
056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
Richard van der Hoff
cc325c7069 Calculate absolute path for email templates 2018-10-19 14:01:59 +01:00
Amber Brown
e404ba9aac
Fix manhole on py3 (pt 2) (#4067) 2018-10-19 22:26:00 +11:00
Amber Brown
b69216f768
Make the metrics less racy (#4061) 2018-10-19 21:45:45 +11:00
Erik Johnston
f9d6c677ea Newsfile 2018-10-19 10:22:45 +01:00
Erik Johnston
0d31109ed5 Newsfile 2018-10-19 10:14:29 +01:00
Amber Brown
1d17fc52ae changelog 2018-10-19 09:27:10 +11:00
Richard van der Hoff
c69026a758 Use the right python when starting workers 
We should use the same python to start the workers as we do for the main
synapse (ie, the same one used to run synctl.)
 2018-10-18 21:06:30 +01:00
Travis Ralston
49a044aa5f Merge branch 'develop' into travis/login-terms 2018-10-18 09:57:58 -06:00
Richard van der Hoff
03287c350e remove redundant changelog file 
this change has been released
 2018-10-18 15:09:34 +01:00
Richard van der Hoff
c632bc8654 Merge branch 'master' into develop 2018-10-18 15:07:03 +01:00
Erik Johnston
e77f24d80a
Merge pull request #4049 from matrix-org/erikj/synctl_colour 
Only colourise synctl output when attached to tty
 2018-10-18 10:51:14 +01:00
Richard van der Hoff
52e3d3813b prep changelog 2018-10-17 17:40:01 +01:00
Richard van der Hoff
0fd2321629 Fix incorrect truncation in get_missing_events 
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
 2018-10-17 17:35:26 +01:00
Richard van der Hoff
1519572961 Ship the email templates as package_data 
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.

In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
 2018-10-17 16:46:02 +01:00
Erik Johnston
3a5d8d5891 Newsfile 2018-10-17 16:12:45 +01:00
Erik Johnston
1af16acd4c Newsfile 2018-10-17 13:48:09 +01:00
Will Hunt
d6a7797dd1 Fix roomlist since tokens on Python 3 (#4046) 
Thanks @Half-Shot !!!
 2018-10-17 23:04:55 +11:00
Richard van der Hoff
6ec9d8ba0a
Merge pull request #4045 from matrix-org/rav/fix_get_missing_events 
Fix incorrect truncation in get_missing_events
 2018-10-17 11:40:03 +01:00
Neil Johnson
c6584f4b5f clean up config error logic and imports 2018-10-17 11:36:41 +01:00
Richard van der Hoff
fc0f13dd03 Fix incorrect truncation in get_missing_events 
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
 2018-10-16 21:10:04 +01:00
Richard van der Hoff
017eb9d17a changelog 2018-10-16 16:31:47 +01:00
Erik Johnston
fc954960e9 Newsfile 2018-10-16 16:28:42 +01:00
Richard van der Hoff
b8a5b0097c
Various cleanups in the federation client code (#4031) 
- Improve logging: log things in the right order, include destination and txids
  in all log lines, don't log successful responses twice

- Fix the docstring on TransportLayerClient.send_transaction

- Don't use treq.request, which is overcomplicated for our purposes: just use a
  twisted.web.client.Agent.

- simplify the logic for setting up the bodyProducer

- fix bytes/str confusions
 2018-10-16 10:44:49 +01:00
Travis Ralston
f293d124b6 Merge branch 'develop' into travis/login-terms 2018-10-15 14:44:32 -06:00
Amber Brown
24bc15eab4 update changelog 2018-10-15 22:23:13 +11:00
Amber Brown
4e50fe3edb update changelog 2018-10-15 22:22:49 +11:00
David Baker
03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups 
E2E backups
 2018-10-15 10:19:43 +01:00
Slavi Pantaleev
c187638ee9 Add information about the matrix-docker-ansible-deploy playbook 
Signed-off-by: Slavi Pantaleev <slavi@devture.com>
 2018-10-14 21:50:18 +03:00
Ivan Shapovalov
06bc8d2fe5 synapse/app: frontend_proxy.py: actually make workers work on py3 2018-10-14 20:08:39 +03:00
Ivan Shapovalov
fb216a22db synapse/visibility.py: fix SyntaxError on py3.7 2018-10-14 20:08:17 +03:00
Travis Ralston
7ede650956 Merge branch 'develop' into travis/login-terms 2018-10-12 16:24:07 -06:00
Travis Ralston
e3586f7c06 Merge branch 'develop' into travis/fix-federated-group-requests 2018-10-12 14:49:58 -06:00
Neil Johnson
a2bfb778c8 improve auto room join logic, comments and tests 2018-10-12 18:17:36 +01:00
Amber Brown
381d2cfdf0
Make workers work on Py3 (#4027) 2018-10-13 00:14:08 +11:00
Erik Johnston
49840f5ab2 Update newsfile 2018-10-10 11:31:02 +01:00
Erik Johnston
20733857ab Newsfile 2018-10-09 14:18:16 +01:00
David Baker
d34657e1f2 Add changelog 2018-10-09 11:15:54 +01:00
Erik Johnston
9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions 
Log looping call exceptions
 2018-10-09 10:12:10 +01:00
Richard van der Hoff
495975e231 Optimisation for filter_events_for_server 
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.

Also, iteritems > items.
 2018-10-08 13:46:52 +01:00
Erik Johnston
8164f6daf3 Newsfile 2018-10-05 11:25:09 +01:00
Neil Johnson
ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin 2018-10-04 17:26:59 +01:00
Richard van der Hoff
c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers 
Fix bug in forward_extremity update logic
 2018-10-04 16:27:05 +01:00
Erik Johnston
d9f3db5081 Newsfile 2018-10-04 16:03:08 +01:00
Amber Brown
d86794325f Merge branch 'master' into develop 2018-10-04 22:41:52 +10:00
Amber Brown
92faeb2a3f changelog 2018-10-04 22:38:10 +10:00
Travis Ralston
158d6c75b6 Changelog 2018-10-03 17:54:08 -06:00
Richard van der Hoff
a59d899668 Pin to prometheus_client<0.4 to avoid renaming all of our metrics 2018-10-03 17:20:15 +01:00
Erik Johnston
8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion 
Fix handling of rejected threepid invites
 2018-10-03 14:47:41 +01:00
Erik Johnston
81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration 
Fix exception in background metrics collection
 2018-10-03 14:14:38 +01:00
Erik Johnston
93a8603904 Newsfile 2018-10-03 11:59:05 +01:00
Erik Johnston
6e0c66f651 Newsfile 2018-10-03 11:40:02 +01:00
Erik Johnston
c69faf8c4a Newsfile 2018-10-03 11:29:44 +01:00
Richard van der Hoff
2a4ea3baa8 fix newsfile name 2018-10-03 07:09:25 +01:00
Richard van der Hoff
ae61ade891 Fix bug in forward_extremity update logic 
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
 2018-10-02 23:36:08 +01:00
Amber Brown
ccce6f26a6 changelog 2018-10-03 02:16:21 +10:00
Erik Johnston
25baf3b2ac
Merge pull request #3991 from matrix-org/erikj/fix_pop_retry_cache 
Fix bug when invalidating destination retry timings
 2018-10-02 16:42:28 +01:00
Amber Brown
058a4c665e
Remove Jenkins & other old dev junk (#3988) 2018-10-03 00:59:11 +10:00
Erik Johnston
8f614f842d Newsfile 2018-10-02 15:54:31 +01:00
Richard van der Hoff
2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error 
Fix error handling for missing auth_event
 2018-10-02 13:57:54 +01:00
Amber Brown
7232917f12
Disable frozen dicts by default (#3987) 2018-10-02 22:53:47 +10:00
Erik Johnston
eeb755d17b Newsfile 2018-10-02 12:06:25 +01:00
Richard van der Hoff
8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces 
Avoid reraise, to improve stacktraces
 2018-10-02 10:47:38 +01:00
Erik Johnston
bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state 
Fix lazy loaded sync with rejected state events
 2018-10-02 11:38:35 +02:00
Richard van der Hoff
8174c6725b Avoid reraise, to improve stacktraces 2018-10-01 18:50:34 +01:00
Richard van der Hoff
5908a8f6f5 Merge branch 'develop' into rav/fix_missing_create_event_error 2018-10-01 17:16:20 +01:00
Amber Brown
abdc141c2b
Update instructions to point to pip install (#3985) 2018-10-02 01:08:38 +10:00
Richard van der Hoff
b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors 
Fix exceptions when handling incoming transactions
 2018-10-01 15:54:24 +01:00
Amber Brown
6e05fd032c
Fix userconsent on Python 3 (#3938) 2018-10-02 00:11:58 +10:00
Erik Johnston
5fa27eac78 Newsfile 2018-10-01 14:24:41 +01:00
Erik Johnston
8f5c23d0cd
Merge pull request #3933 from matrix-org/erikj/destination_retry_cache 
Add a five minute cache to get_destination_retry_timings
 2018-10-01 14:20:30 +02:00
Richard van der Hoff
53c5fa4e6c
Further reduce the size of the docker image (#3972) 
Rewrite the dockerfile as a multistage build: this means we can get rid of a whole load of cruft which we don't need.
 2018-10-01 12:29:17 +01:00
Matthew Hodgson
faa462ef79 changelog 2018-09-29 02:21:01 +01:00
Bruno Windels
9a8bbc9a59 add --no-admin flag to registration script (#3836) 2018-09-28 13:36:56 +01:00
Richard van der Hoff
3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks 
remove spurious federation checks on localhost
 2018-09-28 13:35:47 +01:00
Richard van der Hoff
26557c9db4
Merge pull request #3980 from matrix-org/rav/remove_broken_cache_call 
Remove redundant call to start_get_pdu_cache
 2018-09-28 13:13:19 +01:00
Jan Christian Grünhage
5304449738 build python 3 docker images on circle CI (#3976) 2018-09-28 12:36:35 +01:00
Richard van der Hoff
19475cf337 Remove redundant call to start_get_pdu_cache 
I think this got forgotten in #3932. We were getting away with it because it
was the last call in this function.
 2018-09-28 12:01:23 +01:00
Richard van der Hoff
9c8cec5dab Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-28 10:51:09 +01:00
Richard van der Hoff
f094f715cf Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors 2018-09-27 15:18:21 +01:00
Richard van der Hoff
36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing 
Improve performance of getting typing updates for replication
 2018-09-27 15:14:26 +01:00
Richard van der Hoff
b5c976347b
Merge pull request #3946 from matrix-org/michaelkaye/automate_docker_hub_upload 
Build and push docker image to hub automatically
 2018-09-27 15:07:41 +01:00
Richard van der Hoff
e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups 
Clarifications in FederationHandler
 2018-09-27 15:06:59 +01:00
Amber Brown
6de3884e5e
Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_process 
Run notify_app_services as a bg process
 2018-09-27 23:40:30 +10:00
Amber Brown
a512e637ac
Merge pull request #3970 from schnuffle/develop-py3 
Replaced all occurences of e.message with str(e)
 2018-09-27 23:39:45 +10:00
Amber Brown
b3064532d0
Run our oldest supported configuration in CI (#3952) 2018-09-27 23:21:54 +10:00
Amber Brown
861c063ebc
Update 3970.bugfix 2018-09-27 22:47:01 +10:00
Schnuffle
a873896096 Added changelog fragment 2018-09-27 14:01:44 +02:00
Richard van der Hoff
51d33d5178
Merge pull request #3961 from matrix-org/neilj/lock_mau_upserts 
fix #3854 MAU transaction errors
 2018-09-27 12:31:37 +01:00
Michael Kaye
74bbdd0412 Do the changelog.d dance 2018-09-27 12:01:43 +01:00
Richard van der Hoff
948a6d8776 changelog 2018-09-27 11:37:39 +01:00
Richard van der Hoff
ad8e137062 changelog 2018-09-27 11:31:51 +01:00
Richard van der Hoff
92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2 
Logging improvements
 2018-09-27 11:27:46 +01:00
Richard van der Hoff
4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions (#3959) 
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.

This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
 2018-09-27 11:25:34 +01:00
Richard van der Hoff
ae6ad4cf41
docstrings and unittests for storage.state (#3958) 
I spent ages trying to figure out how I was going mad...
 2018-09-27 11:22:25 +01:00
Travis Ralston
2a7b3439de Changelog 2018-09-26 13:51:34 -06:00
Richard van der Hoff
e70b4ce069 Logging improvements 
Some logging tweaks to help with debugging incoming federation transactions
 2018-09-26 17:36:14 +01:00
Richard van der Hoff
3c37c7e45b changelog 2018-09-26 17:01:30 +01:00
Richard van der Hoff
0c4a99ea2d changelog 2018-09-26 16:54:54 +01:00
Richard van der Hoff
ab59f3d8da changelog 2018-09-26 16:52:24 +01:00
Neil Johnson
d514608b5c towncrier 2018-09-26 16:19:53 +01:00
Amber Brown
d4e0861ff9
Reduce the load on our CI (#3957) 
* changelog

* reduce circleci config

* plus a handy script

* fix regex
 2018-09-27 00:23:21 +10:00
Richard van der Hoff
a5e70b31a1 changelog 2018-09-26 14:41:14 +01:00
Richard van der Hoff
4e8276a34a
Merge pull request #3956 from matrix-org/rav/fix_expiring_cache_len 
Fix ExpiringCache.__len__ to be accurate
 2018-09-26 13:10:13 +01:00
Richard van der Hoff
5b4028fa78 Merge branch 'rav/fix_expiring_cache_len' into erikj/destination_retry_cache 2018-09-26 12:55:53 +01:00
Amber Brown
66a1d57adb
Merge pull request #3948 from matrix-org/rav/no_symlink_synctl 
Move synctl into top dir to avoid a symlink
 2018-09-26 21:41:58 +10:00
Amber Brown
c2185f14d7
Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pdu 
Comments and interface cleanup for on_receive_pdu
 2018-09-26 21:41:26 +10:00
Richard van der Hoff
8ac9fa7375 changelog 2018-09-26 12:35:10 +01:00
Richard van der Hoff
f65163627f
Merge pull request #3911 from matrix-org/jcgruenhage/docker-support-python3 
make python 3 work in the docker container
 2018-09-25 15:18:09 +01:00
Richard van der Hoff
a1cd37390f Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-25 12:03:54 +01:00
Richard van der Hoff
4c3e7eeec5
Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches 
Fix some instances of ExpiringCache not expiring cache items
 2018-09-25 12:02:57 +01:00
Jérémy Farnaud
6cf261930a added "media-src: 'self'" to CSP for resources (#3578) 
Synapse doesn’t allow for media resources to be played directly from
Chrome. It is a problem for users on other networks (e.g. IRC)
communicating with Matrix users through a gateway. The gateway sends
them the raw URL for the resource when a Matrix user uploads a video
and the video cannot be played directly in Chrome using that URL.

Chrome argues it is not authorized to play the video because of the
Content Security Policy. Chrome checks for the "media-src" policy which
is missing, and defauts to the "default-src" policy which is "none".

As Synapse already sends "object-src: 'self'" I thought it wouldn’t be
a problem to add "media-src: 'self'" to the CSP to fix this problem.
 2018-09-25 11:55:02 +01:00
Richard van der Hoff
94f7befc31
Merge pull request #3925 from matrix-org/erikj/fix_producers_unregistered 
Fix spurious exceptions when client closes conncetion
 2018-09-25 11:52:06 +01:00
Richard van der Hoff
bd469adaa9 changelog 2018-09-25 11:22:17 +01:00
Richard van der Hoff
45a1053d44 changelog 2018-09-25 10:45:34 +01:00
Matthew Hodgson
787d22ed6c
Only lazy load self-members on initial sync 
Given we have disabled lazy loading for incr syncs in #3840, we can make self-LL more efficient by only doing it on initial sync.  Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
 2018-09-25 00:49:26 +01:00
Amber Brown
04eed80a73 Merge branch 'master' into develop 2018-09-24 23:42:25 +10:00
Erik Johnston
5230bc1471 Newsfile 2018-09-21 15:05:37 +01:00
Erik Johnston
d3f80cbc9c Newsfile 2018-09-21 14:24:39 +01:00
Erik Johnston
ad53a5497d
Merge pull request #3927 from matrix-org/erikj/handle_background_errors 
Handle exceptions thrown by background tasks
 2018-09-21 09:26:30 +01:00
Matthew Hodgson
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients (#3916) 
Should fix https://github.com/vector-im/riot-web/issues/7209
 2018-09-20 21:21:54 +01:00
Erik Johnston
168491c412 Newsfile 2018-09-20 16:16:52 +01:00
Jan Christian Grünhage
3af7a95a9d add changelog 2018-09-20 14:55:11 +02:00
Erik Johnston
13f6f1624b Newsfile 2018-09-20 13:52:09 +01:00
Neil Johnson
23b53b4ef8
Merge pull request #3868 from matrix-org/neilj/fix_room_invite_mail_links 
Neilj/fix room invite mail links
 2018-09-20 13:32:38 +01:00
Richard van der Hoff
6bb9a4b1d6 changelog 2018-09-20 13:15:20 +01:00
Amber Brown
1f3f5fcf52
Fix client IPs being broken on Python 3 (#3908) 2018-09-20 20:14:34 +10:00
Erik Johnston
3fd68d533b
Merge pull request #3914 from matrix-org/erikj/remove_retry_cache 
Remove get_destination_retry_timings cache
 2018-09-20 10:54:49 +01:00
Amber Brown
741571cf22
Add a way to run tests in PostgreSQL in Docker (#3699) 2018-09-20 18:12:45 +10:00
Amber Brown
aeca5a5ed5
Add a regression test for logging on failed connections (#3912) 2018-09-20 16:28:18 +10:00
Richard van der Hoff
642199570c
Improve the logging when handling a federation transaction (#3904) 
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:

- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
  them into other, more cryptic, exceptions.
 2018-09-19 17:28:18 +01:00
Erik Johnston
ce846bb620 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typing 2018-09-19 15:08:36 +01:00
Erik Johnston
1a24d4effa Newsfile 2018-09-19 15:03:05 +01:00
Erik Johnston
bbab6ebfd9 Fix up changelog and remove spurious comment 2018-09-19 14:45:14 +01:00
Erik Johnston
83ee5592a5 Newsfile 2018-09-19 14:22:59 +01:00
Erik Johnston
80d2d50f47 Fixup 2018-09-19 11:19:47 +01:00
Erik Johnston
6bbe3d5732 Newsfile 2018-09-19 10:43:13 +01:00
Richard van der Hoff
05b9937cd7 update changelog for #3909 2018-09-19 09:17:54 +01:00
Amber Brown
47c02e6332
Merge pull request #3909 from turt2live/travis/fix-logging-1 
Fix matrixfederationclient.py logging: Destination is a string
 2018-09-19 18:14:47 +10:00
Amber Brown
3f0d8e6b09
Remove documentation referencing Cygwin (#3873) 2018-09-19 18:14:30 +10:00
Amber Brown
3d6b24fb1b
Merge pull request #3907 from matrix-org/rav/set_sni_to_server_name 
Set SNI to the server_name, not whatever was in the SRV record
 2018-09-19 17:59:33 +10:00
Amber Brown
f773ecbd61
Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeout 
Bump timeout on get_missing_events request
 2018-09-19 17:57:48 +10:00
Travis Ralston
13b31a9baf Changelog 2018-09-18 15:30:25 -06:00
Richard van der Hoff
38ead946a9 Merge remote-tracking branch 'origin/develop' into neilj/fix_room_invite_mail_links 2018-09-18 19:02:45 +01:00
Richard van der Hoff
a219ce8726
Use directory server for room joins (#3899) 
When we do a join, always try the server we used for the alias lookup first.

Fixes #2418
 2018-09-18 18:27:37 +01:00
Richard van der Hoff
31c15dcb80
Refactor matrixfederationclient to fix logging (#3906) 
We want to wait until we have read the response body before we log the request
as complete, otherwise a confusing thing happens where the request appears to
have completed, but we later fail it.

To do this, we factor the salient details of a request out to a separate
object, which can then keep track of the txn_id, so that it can be logged.
 2018-09-18 18:17:15 +01:00
Amber Brown
c600886d47
Merge pull request #3894 from matrix-org/hs/phone_home_py_version 
Add python_version phone home stat
 2018-09-19 02:40:04 +10:00
Richard van der Hoff
edabc18938 changelog 2018-09-18 17:04:20 +01:00
Richard van der Hoff
8565d9a9fe changelog 2018-09-18 15:05:26 +01:00
Richard van der Hoff
286d6930b7
Merge pull request #3879 from matrix-org/matthew/fix-autojoin 
don't ratelimit autojoins
 2018-09-18 13:07:01 +01:00
Richard van der Hoff
892432c818
Merge pull request #3882 from SimmyD/max_upload_docker_var 
Add variable for changing the max upload size in Docker container
 2018-09-18 13:05:09 +01:00
Richard van der Hoff
1e09a1d48a
Merge pull request #3889 from matrix-org/rav/404_on_remove_unknown_alias 
Return a 404 when deleting unknown room alias
 2018-09-18 12:59:30 +01:00
Richard van der Hoff
ad95ec12ca
Merge pull request #3895 from matrix-org/rav/decode_bytes_in_metrics 
Fix more b'abcd' noise in metrics
 2018-09-18 12:58:49 +01:00
Aaron Raimist
505abb38f0
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-09-17 22:07:19 -05:00
Will Hunt
2b41f2ed76
Create 3894.feature 2018-09-17 17:48:48 +01:00
Richard van der Hoff
06f2dbbb5d changelog 2018-09-17 17:18:26 +01:00
Richard van der Hoff
0cb7afff35 changelog 2018-09-17 16:17:25 +01:00
Richard van der Hoff
c1ae6b1bce changelog 2018-09-17 13:21:08 +01:00
Richard van der Hoff
2a8996b67d changelog 2018-09-17 11:51:38 +01:00
Simon Dwyer
0e46ff6904 Adding the ability to change MAX_UPLOAD_SIZE for the docker container variables. 
Signed-off-by: Simon Dwyer <simon@thedwyers.co>
 2018-09-16 13:33:33 +10:00
Matthew Hodgson
c71b93f2a4 changelog 2018-09-15 22:28:28 +01:00
Matthew Hodgson
9d13ff4da8 missing changelog 2018-09-15 21:04:10 +01:00
Erik Johnston
335b23a078 Newsfile 2018-09-14 19:25:58 +01:00
Matthew Hodgson
024be6cf18
don't filter membership events based on history visibility (#3874) 
don't filter membership events based on history visibility
as we will already have filtered the messages in the timeline, and state events
are always visible.

and because @erikjohnston said so.
 2018-09-14 18:12:52 +01:00
Erik Johnston
3e6e94fe9f
Merge pull request #3872 from matrix-org/hawkowl/timeouts-2 
timeouts 2: electric boogaloo
 2018-09-14 16:58:44 +01:00
Amber Brown
90f8e606e2 changelog 2018-09-15 00:23:55 +10:00
Erik Johnston
941ac0f085 Newsfile 2018-09-14 15:08:37 +01:00
Neil Johnson
bef1d6f3bf towncrier 2018-09-13 22:53:35 +01:00
Travis Ralston
984db8bb08
Create 3860.misc 2018-09-13 12:06:04 -06:00
Amber Brown
7c27c4d51c
merge (#3576) 2018-09-14 03:11:11 +10:00
Erik Johnston
6c0f8d9d50
Merge pull request #3856 from matrix-org/erikj/speed_up_purge 
Make purge history slightly faster
 2018-09-13 16:14:46 +01:00
Amber Brown
cb64fe2cb7
Merge pull request #3859 from matrix-org/erikj/add_iterkeys 
Fix handling of redacted events from federation
 2018-09-14 01:06:44 +10:00
Erik Johnston
13193a6e2b Newsfile 2018-09-13 15:46:45 +01:00
Amber Brown
3126b88d35
fix circleci merged builds (#3858) 
* fix

* changelog
 2018-09-14 00:44:31 +10:00
Amber Brown
bfa0b759e0
Attempt to figure out what's going on with timeouts (#3857) 2018-09-14 00:15:51 +10:00
Erik Johnston
93139a1fb8 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_purge 2018-09-13 12:57:09 +01:00
Erik Johnston
e7cd7cb0f0 Newsfile 2018-09-13 12:55:40 +01:00
Amber Brown
b7d2fb5eb9
Remove some superfluous logging (#3855) 2018-09-13 19:59:32 +10:00
Neil Johnson
f30a303590
Merge pull request #3846 from matrix-org/neilj/expose-registered-users 
expose number of real reserved users
 2018-09-12 17:14:04 +01:00
Matthew Hodgson
2ac1abbc7e
show heroes if a room has a 'deleted' name/canonical_alias (#3851) 2018-09-12 17:11:05 +01:00
Erik Johnston
fa0d464fa4
Merge pull request #3853 from matrix-org/erikj/log_outbound_each_time 
Log outbound requests when we retry
 2018-09-12 16:55:40 +01:00
Erik Johnston
3db016b641 Newsfile 2018-09-12 16:25:18 +01:00
Neil Johnson
8decd6233d improve naming 2018-09-12 16:22:15 +01:00
Erik Johnston
54f8616d2c
Merge pull request #3841 from matrix-org/erikj/manhole_key_length 
Change the manhole SSH key to have more bits
 2018-09-12 14:33:39 +01:00
Amber Brown
65cd8ccc79
Add JUnit summaries to CircleCI as well as merged runs (#3704) 2018-09-12 23:29:21 +10:00
Amber Brown
7ca097f77e
Port federation/ to py3 (#3847) 2018-09-12 23:23:32 +10:00
Neil Johnson
5cea4e16c7 towncrier 2018-09-12 12:03:31 +01:00
Amber Brown
546aee7e52
Merge pull request #3835 from krombel/fix_3821 
fix VOIP crashes under Python 3
 2018-09-12 20:44:18 +10:00
Amber Brown
33716c4aea
Merge pull request #3826 from matrix-org/rav/logging_for_keyring 
add some logging for the keyring queue
 2018-09-12 20:43:47 +10:00
Amber Brown
bc635026c5
Merge pull request #3824 from matrix-org/rav/fix_jwt_import 
Fix jwt import check
 2018-09-12 20:41:57 +10:00
Amber Brown
02aa41809b
Port rest/ to Python 3 (#3823) 2018-09-12 20:41:31 +10:00
Amber Brown
8fd93b5eea
Port crypto/ to Python 3 (#3822) 2018-09-12 20:16:31 +10:00
Erik Johnston
649c647955 Newsfile 2018-09-12 11:03:42 +01:00
Matthew Hodgson
b041115415
Speed up lazy loading (#3827) 
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats  (#3840)
 2018-09-12 00:50:39 +01:00
Luca Weiss
f8825748dd
changelog.d entry somehow got lost 2018-09-11 12:56:31 +02:00
Erik Johnston
9a68778ac2 Newsfile 2018-09-11 10:44:40 +01:00
Mathijs van Gorcum
e957428a15
Newsfile 2018-09-10 14:53:05 +00:00
Krombel
3572a206d3 add changelog 2018-09-10 14:33:08 +02:00
Richard van der Hoff
1e4c7fff5f changelog 2018-09-07 16:37:30 +01:00
Amber Brown
9a5ea511b5
Merge pull request #3810 from matrix-org/erikj/send_tags_down_sync_on_join 
Send existing room tags down sync on join
 2018-09-07 23:28:42 +10:00
Richard van der Hoff
edda9f5cac changelog 2018-09-07 14:23:35 +01:00
Amber Brown
771d213ac5 Merge branch 'master' into develop 2018-09-07 21:45:38 +10:00
Amber Brown
b60749a1ec changelog 2018-09-07 21:41:57 +10:00
Amber Brown
52ec6e9dfa
Port tests/ to Python 3 (#3808) 2018-09-07 02:58:18 +10:00
Neil Johnson
c5440b2ca0
Merge pull request #3800 from matrix-org/neilj/remove-guests-from-mau-count 
guest users should not be part of mau total
 2018-09-06 17:45:55 +01:00
Erik Johnston
7298efd361 Newsfile 2018-09-06 17:13:33 +01:00
Amber Brown
654324eded
Merge pull request #3805 from matrix-org/erikj/limit_transaction_pdus_edus 
Limit the number of PDUs/EDUs per fedreation transaction
 2018-09-07 01:33:31 +10:00
Amber Brown
1d371fc5b3
Merge pull request #3806 from matrix-org/erikj/limit_postgres_travis 
Only start postgres instance for postgres tests on Travis CI
 2018-09-07 01:06:21 +10:00
Erik Johnston
b07a2cbee9 Spelling 2018-09-06 15:53:02 +01:00
Amber Brown
70fd75cd1d
Merge pull request #3788 from matrix-org/erikj/remove_conn_id 
Remove conn_id from repl prometheus metrics
 2018-09-07 00:48:19 +10:00
Erik Johnston
5d848992bf Newsfile 2018-09-06 15:28:46 +01:00
Erik Johnston
28f5bfdcf7 Newsfile 2018-09-06 15:25:58 +01:00
Amber Brown
ee7c8bd2b5
Merge pull request #3795 from matrix-org/erikj/faster_sync_state 
User iter* during sync state calculations
 2018-09-07 00:24:43 +10:00
Amber Brown
135f3b4390
Merge pull request #3804 from matrix-org/rav/fix_openssl_dep 
bump dep on pyopenssl to 16.x
 2018-09-07 00:23:39 +10:00
Amber Brown
2608ebc04c
Port handlers/ to Python 3 (#3803) 2018-09-07 00:22:23 +10:00
Richard van der Hoff
b3c2ebba32 changelog 2018-09-06 12:55:17 +01:00
Jan Christian Grünhage
af3125226d
Create 3802.misc 2018-09-06 10:46:00 +02:00
Neil Johnson
92657be7d0 towncrier 2018-09-05 22:33:45 +01:00
Richard van der Hoff
dfba1d843d
Merge pull request #3790 from matrix-org/rav/respect_event_format_in_filter 
Implement 'event_format' filter param in /sync
 2018-09-05 16:24:14 +01:00
Erik Johnston
2254790ae4 Newsfile 2018-09-05 16:21:18 +01:00
Amber Brown
2d2828dcbc
Port http/ to Python 3 (#3771) 2018-09-06 00:10:47 +10:00
Richard van der Hoff
c91bd295f5 changelog 2018-09-04 15:23:19 +01:00
Neil Johnson
a6cf7d9d9a
Merge pull request #3789 from matrix-org/neilj/improve_threepid_error_strings 
improve human readable error messages
 2018-09-04 13:16:00 +00:00
Amber Brown
7e9ced4178 version and towncrier 2018-09-04 21:12:04 +10:00
Neil Johnson
70fc599ede towncrier 2018-09-04 12:08:27 +01:00
Erik Johnston
87b111f96a Newsfile 2018-09-03 17:26:15 +01:00
Amber Brown
77055dba92
Fix tests on postgresql (#3740) 2018-09-04 02:21:48 +10:00
Erik Johnston
567363e497
Merge pull request #3737 from matrix-org/erikj/remove_redundant_state_func 
Remove unnecessary resolve_events_with_state_map
 2018-09-03 16:19:41 +01:00
Erik Johnston
06ee2b7cc5 Newsfile 2018-09-03 15:43:01 +01:00
Amber Brown
905f8de673
Create 3378.misc 2018-09-03 22:17:06 +10:00
Neil Johnson
a796bdd35e typo 2018-08-31 15:57:33 +01:00
Neil Johnson
3d6aa06577 news fragemnt 2018-08-31 10:56:37 +01:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 (#3725) 2018-08-31 00:19:58 +10:00
Richard van der Hoff
475253a88e
Merge pull request #3764 from matrix-org/rav/close_db_conn_after_init 
Make sure that we close db connections opened during init
 2018-08-30 10:47:27 +01:00
Krombel
8c0c51ecb3 changelog 2018-08-29 16:49:26 +02:00
Krombel
79a8a347a6 fix #3445 
itervalues(d) calls d.itervalues() [PY2] and d.values() [PY3]
but SortedDict only implements d.values()
 2018-08-29 16:28:25 +02:00
Amber Brown
82276a18d1
Update CONTRIBUTING to clarify miscs & Markdown (#3730) 2018-08-29 19:06:59 +10:00
Matthew Hodgson
b1580f50fe
don't return non-LL-member state in incremental sync state blocks (#3760) 
don't return non-LL-member state in incremental sync state blocks
 2018-08-28 23:25:58 +01:00
Richard van der Hoff
71990b3cae changelog 2018-08-28 13:42:20 +01:00
Erik Johnston
9db2476991 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_contact 2018-08-24 17:00:37 +01:00
Erik Johnston
69f7f1418f Newsfile 2018-08-24 16:54:06 +01:00
Erik Johnston
08abe8e13c Newsfile 2018-08-24 16:52:52 +01:00
Erik Johnston
45fc0ead10 Newsfile 2018-08-24 15:04:29 +01:00
Erik Johnston
84b4e76fed
Merge pull request #3754 from matrix-org/erikj/fix_whitelist 
Allow federation_domain_whitelist to be emtpy list
 2018-08-24 12:23:39 +01:00
Erik Johnston
c780d84d66 Newsfile 2018-08-24 12:13:50 +01:00
Erik Johnston
92d50e3c2a
Merge pull request #3753 from matrix-org/erikj/fix_no_server_noticse 
Fix bug where we broke sync when using limit_usage_by_mau
 2018-08-24 11:56:08 +01:00
Richard van der Hoff
e94cdbaecf
Merge pull request #3751 from matrix-org/rav/twisted_17 
Pin to twisted 17.1 or later
 2018-08-24 11:55:52 +01:00
Erik Johnston
9b1bc593c5 Newsfile 2018-08-24 11:36:39 +01:00
Richard van der Hoff
05fe8a6c1c changelog 2018-08-24 10:04:52 +01:00
Erik Johnston
28d7b546cb Newsfile 2018-08-23 19:18:52 +01:00
Erik Johnston
60cf1c6e16 Newsfile 2018-08-23 16:34:32 +01:00
Travis Ralston
1ca2744621
Merge pull request #3734 from matrix-org/travis/worker-docs 
Reference that the federation_reader needs the HTTP replication port set
 2018-08-23 07:51:46 -06:00
Erik Johnston
4eb8408ed2 Newsfile 2018-08-23 10:46:13 +01:00
Erik Johnston
fd2dbf1836 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-22 17:06:10 +01:00
Richard van der Hoff
c7181dcc6c Merge branch 'master' into develop 2018-08-22 14:37:11 +01:00
Erik Johnston
db10f553ba
Merge pull request #3724 from Half-Shot/hs/guest-fetch-event 
Allow guests to use /rooms/:roomId/event/:eventId
 2018-08-22 13:41:08 +01:00
Erik Johnston
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles 
Allow profile updates to happen on workers
 2018-08-22 11:35:55 +01:00
Erik Johnston
3bf8bab8f9
Merge pull request #3673 from matrix-org/erikj/refactor_state_handler 
Refactor state module to support multiple room versions
 2018-08-22 10:04:55 +01:00
Richard van der Hoff
a4cf660a32
Merge pull request #3735 from matrix-org/travis/federation-spelling 
limt -> limit
 2018-08-22 09:34:21 +01:00
Richard van der Hoff
0d568ff403 Merge remote-tracking branch 'origin/release-v0.33.3' into develop 2018-08-22 09:15:44 +01:00
Travis Ralston
365f588d98
Create 3734.misc 2018-08-21 23:38:38 -06:00
Travis Ralston
eb7be75a10
Create 3735.misc 2018-08-21 23:38:06 -06:00
Matthew Hodgson
bb81e78ec6
Split the state_group_cache in two (#3726) 
Splits the state_group_cache in two.

One half contains normal state events; the other contains member events.

The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache.  This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching.

Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow.  Hopefully fixes https://github.com/matrix-org/synapse/issues/3720.
 2018-08-22 00:56:37 +02:00
Richard van der Hoff
afb4b490a4 changelog 2018-08-21 23:19:14 +01:00
Richard van der Hoff
a52f276990 Bugfixes 
--------
 
 - Fix bug in v0.33.3rc1 which caused infinite loops and OOMs
 ([\#3723](https://github.com/matrix-org/synapse/issues/3723))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEIQBQJ4l+yK4dlKkFIwi0edOSShEFAlt6/yMACgkQIwi0edOS
 ShFrzAgAhLA74fClydGLfhPNCXIw12Amv2ASA6+r0ukqjYQ/9MZ0CRsL+PMZJTk6
 Z75PK3TkX38zSpPaUx9HdMQO/yE+rpI99ZxlBM+8jzsf1jyF8ljqxSfixXUlaXX9
 698PQQ3f+IDoSs59NKrHvoWhUThaBpf/1GwRxG5mUcm2oKJItNMHk/uLurNCaIIZ
 YSwWcxvZdluOMX0WrlF/S524DqtgULcRf3E9x3mumobkVGxI3UH73hXgGUcPi7XV
 ttLuy7fVbV2Cun/wvjBqD7kc8LNbKwSaT9y+/xS0D5QYjph4rpQiTG7S5rtPLPMU
 Z2lL1C5ggBCkD43g3orDvehbNWCL5g==
 =0vEe
 -----END PGP SIGNATURE-----

Merge tag 'v0.33.3rc2' into develop

Bugfixes
--------

- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs
([\#3723](https://github.com/matrix-org/synapse/issues/3723))
 2018-08-21 20:30:09 +01:00
Erik Johnston
46c832eaac
Merge pull request #3727 from matrix-org/erikj/dont_error_on_missing_keys 
Don't log exceptions when failing to fetch server keys
 2018-08-21 17:07:20 +01:00
Erik Johnston
79d3b4689e Newsfile 2018-08-21 11:21:48 +01:00
Amber Brown
3b5b64ac99 changelog 2018-08-21 03:48:55 +10:00
Will Hunt
8bd585b09b 3724.feature 2018-08-20 18:27:42 +01:00
Richard van der Hoff
012d612f9d changelog 2018-08-20 18:26:27 +01:00
Erik Johnston
e2c0aa2c26 Newsfile 2018-08-20 17:40:59 +01:00
Amber Brown
80bf7d3580 changelog 2018-08-21 00:01:14 +10:00
Amber Brown
324525f40c
Port over enough to get some sytests running on Python 3 (#3668) 2018-08-20 23:54:49 +10:00
Erik Johnston
4d664278af Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler 2018-08-20 14:49:43 +01:00
Erik Johnston
48a910e128 Newsfile 2018-08-20 13:33:20 +01:00
Erik Johnston
2aa7cc6a46
Merge pull request #3713 from matrix-org/erikj/fixup_fed_logging 
Fix logging bug in EDU handling over replication
 2018-08-20 10:51:45 +01:00
Richard van der Hoff
3cef867cc1
Merge pull request #3709 from matrix-org/rav/logcontext_for_replication_commands 
Logcontexts for replication command handlers
 2018-08-17 16:22:07 +01:00
Richard van der Hoff
c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates 
Fix logcontexts for running pushers
 2018-08-17 16:21:49 +01:00
Amber Brown
c334ca67bb
Integrate presence from hotfixes (#3694) 2018-08-18 01:08:45 +10:00
Amber Brown
04f5d2db62
Remove v1/register's broken shared secret functionality (#3703) 2018-08-18 00:55:01 +10:00
Richard van der Hoff
63260397c6
Merge pull request #3701 from matrix-org/rav/use_producer_for_responses 
Use a producer to stream back responses
 2018-08-17 14:58:45 +01:00
Richard van der Hoff
3f8709ffe4
Merge pull request #3700 from matrix-org/rav/wait_for_producers 
Refactor request logging code
 2018-08-17 14:57:45 +01:00
Neil Johnson
3ee57bdcbb Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-17 14:34:10 +01:00
Erik Johnston
782689bd40 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles 2018-08-17 14:15:48 +01:00
Neil Johnson
b5f638f1f4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-17 14:04:15 +01:00
Neil Johnson
69c49d3fa3
Merge branch 'develop' into neilj/limit_exceeded_error 2018-08-17 12:44:26 +00:00
Neil Johnson
a2d872e7b3 Merge pull request #3708 from matrix-org/neilj/resource_Limit_block_event_creation 
Neilj/resource limit block event creation
 2018-08-17 12:42:59 +00:00
Erik Johnston
73737bd0f9 Newsfile 2018-08-17 11:14:05 +01:00
Travis Ralston
b99a0f3941
Create 3712.misc 2018-08-17 02:47:31 -06:00
Richard van der Hoff
4c9da1440f changelog 2018-08-17 00:50:36 +01:00
Richard van der Hoff
d9efd87d55 changelog 2018-08-17 00:49:22 +01:00
Neil Johnson
7e51342196 For resource limit blocked users, prevent writing into rooms 2018-08-16 23:05:20 +01:00
Neil Johnson
7edd11623d add new error type ResourceLimit 2018-08-16 21:04:30 +01:00
Erik Johnston
aae86a81ef Newsfile 2018-08-16 15:41:48 +01:00
Neil Johnson
bfb6c58624 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-16 15:10:16 +01:00
Will Hunt
c151b32b1d Add GET media/v1/config (#3184) 2018-08-16 14:23:38 +01:00
Matthew Hodgson
762a758fea lazyload aware /messages (#3589) 2018-08-16 14:22:47 +01:00
Neil Johnson
25d2b5d55f Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-16 11:37:17 +01:00
Matthew Hodgson
3f543dc021 initial cut at a room summary API (#3574) 2018-08-16 09:46:50 +01:00
Neil Johnson
8cfad2e686 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-15 17:20:38 +01:00
Neil Johnson
81d727efa9
Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus 
Fix Mau off by one errors
 2018-08-15 16:19:41 +00:00
Neil Johnson
da7fe43899 Fix mau blocking calulation bug on login 2018-08-15 17:03:30 +01:00
Richard van der Hoff
e2e846628d Remove incorrectly re-added changelog files 
These were removed for the 0.33.2 release, but were incorrectly re-added in
 2018-08-15 16:38:46 +01:00
Matthew Hodgson
2f78f432c4 speed up /members and add at= and membership params (#3568) 2018-08-15 16:35:22 +01:00
Neil Johnson
fc5d937550 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-15 16:31:40 +01:00
Richard van der Hoff
d82fa0e9a6 changelog 2018-08-15 15:12:35 +01:00
Erik Johnston
dc56c47dc0
Merge pull request #3653 from matrix-org/erikj/split_federation 
Move more federation APIs to workers
 2018-08-15 14:59:02 +01:00
Neil Johnson
4601129c44
Merge pull request #3687 from matrix-org/neilj/admin_email 
support admin_email config and pass through into blocking errors,
 2018-08-15 13:52:25 +00:00
Erik Johnston
ef184caf30 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation 2018-08-15 14:25:46 +01:00
Neil Johnson
39176f27f7 remove changelog referencing another PR 2018-08-15 13:53:51 +01:00
Richard van der Hoff
2de813a611 changelog 2018-08-15 13:49:03 +01:00
Neil Johnson
c4eb97518f Merge branch 'neilj/update_limits_error_codes' of github.com:matrix-org/synapse into neilj/admin_email 2018-08-15 11:53:01 +01:00
Neil Johnson
55afba0fc5 update admin email to uri 2018-08-15 11:41:18 +01:00
Neil Johnson
1c5e690a6b
Merge pull request #3690 from matrix-org/neilj/change_prometheus_mau_metric_name 
combine mau metrics into one group
 2018-08-15 09:53:59 +00:00
Erik Johnston
fef2e65d12
Merge pull request #3667 from matrix-org/erikj/fixup_unbind 
Don't fail requests to unbind 3pids for non supporting ID servers
 2018-08-15 10:32:12 +01:00
Neil Johnson
596ce63576 update resource limit error codes 2018-08-15 10:23:28 +01:00
Neil Johnson
19b433e3f4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email 2018-08-14 17:44:46 +01:00
Neil Johnson
70e48cbbb1 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/change_prometheus_mau_metric_name 2018-08-14 17:40:59 +01:00
Neil Johnson
cd0c749c4f Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users 2018-08-14 17:06:51 +01:00
Neil Johnson
629f390035 Rename MAU prometheus metric 2018-08-14 16:38:05 +01:00
Neil Johnson
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block 
Block ability to read via sync if mau limit exceeded
 2018-08-14 15:21:31 +00:00
Amber Brown
614e6d517d
Dockerised sytest (#3660) 2018-08-14 21:30:34 +10:00
Amber Brown
591bf87c6a Merge remote-tracking branch 'origin/develop' into neilj/fix_reap_users_in_postgres 2018-08-14 20:56:23 +10:00
Amber Brown
bdfbd934d6
Implement a new test baseclass to cut down on boilerplate (#3684) 2018-08-14 20:53:43 +10:00
Jan Christian Grünhage
8f0c430ca4
Merge pull request #3669 from matrix-org/jcgruenhage/update-docker-base 
update docker base-image to alpine 3.8
 2018-08-14 11:43:59 +02:00
Neil Johnson
1f24d8681b set admin email via config 2018-08-13 21:26:43 +01:00
Richard van der Hoff
8ba2dac6ea Name changelog after PR, not bug 2018-08-13 18:54:01 +01:00
Richard van der Hoff
50bcaf1a27 Merge pull request #3677 from andrewshadura/master 
Use recaptcha_ajax.js directly from Google
 2018-08-13 18:52:56 +01:00
Richard van der Hoff
34f51babc5
Revert "Use recaptcha_ajax.js directly from Google" 2018-08-13 18:48:03 +01:00
Amber Brown
99dd975dae
Run tests under PostgreSQL (#3423) 2018-08-13 16:47:46 +10:00
Neil Johnson
ac205a54b2 Fixes test_reap_monthly_active_users so it passes under postgres 2018-08-11 22:50:09 +01:00
Neil Johnson
c79c4c0a7d server notices for resource limit blocking 2018-08-10 15:25:47 +01:00
Amber Brown
a001038b92
Merge pull request #3679 from matrix-org/hawkowl/blackify-tests 
Blackify the tests
 2018-08-11 00:12:56 +10:00
Richard van der Hoff
c31793a784 Merge branch 'rav/fix_linearizer_cancellation' into develop 2018-08-10 14:57:27 +01:00
Amber Brown
dd16e7dfcc changelog 2018-08-10 23:54:46 +10:00
Amber Brown
b37c472419
Rename async to async_helpers because async is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Andrej Shadura
c75b71a397
Use recaptcha_ajax.js directly from Google 
The script recaptcha_ajax.js contains minified non-free code which
we probably cannot redistribute. Since it talks to Google servers
anyway, it is better to just download it from Google directly instead
of shipping it.

This fixes #1932.
 2018-08-10 13:36:14 +02:00
Richard van der Hoff
3c0213a217
Merge pull request #3439 from vojeroen/send_sni_for_federation_requests 
send SNI for federation requests
 2018-08-10 12:23:54 +01:00
Richard van der Hoff
178ab76ac0 changelog 2018-08-10 11:05:23 +01:00
Erik Johnston
5075e444f4 Newsfile 2018-08-09 14:58:49 +01:00
Neil Johnson
c6b28fb479 Where server is disabled, block ability for locked out users to read new messages 2018-08-09 12:45:58 +01:00
Jan Christian Grünhage
d967653705 update docker base-image to alpine 3.8 2018-08-09 13:31:10 +02:00
Neil Johnson
0ad98e38d0
Merge pull request #3655 from matrix-org/neilj/disable_hs 
Flag to disable HS without disabling federation
 2018-08-09 10:41:43 +00:00
Erik Johnston
54a9bea88c Newsfile 2018-08-09 10:39:29 +01:00
Erik Johnston
8876ce7f77 Newsfile 2018-08-09 10:37:42 +01:00
Amber Brown
984376745b Merge branch 'master' into develop 2018-08-09 19:23:47 +10:00
Erik Johnston
2bdafaf3c1
Merge pull request #3632 from matrix-org/erikj/refactor_repl_servlet 
Add helper base class for generating new replication endpoints
 2018-08-09 10:06:23 +01:00
Erik Johnston
62564797f5 Fixup wording and remove dead code 2018-08-09 09:56:10 +01:00
Amber Brown
2511f3f8a0
Test fixes for Python 3 (#3647) 2018-08-09 12:22:01 +10:00
Richard van der Hoff
bb89c84614
Merge pull request #3664 from matrix-org/rav/federation_metrics 
more metrics for the federation and appservice senders
 2018-08-08 23:16:58 +01:00
Neil Johnson
5298d79fb5
Merge branch 'develop' into neilj/disable_hs 2018-08-08 16:13:03 +00:00
Richard van der Hoff
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions 
Support for room versioning
 2018-08-08 17:10:53 +01:00
Neil Johnson
d2f3ef98ac
Merge branch 'develop' into neilj/disable_hs 2018-08-08 15:55:47 +00:00
Neil Johnson
54685d294d Ability to disable client/server Synapse via conf toggle 2018-08-08 15:38:54 +01:00
Neil Johnson
be59910b93 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/reserved_users 2018-08-08 13:45:21 +01:00
Neil Johnson
990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker 
API for monthly_active_users table
 2018-08-08 12:44:15 +00:00
Neil Johnson
d92675a486 Ability to whitelist specific threepids against monthly active user limiting 2018-08-08 12:06:42 +01:00
Erik Johnston
360ba89c50 Don't fail requests to unbind 3pids for non supporting ID servers 
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.

Fixes #3661
 2018-08-08 12:06:18 +01:00
Richard van der Hoff
e6d73b8582 changelog 2018-08-07 22:14:35 +01:00
Amber Brown
cf8ef11f35 changelog 2018-08-07 21:11:24 +10:00
Richard van der Hoff
865d07cd38 changelog 2018-08-07 10:02:01 +01:00