286 commits

Author	SHA1	Message	Date
Olivier 'reivilibre	843e048673	Merge branch 'develop' into anoa/configure_workers_updates	2024-09-26 17:45:27 +01:00
Olivier 'reivilibre	d5f42f4655	Use sticky routing for synchrotrons	2024-09-24 13:21:06 +01:00
Andrew Ferrazzutti	5173741c71	Support MSC4140: Delayed events (Futures) (#17326)	2024-09-23 13:33:48 +01:00
Olivier 'reivilibre	b21487300c	Add application_names to the postgres connections based on the worker names	2024-09-09 16:15:52 +01:00
Olivier 'reivilibre	5d2241e372	Configure sliding-sync workers with sticky load balancing	2024-09-02 16:27:02 +01:00
Till Faelligen	e64a35cb00	Route key queries to the client_reader	2024-08-30 12:01:31 +02:00
Olivier 'reivilibre	56bcba2b16	Add metrics listener config for main process	2024-08-29 11:48:22 +01:00
Olivier 'reivilibre	40a684e169	Gen metrics scraping configurations, for use with file_sd_configs	2024-08-28 18:58:02 +01:00
Till Faelligen	84880cbe94	Route createRoom to event creators, add sliding_sync worker type	2024-08-21 11:56:08 +02:00
Till Faelligen	50bb114032	Add metrics	2024-08-19 14:53:24 +02:00
Till Faelligen	e466404b4f	Fix paths, remove logging (for now), disable passwords	2024-08-15 14:22:50 +02:00
Till Faelligen	4dd02db657	Fix parameter ordering	2024-08-15 11:33:10 +02:00
Andrew Morgan	333f806d06	fixup: Add SYNAPSE_CONFIG_TEMPLATE_DIR env var	2024-08-14 19:00:57 +01:00
Andrew Morgan	550d2a1e77	start.py: Support SYNAPSE_CONFIG_TEMPLATE_DIR	2024-08-14 18:57:23 +01:00
Andrew Morgan	61349bd860	create data directory if it doesn't exist ... ideally this would be done in `main`	2024-08-14 18:52:47 +01:00
Andrew Morgan	20bd26ecad	Allow configuring data dir in start.py	2024-08-14 18:47:47 +01:00
Andrew Morgan	e91cd9c722	Allow running python from a venv	2024-08-14 17:27:36 +01:00
Andrew Morgan	18a331a18c	Document additional environment variables	2024-08-14 17:27:34 +01:00
Andrew Morgan	d7e3925a14	Allow specifying where generated config files are stored ... * Store generated worker config files in SYNAPSE_CONFIG_DIR, rather than hardcoding `/conf`. * Move the `workers_have_been_configured` filepath to the config dir.	2024-08-14 17:27:30 +01:00
Andrew Morgan	9df46343b9	Remove unused config_path dict entry	2024-08-14 17:27:27 +01:00
Andrew Morgan	a035d84ce0	Add SYNAPSE_CONFIG_TEMPLATE_DIR env var to worker config script ... This allows specifying a directory other than `/conf` where config templates should live. This allows this script to be used outside of the context of a docker container where /conf is bound to a dir on the host.	2024-08-14 17:27:24 +01:00
Joe Groocock	5871daf877	Use consistent casing between FROM and AS (#17431) ... Signed-off-by: Joe Groocock <me@frebib.net>	2024-07-11 13:56:25 +01:00
Andrew Morgan	4d6f7c0fc9	Route auth'd fed media requests to media repo in Complement tests (#17422)	2024-07-09 15:12:11 +00:00
Erik Johnston	99c107920d	Merge remote-tracking branch 'origin/release-v1.110' into develop	2024-07-02 15:12:36 +01:00
Shay	8f890447b0	Support MSC3916 by adding _matrix/client/v1/media/download endpoint (#17365)	2024-07-02 14:07:04 +01:00
Erik Johnston	9c8f1a6d41	Fix building debian packages on non-clean checkouts (#17390) ... If we leave the `.so` in place it causes the tests to fail, as it gets picked up (instead of the newly built .so) and so fails with mismatched GLIBC errors.	2024-07-02 12:39:32 +01:00
Erik Johnston	5b5280e3e5	Fix building debian packages for sid (#17389) ... Sid now defaults to python3.12, and our pinned version of cffi (1.5.1) does not have wheels for 3.12. This installing cffi to fail as we did not have the correct libs installed to build from source.	2024-07-02 12:38:29 +01:00
Aaron Dewes	7ef89b985d	Remove expire_access_token from Docker configuration (#17198) ... Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>	2024-06-19 10:58:06 +01:00
Richard van der Hoff	2c36a679ae	Include user membership on events (#17282) ... MSC4115 has now completed FCP, so we can enable it by default and switch to the stable identifier.	2024-06-13 21:45:54 +00:00
Andrew Morgan	7cb3f8a979	Route /make_knock and /send_knock to workers in Complement docker image (#17195)	2024-05-16 16:53:26 +00:00
Benjamin Bouvier	c0ea2bf800	synapse complement image: hardcode enabling msc3266 (#17105) ... This is an alternative to https://github.com/matrix-org/matrix-rust-sdk/issues/3340 where we don't need to change our CI setup too much in the Rust SDK repository, and still can test MSC3266.	2024-05-02 11:48:27 +01:00
Ben Banfield-Zanin	7254015665	Correct licensing metadata on the Docker image (#17141) ... ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [x] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [x] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))	2024-05-01 16:23:42 +01:00
Richard van der Hoff	b548f7803a	Add support for MSC4115 (#17104) ... Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-04-29 15:22:13 +01:00
Kegan Dougal	259442fa4c	bugfix: make msc3967 idempotent (#16943) ... MSC3967 was updated recently to make it more robust to network failures: > there is an existing cross-signing master key and it exactly matches the cross-signing master key provided in the request body. If there are any additional keys provided in the request (self signing key, user signing key) they MUST also match the existing keys stored on the server. In other words, the request contains no new keys. If there are new keys, UIA MUST be performed. https://github.com/matrix-org/matrix-spec-proposals/blob/hughns/device-signing-upload-uia/proposals/3967-device-signing-upload-uia.md#proposal This covers the case where the 200 OK is lost in transit so the client retries the upload, only to then get UIA'd. Complement tests: https://github.com/matrix-org/complement/pull/713 - passing example https://github.com/element-hq/synapse/actions/runs/7976948122/job/21778795094?pr=16943#step:7:8820 ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [x] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [x] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: reivilibre <oliverw@matrix.org>	2024-04-15 10:57:56 +00:00
Erik Johnston	fd48fc4585	Fixups to new push stream (#17038) ... Follow on from #17037	2024-03-28 16:29:23 +00:00
Sam Wedgwood	bef765b262	generate configuration with correct user in start.py for docker (#16978)	2024-03-21 17:55:44 +00:00
Andrew Morgan	21daa56ee1	Prevent start_for_complement.sh from setting START_POSTGRES to false when it's already set (#16985) ... I have a use case where I'd like the Synapse image to start up a postgres instance that I can use, but don't want to force Synapse to use postgres as well. This commit prevents postgres from being started when it has already been explicitly enabled elsewhere.	2024-03-21 13:50:51 +00:00
Richard van der Hoff	92f2069627	Multi-worker-docker-container: disable log buffering (#16919) ... Background: we have a `matrixdotorg/synapse-workers` docker image, which is intended for running multiple workers within the same container. That image includes a `prefix-log` script which, for each line printed to stdout or stderr by one of the processes, prepends the name of the process. This commit disables buffering in that script, so that lines are logged quickly after they are printed. This makes it much easier to understand the output, since they then come out in a natural order.	2024-03-13 17:21:37 +00:00
kegsay	0c55c76da8	Better complement docs (#16946)	2024-02-20 17:14:50 +00:00
Erik Johnston	23740eaa3d	Correctly mention previous copyright (#16820) ... During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.	2024-01-23 11:26:48 +00:00
reivilibre	fa2700f001	Add a --generate-only option to the Complement launcher. (#16828) ... Pulled out of #16803 since the drive-by cleanup was maybe not as drive-by as I had hoped. <!-- Fixes: # <!-- --> <!-- Supersedes: # <!-- --> <!-- Follows: # <!-- --> <!-- Part of: # <!-- --> Base: `develop` <!-- git-stack-base-branch:develop --> <!-- This pull request is commit-by-commit review friendly. <!-- --> <!-- This pull request is intended for commit-by-commit review. <!-- --> Original commit schedule, with full messages: <ol> <li> Add a --generate-only option </li> </ol> --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	2024-01-22 16:00:04 +00:00
Erik Johnston	930dc9e2d3	Fix typo	2023-12-13 16:37:10 +00:00
Erik Johnston	0455c40085	Update book location	2023-12-13 16:15:22 +00:00
Erik Johnston	8613f7693e	More renaming	2023-12-13 15:41:11 +00:00
Patrick Cloke	8e1e62c9e0	Update license headers	2023-11-21 15:29:58 -05:00
kegsay	11a8ae0632	complement: enable dirty runs (#16520) ... * complement: enable dirty runs * Add changelog * Set a low connpool limit when running in Complement Dirty runs can cause many containers to be running concurrently, which seems to easily exhaust resources on the host. The increased speedup from dirty runs also seems to use more db connections on workers, which are misconfigured currently to have `SUM(workers * cp_max) > max_connections`, causing ``` FATAL: sorry, too many clients already ``` which results in tests failing. * Try p=2 concurrency to restrict slowness of servers which causes partial state join tests to flake * Debug logging * Only run flakey tests * Only adjust connection pool limits in worker mode * Move cp vars to somewhere where they get executed in CI * Move cp values back to where they actually work * Debug logging * Try p=1 to see if this makes worker mode happier * Remove debug logging	2023-10-27 18:29:20 +01:00
Erik Johnston	98d39e590a	Merge branch 'master' into develop	2023-09-15 14:39:20 +01:00
Erik Johnston	3cf1a3aa17	Use bookwork as docker base image (#16324)	2023-09-15 13:14:10 +01:00
Patrick Cloke	aa483cb4c9	Update ruff config (#16283) ... Enable additional checks & clean-up unneeded configuration.	2023-09-08 11:24:36 -04:00
Erik Johnston	8940d1b28e	Add /notifications endpoint to workers (#16265)	2023-09-07 09:26:07 +00:00