mirrors/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2024-12-22 04:34:28 +03:00
Code Issues Projects Releases Packages Wiki Activity

Merge branch 'master' of github.com:matrix-org/synapse into erikj/dinsic-merged-master

Browse source
This commit is contained in:
Erik Johnston 2019-03-15 14:36:44 +00:00
commit 44c0661d97
350 changed files with 19071 additions and 6403 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
.circleci/config.yml
View file

@ -4,23 +4,21 @@ jobs:
machine: true
steps:
- checkout
- run: docker build -f docker/Dockerfile -t matrixdotorg/synapse:${CIRCLE_TAG} .
- run: docker build -f docker/Dockerfile -t matrixdotorg/synapse:${CIRCLE_TAG}-py3 --build-arg PYTHON_VERSION=3.6 .
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG}-py2 .
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} -t matrixdotorg/synapse:${CIRCLE_TAG}-py3 --build-arg PYTHON_VERSION=3.6 .
- run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
- run: docker push matrixdotorg/synapse:${CIRCLE_TAG}
- run: docker push matrixdotorg/synapse:${CIRCLE_TAG}-py2
- run: docker push matrixdotorg/synapse:${CIRCLE_TAG}-py3
dockerhubuploadlatest:
machine: true
steps:
- checkout
- run: docker build -f docker/Dockerfile -t matrixdotorg/synapse:${CIRCLE_SHA1} .
- run: docker build -f docker/Dockerfile -t matrixdotorg/synapse:${CIRCLE_SHA1}-py3 --build-arg PYTHON_VERSION=3.6 .
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest-py2 .
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest -t matrixdotorg/synapse:latest-py3 --build-arg PYTHON_VERSION=3.6 .
- run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
- run: docker tag matrixdotorg/synapse:${CIRCLE_SHA1} matrixdotorg/synapse:latest
- run: docker tag matrixdotorg/synapse:${CIRCLE_SHA1}-py3 matrixdotorg/synapse:latest-py3
- run: docker push matrixdotorg/synapse:${CIRCLE_SHA1}
- run: docker push matrixdotorg/synapse:${CIRCLE_SHA1}-py3
- run: docker push matrixdotorg/synapse:latest
- run: docker push matrixdotorg/synapse:latest-py2
- run: docker push matrixdotorg/synapse:latest-py3
sytestpy2:
docker:

4
.circleci/merge_base_branch.sh
View file

@ -20,7 +20,7 @@ else
fi
# Show what we are before
git show -s
git --no-pager show -s
# Set up username so it can do a merge
git config --global user.email bot@matrix.org
@ -31,4 +31,4 @@ git fetch -u origin $GITBASE
git merge --no-edit origin/$GITBASE
# Show what we are after.
git show -s
git --no-pager show -s

15
.codecov.yml Normal file
View file

@ -0,0 +1,15 @@
comment:
layout: "diff"
coverage:
status:
project:
default:
target: 0 # Target % coverage, can be auto. Turned off for now
threshold: null
base: auto
patch:
default:
target: 0
threshold: null
base: auto

7
.coveragerc Normal file
View file

@ -0,0 +1,7 @@
[run]
branch = True
parallel = True
include = synapse/*
[report]
precision = 2

2
.dockerignore
View file

@ -5,3 +5,5 @@ demo/etc
tox.ini
.git/*
.tox/*
debian/matrix-synapse/
debian/matrix-synapse-*/

9
.editorconfig Normal file
View file

@ -0,0 +1,9 @@
# EditorConfig https://EditorConfig.org
# top-most EditorConfig file
root = true
# 4 space indentation
[*.py]
indent_style = space
indent_size = 4

36
.github/ISSUE_TEMPLATE.md → .github/ISSUE_TEMPLATE/BUG_REPORT.md vendored
View file

@ -1,3 +1,9 @@
---
name: Bug report
about: Create a report to help us improve
---
<!--
**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**:
@ -17,32 +23,44 @@ Text between <!-- and --​> marks will be invisible in the report.
### Description
Describe here the problem that you are experiencing, or the feature you are requesting.
<!-- Describe here the problem that you are experiencing -->
### Steps to reproduce
- For bugs, list the steps
- list the steps
- that reproduce the bug
- using hyphens as bullet points
<!--
Describe how what happens differs from what you expected.
<!-- If you can identify any relevant log snippets from _homeserver.log_, please include
If you can identify any relevant log snippets from _homeserver.log_, please include
those (please be careful to remove any personal or private data). Please surround them with
``` (three backticks, on a line on their own), so that they are formatted legibly. -->
``` (three backticks, on a line on their own), so that they are formatted legibly.
-->
### Version information
<!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->
- **Homeserver**: Was this issue identified on matrix.org or another homeserver?
<!-- Was this issue identified on matrix.org or another homeserver? -->
- **Homeserver**:
If not matrix.org:
- **Version**: What version of Synapse is running? <!--
<!--
What version of Synapse is running?
You can find the Synapse version by inspecting the server headers (replace matrix.org with
your own homeserver domain):
$ curl -v https://matrix.org/_matrix/client/versions 2>&1 | grep "Server:"
-->
- **Install method**: package manager/git clone/pip
- **Platform**: Tell us about the environment in which your homeserver is operating
- distro, hardware, if it's running in a vm/container, etc.
- **Version**:
- **Install method**:
<!-- examples: package manager/git clone/pip -->
- **Platform**:
<!--
Tell us about the environment in which your homeserver is operating
distro, hardware, if it's running in a vm/container, etc.
-->

9
.github/ISSUE_TEMPLATE/FEATURE_REQUEST.md vendored Normal file
View file

@ -0,0 +1,9 @@
---
name: Feature request
about: Suggest an idea for this project
---
**Description:**
<!-- Describe here the feature you are requesting. -->

9
.github/ISSUE_TEMPLATE/SUPPORT_REQUEST.md vendored Normal file
View file

@ -0,0 +1,9 @@
---
name: Support request
about: I need support for Synapse
---
# Please ask for support in [**#matrix:matrix.org**](https://matrix.to/#/#matrix:matrix.org)
## Don't file an issue as a support request.

7
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View file

@ -0,0 +1,7 @@
### Pull Request Checklist
<!-- Please read CONTRIBUTING.rst before submitting your pull request -->
* [ ] Pull request is based on the develop branch
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#changelog)
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#sign-off)

3
.github/SUPPORT.md vendored Normal file
View file

@ -0,0 +1,3 @@
[**#matrix:matrix.org**](https://matrix.to/#/#matrix:matrix.org) is the official support room for Matrix, and can be accessed by any client from https://matrix.org/docs/projects/try-matrix-now.html
It can also be access via IRC bridge at irc://irc.freenode.net/matrix or on the web here: https://webchat.freenode.net/?channels=matrix

79
.gitignore vendored
View file

@ -1,59 +1,36 @@
*.pyc
.*.swp
# filename patterns
*~
.*.swp
.#*
*.deb
*.egg
*.egg-info
*.lock
.DS_Store
*.pyc
*.tac
_trial_temp/
_trial_temp*/
logs/
dbs/
*.egg
dist/
docs/build/
*.egg-info
cmdclient_config.json
homeserver*.db
homeserver*.log
homeserver*.log.*
homeserver*.pid
homeserver*.yaml
# stuff that is likely to exist when you run a server locally
/*.signing.key
/*.tls.crt
/*.tls.key
/uploads
/media_store/
*.signing.key
*.tls.crt
*.tls.dh
*.tls.key
# IDEs
/.idea/
/.ropeproject/
/.vscode/
.coverage
htmlcov
# build products
/.coverage*
!/.coveragerc
/.tox
/build/
/coverage.*
/dist/
/docs/build/
/htmlcov
/pip-wheel-metadata/
demo/*/*.db
demo/*/*.log
demo/*/*.log.*
demo/*/*.pid
demo/media_store.*
demo/etc
uploads
cache
.idea/
media_store/
*.tac
build/
venv/
venv*/
*venv/
localhost-800*/
static/client/register/register_config.js
.tox
env/
*.config
.vscode/
.ropeproject/

73
.travis.yml
View file

@ -1,4 +1,4 @@
sudo: false
dist: xenial
language: python
cache:
@ -22,49 +22,76 @@ branches:
- master
- develop
- /^release-v/
- rav/pg95
# When running the tox environments that call Twisted Trial, we can pass the -j
# flag to run the tests concurrently. We set this to 2 for CPU bound tests
# (SQLite) and 4 for I/O bound tests (PostgreSQL).
matrix:
fast_finish: true
include:
- python: 2.7
env: TOX_ENV=packaging
- name: "pep8"
python: 3.6
env: TOX_ENV="pep8,check_isort,packaging"
- python: 3.6
env: TOX_ENV="pep8,check_isort"
- name: "py2.7 / sqlite"
python: 2.7
env: TOX_ENV=py27,codecov TRIAL_FLAGS="-j 2"
- python: 2.7
env: TOX_ENV=py27
- name: "py2.7 / sqlite / olddeps"
python: 2.7
env: TOX_ENV=py27-old TRIAL_FLAGS="-j 2"
- python: 2.7
env: TOX_ENV=py27-old
- python: 2.7
env: TOX_ENV=py27-postgres TRIAL_FLAGS="-j 4"
- name: "py2.7 / postgres9.5"
python: 2.7
addons:
postgresql: "9.5"
env: TOX_ENV=py27-postgres,codecov TRIAL_FLAGS="-j 4"
services:
- postgresql
- python: 3.5
env: TOX_ENV=py35
- name: "py3.5 / sqlite"
python: 3.5
env: TOX_ENV=py35,codecov TRIAL_FLAGS="-j 2"
- python: 3.6
env: TOX_ENV=py36
- name: "py3.7 / sqlite"
python: 3.7
env: TOX_ENV=py37,codecov TRIAL_FLAGS="-j 2"
- python: 3.6
env: TOX_ENV=py36-postgres TRIAL_FLAGS="-j 4"
- name: "py3.7 / postgres9.4"
python: 3.7
addons:
postgresql: "9.4"
env: TOX_ENV=py37-postgres TRIAL_FLAGS="-j 4"
services:
- postgresql
- name: "py3.7 / postgres9.5"
python: 3.7
addons:
postgresql: "9.5"
env: TOX_ENV=py37-postgres,codecov TRIAL_FLAGS="-j 4"
services:
- postgresql
- # we only need to check for the newsfragment if it's a PR build
if: type = pull_request
name: "check-newsfragment"
python: 3.6
env: TOX_ENV=check-newsfragment
script:
- git remote set-branches --add origin develop
- git fetch origin develop
- tox -e $TOX_ENV
script: scripts-dev/check-newsfragment
install:
# this just logs the postgres version we will be testing against (if any)
- psql -At -U postgres -c 'select version();' || true
- pip install tox
# if we don't have python3.6 in this environment, travis unhelpfully gives us
# a `python3.6` on our path which does nothing but spit out a warning. Tox
# tries to run it (even if we're not running a py36 env), so the build logs
# then have warnings which look like errors. To reduce the noise, remove the
# non-functional python3.6.
- ( ! command -v python3.6 || python3.6 --version ) &>/dev/null || rm -f $(command -v python3.6)
script:
- tox -e $TOX_ENV

3
AUTHORS.rst
View file

@ -66,3 +66,6 @@ Pierre Jaury <pierre at jaury.eu>
Serban Constantin <serban.constantin at gmail dot com>
* Small bug fix
Jason Robinson <jasonr at matrix.org>
* Minor fixes

399
CHANGES.md
View file

@ -1,3 +1,402 @@
Synapse 0.99.2 (2019-03-01)
===========================
Features
--------
- Added an HAProxy example in the reverse proxy documentation. Contributed by Benoît S. (“Benpro”). ([\#4541](https://github.com/matrix-org/synapse/issues/4541))
- Add basic optional sentry integration. ([\#4632](https://github.com/matrix-org/synapse/issues/4632), [\#4694](https://github.com/matrix-org/synapse/issues/4694))
- Transfer bans on room upgrade. ([\#4642](https://github.com/matrix-org/synapse/issues/4642))
- Add configurable room list publishing rules. ([\#4647](https://github.com/matrix-org/synapse/issues/4647))
- Support .well-known delegation when issuing certificates through ACME. ([\#4652](https://github.com/matrix-org/synapse/issues/4652))
- Allow registration and login to be handled by a worker instance. ([\#4666](https://github.com/matrix-org/synapse/issues/4666), [\#4670](https://github.com/matrix-org/synapse/issues/4670), [\#4682](https://github.com/matrix-org/synapse/issues/4682))
- Reduce the overhead of creating outbound federation connections over TLS by caching the TLS client options. ([\#4674](https://github.com/matrix-org/synapse/issues/4674))
- Add prometheus metrics for number of outgoing EDUs, by type. ([\#4695](https://github.com/matrix-org/synapse/issues/4695))
- Return correct error code when inviting a remote user to a room whose homeserver does not support the room version. ([\#4721](https://github.com/matrix-org/synapse/issues/4721))
- Prevent showing rooms to other servers that were set to not federate. ([\#4746](https://github.com/matrix-org/synapse/issues/4746))
Bugfixes
--------
- Fix possible exception when paginating. ([\#4263](https://github.com/matrix-org/synapse/issues/4263))
- The dependency checker now correctly reports a version mismatch for optional
dependencies, instead of reporting the dependency missing. ([\#4450](https://github.com/matrix-org/synapse/issues/4450))
- Set CORS headers on .well-known requests. ([\#4651](https://github.com/matrix-org/synapse/issues/4651))
- Fix kicking guest users on guest access revocation in worker mode. ([\#4667](https://github.com/matrix-org/synapse/issues/4667))
- Fix an issue in the database migration script where the
`e2e_room_keys.is_verified` column wasn't considered as
a boolean. ([\#4680](https://github.com/matrix-org/synapse/issues/4680))
- Fix TaskStopped exceptions in logs when outbound requests time out. ([\#4690](https://github.com/matrix-org/synapse/issues/4690))
- Fix ACME config for python 2. ([\#4717](https://github.com/matrix-org/synapse/issues/4717))
- Fix paginating over federation persisting incorrect state. ([\#4718](https://github.com/matrix-org/synapse/issues/4718))
Internal Changes
----------------
- Run `black` to reformat user directory code. ([\#4635](https://github.com/matrix-org/synapse/issues/4635))
- Reduce number of exceptions we log. ([\#4643](https://github.com/matrix-org/synapse/issues/4643), [\#4668](https://github.com/matrix-org/synapse/issues/4668))
- Introduce upsert batching functionality in the database layer. ([\#4644](https://github.com/matrix-org/synapse/issues/4644))
- Fix various spelling mistakes. ([\#4657](https://github.com/matrix-org/synapse/issues/4657))
- Cleanup request exception logging. ([\#4669](https://github.com/matrix-org/synapse/issues/4669), [\#4737](https://github.com/matrix-org/synapse/issues/4737), [\#4738](https://github.com/matrix-org/synapse/issues/4738))
- Improve replication performance by reducing cache invalidation traffic. ([\#4671](https://github.com/matrix-org/synapse/issues/4671), [\#4715](https://github.com/matrix-org/synapse/issues/4715), [\#4748](https://github.com/matrix-org/synapse/issues/4748))
- Test against Postgres 9.5 as well as 9.4. ([\#4676](https://github.com/matrix-org/synapse/issues/4676))
- Run unit tests against python 3.7. ([\#4677](https://github.com/matrix-org/synapse/issues/4677))
- Attempt to clarify installation instructions/config. ([\#4681](https://github.com/matrix-org/synapse/issues/4681))
- Clean up gitignores. ([\#4688](https://github.com/matrix-org/synapse/issues/4688))
- Minor tweaks to acme docs. ([\#4689](https://github.com/matrix-org/synapse/issues/4689))
- Improve the logging in the pusher process. ([\#4691](https://github.com/matrix-org/synapse/issues/4691))
- Better checks on newsfragments. ([\#4698](https://github.com/matrix-org/synapse/issues/4698), [\#4750](https://github.com/matrix-org/synapse/issues/4750))
- Avoid some redundant work when processing read receipts. ([\#4706](https://github.com/matrix-org/synapse/issues/4706))
- Run `push_receipts_to_remotes` as background job. ([\#4707](https://github.com/matrix-org/synapse/issues/4707))
- Add prometheus metrics for number of badge update pushes. ([\#4709](https://github.com/matrix-org/synapse/issues/4709))
- Reduce pusher logging on startup ([\#4716](https://github.com/matrix-org/synapse/issues/4716))
- Don't log exceptions when failing to fetch remote server keys. ([\#4722](https://github.com/matrix-org/synapse/issues/4722))
- Correctly proxy exception in frontend_proxy worker. ([\#4723](https://github.com/matrix-org/synapse/issues/4723))
- Add database version to phonehome stats. ([\#4753](https://github.com/matrix-org/synapse/issues/4753))
Synapse 0.99.1.1 (2019-02-14)
=============================
Bugfixes
--------
- Fix "TypeError: '>' not supported" when starting without an existing certificate.
Fix a bug where an existing certificate would be reprovisoned every day. ([\#4648](https://github.com/matrix-org/synapse/issues/4648))
Synapse 0.99.1 (2019-02-14)
===========================
Features
--------
- Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902))
- Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420))
- Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522))
- Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580))
- Allow the "unavailable" presence status for /sync.
This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592))
- There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636))
- The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614))
Bugfixes
--------
- Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530))
- Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546))
- Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589))
- Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591))
- Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608))
- Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618))
- Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625))
- Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627))
Internal Changes
----------------
- Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462))
- Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513))
- Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567))
- Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576))
- Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578))
- Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584))
- Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586))
- Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607))
- Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611))
- Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616))
- Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619))
- README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621))
Synapse 0.99.0 (2019-02-05)
===========================
Synapse v0.99.x is a precursor to the upcoming Synapse v1.0 release. It contains foundational changes to room architecture and the federation security model necessary to support the upcoming r0 release of the Server to Server API.
Features
--------
- Synapse's cipher string has been updated to require ECDH key exchange. Configuring and generating dh_params is no longer required, and they will be ignored. ([\#4229](https://github.com/matrix-org/synapse/issues/4229))
- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt). ([\#4384](https://github.com/matrix-org/synapse/issues/4384), [\#4492](https://github.com/matrix-org/synapse/issues/4492), [\#4525](https://github.com/matrix-org/synapse/issues/4525), [\#4572](https://github.com/matrix-org/synapse/issues/4572), [\#4564](https://github.com/matrix-org/synapse/issues/4564), [\#4566](https://github.com/matrix-org/synapse/issues/4566), [\#4547](https://github.com/matrix-org/synapse/issues/4547), [\#4557](https://github.com/matrix-org/synapse/issues/4557))
- Implement MSC1708 (.well-known routing for server-server federation) ([\#4408](https://github.com/matrix-org/synapse/issues/4408), [\#4409](https://github.com/matrix-org/synapse/issues/4409), [\#4426](https://github.com/matrix-org/synapse/issues/4426), [\#4427](https://github.com/matrix-org/synapse/issues/4427), [\#4428](https://github.com/matrix-org/synapse/issues/4428), [\#4464](https://github.com/matrix-org/synapse/issues/4464), [\#4468](https://github.com/matrix-org/synapse/issues/4468), [\#4487](https://github.com/matrix-org/synapse/issues/4487), [\#4488](https://github.com/matrix-org/synapse/issues/4488), [\#4489](https://github.com/matrix-org/synapse/issues/4489), [\#4497](https://github.com/matrix-org/synapse/issues/4497), [\#4511](https://github.com/matrix-org/synapse/issues/4511), [\#4516](https://github.com/matrix-org/synapse/issues/4516), [\#4520](https://github.com/matrix-org/synapse/issues/4520), [\#4521](https://github.com/matrix-org/synapse/issues/4521), [\#4539](https://github.com/matrix-org/synapse/issues/4539), [\#4542](https://github.com/matrix-org/synapse/issues/4542), [\#4544](https://github.com/matrix-org/synapse/issues/4544))
- Search now includes results from predecessor rooms after a room upgrade. ([\#4415](https://github.com/matrix-org/synapse/issues/4415))
- Config option to disable requesting MSISDN on registration. ([\#4423](https://github.com/matrix-org/synapse/issues/4423))
- Add a metric for tracking event stream position of the user directory. ([\#4445](https://github.com/matrix-org/synapse/issues/4445))
- Support exposing server capabilities in CS API (MSC1753, MSC1804) ([\#4472](https://github.com/matrix-org/synapse/issues/4472), [81b7e7eed](https://github.com/matrix-org/synapse/commit/81b7e7eed323f55d6550e7a270a9dc2c4c7b0fe0)))
- Add support for room version 3 ([\#4483](https://github.com/matrix-org/synapse/issues/4483), [\#4499](https://github.com/matrix-org/synapse/issues/4499), [\#4515](https://github.com/matrix-org/synapse/issues/4515), [\#4523](https://github.com/matrix-org/synapse/issues/4523), [\#4535](https://github.com/matrix-org/synapse/issues/4535))
- Synapse will now reload TLS certificates from disk upon SIGHUP. ([\#4495](https://github.com/matrix-org/synapse/issues/4495), [\#4524](https://github.com/matrix-org/synapse/issues/4524))
- The matrixdotorg/synapse Docker images now use Python 3 by default. ([\#4558](https://github.com/matrix-org/synapse/issues/4558))
Bugfixes
--------
- Prevent users with access tokens predating the introduction of device IDs from creating spurious entries in the user_ips table. ([\#4369](https://github.com/matrix-org/synapse/issues/4369))
- Fix typo in ALL_USER_TYPES definition to ensure type is a tuple ([\#4392](https://github.com/matrix-org/synapse/issues/4392))
- Fix high CPU usage due to remote devicelist updates ([\#4397](https://github.com/matrix-org/synapse/issues/4397))
- Fix potential bug where creating or joining a room could fail ([\#4404](https://github.com/matrix-org/synapse/issues/4404))
- Fix bug when rejecting remote invites ([\#4405](https://github.com/matrix-org/synapse/issues/4405), [\#4527](https://github.com/matrix-org/synapse/issues/4527))
- Fix incorrect logcontexts after a Deferred was cancelled ([\#4407](https://github.com/matrix-org/synapse/issues/4407))
- Ensure encrypted room state is persisted across room upgrades. ([\#4411](https://github.com/matrix-org/synapse/issues/4411))
- Copy over whether a room is a direct message and any associated room tags on room upgrade. ([\#4412](https://github.com/matrix-org/synapse/issues/4412))
- Fix None guard in calling config.server.is_threepid_reserved ([\#4435](https://github.com/matrix-org/synapse/issues/4435))
- Don't send IP addresses as SNI ([\#4452](https://github.com/matrix-org/synapse/issues/4452))
- Fix UnboundLocalError in post_urlencoded_get_json ([\#4460](https://github.com/matrix-org/synapse/issues/4460))
- Add a timeout to filtered room directory queries. ([\#4461](https://github.com/matrix-org/synapse/issues/4461))
- Workaround for login error when using both LDAP and internal authentication. ([\#4486](https://github.com/matrix-org/synapse/issues/4486))
- Fix a bug where setting a relative consent directory path would cause a crash. ([\#4512](https://github.com/matrix-org/synapse/issues/4512))
Deprecations and Removals
-------------------------
- Synapse no longer generates self-signed TLS certificates when generating a configuration file. ([\#4509](https://github.com/matrix-org/synapse/issues/4509))
Improved Documentation
----------------------
- Update debian installation instructions ([\#4526](https://github.com/matrix-org/synapse/issues/4526))
Internal Changes
----------------
- Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+. ([\#4306](https://github.com/matrix-org/synapse/issues/4306), [\#4459](https://github.com/matrix-org/synapse/issues/4459), [\#4466](https://github.com/matrix-org/synapse/issues/4466), [\#4471](https://github.com/matrix-org/synapse/issues/4471), [\#4477](https://github.com/matrix-org/synapse/issues/4477), [\#4505](https://github.com/matrix-org/synapse/issues/4505))
- Update README to use the new virtualenv everywhere ([\#4342](https://github.com/matrix-org/synapse/issues/4342))
- Add better logging for unexpected errors while sending transactions ([\#4368](https://github.com/matrix-org/synapse/issues/4368))
- Apply a unique index to the user_ips table, preventing duplicates. ([\#4370](https://github.com/matrix-org/synapse/issues/4370), [\#4432](https://github.com/matrix-org/synapse/issues/4432), [\#4434](https://github.com/matrix-org/synapse/issues/4434))
- Silence travis-ci build warnings by removing non-functional python3.6 ([\#4377](https://github.com/matrix-org/synapse/issues/4377))
- Fix a comment in the generated config file ([\#4387](https://github.com/matrix-org/synapse/issues/4387))
- Add ground work for implementing future federation API versions ([\#4390](https://github.com/matrix-org/synapse/issues/4390))
- Update dependencies on msgpack and pymacaroons to use the up-to-date packages. ([\#4399](https://github.com/matrix-org/synapse/issues/4399))
- Tweak codecov settings to make them less loud. ([\#4400](https://github.com/matrix-org/synapse/issues/4400))
- Implement server support for MSC1794 - Federation v2 Invite API ([\#4402](https://github.com/matrix-org/synapse/issues/4402))
- debian package: symlink to explicit python version ([\#4433](https://github.com/matrix-org/synapse/issues/4433))
- Add infrastructure to support different event formats ([\#4437](https://github.com/matrix-org/synapse/issues/4437), [\#4447](https://github.com/matrix-org/synapse/issues/4447), [\#4448](https://github.com/matrix-org/synapse/issues/4448), [\#4470](https://github.com/matrix-org/synapse/issues/4470), [\#4481](https://github.com/matrix-org/synapse/issues/4481), [\#4482](https://github.com/matrix-org/synapse/issues/4482), [\#4493](https://github.com/matrix-org/synapse/issues/4493), [\#4494](https://github.com/matrix-org/synapse/issues/4494), [\#4496](https://github.com/matrix-org/synapse/issues/4496), [\#4510](https://github.com/matrix-org/synapse/issues/4510), [\#4514](https://github.com/matrix-org/synapse/issues/4514))
- Generate the debian config during build ([\#4444](https://github.com/matrix-org/synapse/issues/4444))
- Clarify documentation for the `public_baseurl` config param ([\#4458](https://github.com/matrix-org/synapse/issues/4458), [\#4498](https://github.com/matrix-org/synapse/issues/4498))
- Fix quoting for allowed_local_3pids example config ([\#4476](https://github.com/matrix-org/synapse/issues/4476))
- Remove deprecated --process-dependency-links option from UPGRADE.rst ([\#4485](https://github.com/matrix-org/synapse/issues/4485))
- Make it possible to set the log level for tests via an environment variable ([\#4506](https://github.com/matrix-org/synapse/issues/4506))
- Reduce the log level of linearizer lock acquirement to DEBUG. ([\#4507](https://github.com/matrix-org/synapse/issues/4507))
- Fix code to comply with linting in PyFlakes 3.7.1. ([\#4519](https://github.com/matrix-org/synapse/issues/4519))
- Add some debug for membership syncing issues ([\#4538](https://github.com/matrix-org/synapse/issues/4538))
- Docker: only copy what we need to the build image ([\#4562](https://github.com/matrix-org/synapse/issues/4562))
Synapse 0.34.1.1 (2019-01-11)
=============================
This release fixes CVE-2019-5885 and is recommended for all users of Synapse 0.34.1.
This release is compatible with Python 2.7 and 3.5+. Python 3.7 is fully supported.
Bugfixes
--------
- Fix spontaneous logout on upgrade
([\#4374](https://github.com/matrix-org/synapse/issues/4374))
Synapse 0.34.1 (2019-01-09)
===========================
Internal Changes
----------------
- Add better logging for unexpected errors while sending transactions ([\#4361](https://github.com/matrix-org/synapse/issues/4361), [\#4362](https://github.com/matrix-org/synapse/issues/4362))
Synapse 0.34.1rc1 (2019-01-08)
==============================
Features
--------
- Special-case a support user for use in verifying behaviour of a given server. The support user does not appear in user directory or monthly active user counts. ([\#4141](https://github.com/matrix-org/synapse/issues/4141), [\#4344](https://github.com/matrix-org/synapse/issues/4344))
- Support for serving .well-known files ([\#4262](https://github.com/matrix-org/synapse/issues/4262))
- Rework SAML2 authentication ([\#4265](https://github.com/matrix-org/synapse/issues/4265), [\#4267](https://github.com/matrix-org/synapse/issues/4267))
- SAML2 authentication: Initialise user display name from SAML2 data ([\#4272](https://github.com/matrix-org/synapse/issues/4272))
- Synapse can now have its conditional/extra dependencies installed by pip. This functionality can be used by using `pip install matrix-synapse[feature]`, where feature is a comma separated list with the possible values `email.enable_notifs`, `matrix-synapse-ldap3`, `postgres`, `resources.consent`, `saml2`, `url_preview`, and `test`. If you want to install all optional dependencies, you can use "all" instead. ([\#4298](https://github.com/matrix-org/synapse/issues/4298), [\#4325](https://github.com/matrix-org/synapse/issues/4325), [\#4327](https://github.com/matrix-org/synapse/issues/4327))
- Add routes for reading account data. ([\#4303](https://github.com/matrix-org/synapse/issues/4303))
- Add opt-in support for v2 rooms ([\#4307](https://github.com/matrix-org/synapse/issues/4307))
- Add a script to generate a clean config file ([\#4315](https://github.com/matrix-org/synapse/issues/4315))
- Return server data in /login response ([\#4319](https://github.com/matrix-org/synapse/issues/4319))
Bugfixes
--------
- Fix contains_url check to be consistent with other instances in code-base and check that value is an instance of string. ([\#3405](https://github.com/matrix-org/synapse/issues/3405))
- Fix CAS login when username is not valid in an MXID ([\#4264](https://github.com/matrix-org/synapse/issues/4264))
- Send CORS headers for /media/config ([\#4279](https://github.com/matrix-org/synapse/issues/4279))
- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284))
- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294))
- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295))
- The metric synapse_admin_mau:current previously did not update when config.mau_stats_only was set to True ([\#4305](https://github.com/matrix-org/synapse/issues/4305))
- Fixed per-room account data filters ([\#4309](https://github.com/matrix-org/synapse/issues/4309))
- Fix indentation in default config ([\#4313](https://github.com/matrix-org/synapse/issues/4313))
- Fix synapse:latest docker upload ([\#4316](https://github.com/matrix-org/synapse/issues/4316))
- Fix test_metric.py compatibility with prometheus_client 0.5. Contributed by Maarten de Vries <maarten@de-vri.es>. ([\#4317](https://github.com/matrix-org/synapse/issues/4317))
- Avoid packaging _trial_temp directory in -py3 debian packages ([\#4326](https://github.com/matrix-org/synapse/issues/4326))
- Check jinja version for consent resource ([\#4327](https://github.com/matrix-org/synapse/issues/4327))
- fix NPE in /messages by checking if all events were filtered out ([\#4330](https://github.com/matrix-org/synapse/issues/4330))
- Fix `python -m synapse.config` on Python 3. ([\#4356](https://github.com/matrix-org/synapse/issues/4356))
Deprecations and Removals
-------------------------
- Remove the deprecated v1/register API on Python 2. It was never ported to Python 3. ([\#4334](https://github.com/matrix-org/synapse/issues/4334))
Internal Changes
----------------
- Getting URL previews of IP addresses no longer fails on Python 3. ([\#4215](https://github.com/matrix-org/synapse/issues/4215))
- drop undocumented dependency on dateutil ([\#4266](https://github.com/matrix-org/synapse/issues/4266))
- Update the example systemd config to use a virtualenv ([\#4273](https://github.com/matrix-org/synapse/issues/4273))
- Update link to kernel DCO guide ([\#4274](https://github.com/matrix-org/synapse/issues/4274))
- Make isort tox check print diff when it fails ([\#4283](https://github.com/matrix-org/synapse/issues/4283))
- Log room_id in Unknown room errors ([\#4297](https://github.com/matrix-org/synapse/issues/4297))
- Documentation improvements for coturn setup. Contributed by Krithin Sitaram. ([\#4333](https://github.com/matrix-org/synapse/issues/4333))
- Update pull request template to use absolute links ([\#4341](https://github.com/matrix-org/synapse/issues/4341))
- Update README to not lie about required restart when updating TLS certificates ([\#4343](https://github.com/matrix-org/synapse/issues/4343))
- Update debian packaging for compatibility with transitional package ([\#4349](https://github.com/matrix-org/synapse/issues/4349))
- Fix command hint to generate a config file when trying to start without a config file ([\#4353](https://github.com/matrix-org/synapse/issues/4353))
- Add better logging for unexpected errors while sending transactions ([\#4358](https://github.com/matrix-org/synapse/issues/4358))
Synapse 0.34.0 (2018-12-20)
===========================
Synapse 0.34.0 is the first release to fully support Python 3. Synapse will now
run on Python versions 3.5 or 3.6 (as well as 2.7). Support for Python 3.7
remains experimental.
We recommend upgrading to Python 3, but make sure to read the [upgrade
notes](UPGRADE.rst#upgrading-to-v0340) when doing so.
Features
--------
- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284))
- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294))
- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295))
- Add a welcome page for the client API port. Credit to @krombel! ([\#4289](https://github.com/matrix-org/synapse/issues/4289))
- Remove Matrix console from the default distribution ([\#4290](https://github.com/matrix-org/synapse/issues/4290))
- Add option to track MAU stats (but not limit people) ([\#3830](https://github.com/matrix-org/synapse/issues/3830))
- Add an option to enable recording IPs for appservice users ([\#3831](https://github.com/matrix-org/synapse/issues/3831))
- Rename login type `m.login.cas` to `m.login.sso` ([\#4220](https://github.com/matrix-org/synapse/issues/4220))
- Add an option to disable search for homeservers that may not be interested in it. ([\#4230](https://github.com/matrix-org/synapse/issues/4230))
Bugfixes
--------
- Pushrules can now again be made with non-ASCII rule IDs. ([\#4165](https://github.com/matrix-org/synapse/issues/4165))
- The media repository now no longer fails to decode UTF-8 filenames when downloading remote media. ([\#4176](https://github.com/matrix-org/synapse/issues/4176))
- URL previews now correctly decode non-UTF-8 text if the header contains a `<meta http-equiv="Content-Type"` header. ([\#4183](https://github.com/matrix-org/synapse/issues/4183))
- Fix an issue where public consent URLs had two slashes. ([\#4192](https://github.com/matrix-org/synapse/issues/4192))
- Fallback auth now accepts the session parameter on Python 3. ([\#4197](https://github.com/matrix-org/synapse/issues/4197))
- Remove riot.im from the list of trusted Identity Servers in the default configuration ([\#4207](https://github.com/matrix-org/synapse/issues/4207))
- fix start up failure when mau_limit_reserved_threepids set and db is postgres ([\#4211](https://github.com/matrix-org/synapse/issues/4211))
- Fix auto join failures for servers that require user consent ([\#4223](https://github.com/matrix-org/synapse/issues/4223))
- Fix exception caused by non-ascii event IDs ([\#4241](https://github.com/matrix-org/synapse/issues/4241))
- Pushers can now be unsubscribed from on Python 3. ([\#4250](https://github.com/matrix-org/synapse/issues/4250))
- Fix UnicodeDecodeError when postgres is configured to give non-English errors ([\#4253](https://github.com/matrix-org/synapse/issues/4253))
Internal Changes
----------------
- Debian packages utilising a virtualenv with bundled dependencies can now be built. ([\#4212](https://github.com/matrix-org/synapse/issues/4212))
- Disable pager when running git-show in CI ([\#4291](https://github.com/matrix-org/synapse/issues/4291))
- A coveragerc file has been added. ([\#4180](https://github.com/matrix-org/synapse/issues/4180))
- Add a GitHub pull request template and add multiple issue templates ([\#4182](https://github.com/matrix-org/synapse/issues/4182))
- Update README to reflect the fact that [\#1491](https://github.com/matrix-org/synapse/issues/1491) is fixed ([\#4188](https://github.com/matrix-org/synapse/issues/4188))
- Run the AS senders as background processes to fix warnings ([\#4189](https://github.com/matrix-org/synapse/issues/4189))
- Add some diagnostics to the tests to detect logcontext problems ([\#4190](https://github.com/matrix-org/synapse/issues/4190))
- Add missing `jpeg` package prerequisite for OpenBSD in README. ([\#4193](https://github.com/matrix-org/synapse/issues/4193))
- Add a note saying you need to manually reclaim disk space after using the Purge History API ([\#4200](https://github.com/matrix-org/synapse/issues/4200))
- More logcontext checking in unittests ([\#4205](https://github.com/matrix-org/synapse/issues/4205))
- Ignore `__pycache__` directories in the database schema folder ([\#4214](https://github.com/matrix-org/synapse/issues/4214))
- Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers ([\#4224](https://github.com/matrix-org/synapse/issues/4224))
- Added automated coverage reporting to CI. ([\#4225](https://github.com/matrix-org/synapse/issues/4225))
- Garbage-collect after each unit test to fix logcontext leaks ([\#4227](https://github.com/matrix-org/synapse/issues/4227))
- add more detail to logging regarding "More than one row matched" error ([\#4234](https://github.com/matrix-org/synapse/issues/4234))
- Drop sent_transactions table ([\#4244](https://github.com/matrix-org/synapse/issues/4244))
- Add a basic .editorconfig ([\#4257](https://github.com/matrix-org/synapse/issues/4257))
- Update README.rst and UPGRADE.rst for Python 3. ([\#4260](https://github.com/matrix-org/synapse/issues/4260))
- Remove obsolete `verbose` and `log_file` settings from `homeserver.yaml` for Docker image. ([\#4261](https://github.com/matrix-org/synapse/issues/4261))
Synapse 0.33.9 (2018-11-19)
===========================
No significant changes.
Synapse 0.33.9rc1 (2018-11-14)
==============================
Features
--------
- Include flags to optionally add `m.login.terms` to the registration flow when consent tracking is enabled. ([\#4004](https://github.com/matrix-org/synapse/issues/4004), [\#4133](https://github.com/matrix-org/synapse/issues/4133), [\#4142](https://github.com/matrix-org/synapse/issues/4142), [\#4184](https://github.com/matrix-org/synapse/issues/4184))
- Support for replacing rooms with new ones ([\#4091](https://github.com/matrix-org/synapse/issues/4091), [\#4099](https://github.com/matrix-org/synapse/issues/4099), [\#4100](https://github.com/matrix-org/synapse/issues/4100), [\#4101](https://github.com/matrix-org/synapse/issues/4101))
Bugfixes
--------
- Fix exceptions when using the email mailer on Python 3. ([\#4095](https://github.com/matrix-org/synapse/issues/4095))
- Fix e2e key backup with more than 9 backup versions ([\#4113](https://github.com/matrix-org/synapse/issues/4113))
- Searches that request profile info now no longer fail with a 500. ([\#4122](https://github.com/matrix-org/synapse/issues/4122))
- fix return code of empty key backups ([\#4123](https://github.com/matrix-org/synapse/issues/4123))
- If the typing stream ID goes backwards (as on a worker when the master restarts), the worker's typing handler will no longer erroneously report rooms containing new typing events. ([\#4127](https://github.com/matrix-org/synapse/issues/4127))
- Fix table lock of device_lists_remote_cache which could freeze the application ([\#4132](https://github.com/matrix-org/synapse/issues/4132))
- Fix exception when using state res v2 algorithm ([\#4135](https://github.com/matrix-org/synapse/issues/4135))
- Generating the user consent URI no longer fails on Python 3. ([\#4140](https://github.com/matrix-org/synapse/issues/4140), [\#4163](https://github.com/matrix-org/synapse/issues/4163))
- Loading URL previews from the DB cache on Postgres will no longer cause Unicode type errors when responding to the request, and URL previews will no longer fail if the remote server returns a Content-Type header with the chartype in quotes. ([\#4157](https://github.com/matrix-org/synapse/issues/4157))
- The hash_password script now works on Python 3. ([\#4161](https://github.com/matrix-org/synapse/issues/4161))
- Fix noop checks when updating device keys, reducing spurious device list update notifications. ([\#4164](https://github.com/matrix-org/synapse/issues/4164))
Deprecations and Removals
-------------------------
- The disused and un-specced identicon generator has been removed. ([\#4106](https://github.com/matrix-org/synapse/issues/4106))
- The obsolete and non-functional /pull federation endpoint has been removed. ([\#4118](https://github.com/matrix-org/synapse/issues/4118))
- The deprecated v1 key exchange endpoints have been removed. ([\#4119](https://github.com/matrix-org/synapse/issues/4119))
- Synapse will no longer fetch keys using the fallback deprecated v1 key exchange method and will now always use v2. ([\#4120](https://github.com/matrix-org/synapse/issues/4120))
Internal Changes
----------------
- Fix build of Docker image with docker-compose ([\#3778](https://github.com/matrix-org/synapse/issues/3778))
- Delete unreferenced state groups during history purge ([\#4006](https://github.com/matrix-org/synapse/issues/4006))
- The "Received rdata" log messages on workers is now logged at DEBUG, not INFO. ([\#4108](https://github.com/matrix-org/synapse/issues/4108))
- Reduce replication traffic for device lists ([\#4109](https://github.com/matrix-org/synapse/issues/4109))
- Fix `synapse_replication_tcp_protocol_*_commands` metric label to be full command name, rather than just the first character ([\#4110](https://github.com/matrix-org/synapse/issues/4110))
- Log some bits about room creation ([\#4121](https://github.com/matrix-org/synapse/issues/4121))
- Fix `tox` failure on old systems ([\#4124](https://github.com/matrix-org/synapse/issues/4124))
- Add STATE_V2_TEST room version ([\#4128](https://github.com/matrix-org/synapse/issues/4128))
- Clean up event accesses and tests ([\#4137](https://github.com/matrix-org/synapse/issues/4137))
- The default logging config will now set an explicit log file encoding of UTF-8. ([\#4138](https://github.com/matrix-org/synapse/issues/4138))
- Add helpers functions for getting prev and auth events of an event ([\#4139](https://github.com/matrix-org/synapse/issues/4139))
- Add some tests for the HTTP pusher. ([\#4149](https://github.com/matrix-org/synapse/issues/4149))
- add purge_history.sh and purge_remote_media.sh scripts to contrib/ ([\#4155](https://github.com/matrix-org/synapse/issues/4155))
- HTTP tests have been refactored to contain less boilerplate. ([\#4156](https://github.com/matrix-org/synapse/issues/4156))
- Drop incoming events from federation for unknown rooms ([\#4165](https://github.com/matrix-org/synapse/issues/4165))
Synapse 0.33.8 (2018-11-01)
===========================

37
CONTRIBUTING.rst
View file

@ -74,16 +74,39 @@ entry. These are managed by Towncrier
To create a changelog entry, make a new file in the ``changelog.d``
file named in the format of ``PRnumber.type``. The type can be
one of ``feature``, ``bugfix``, ``removal`` (also used for
deprecations), or ``misc`` (for internal-only changes). The content of
the file is your changelog entry, which can contain Markdown
formatting. Adding credits to the changelog is encouraged, we value
your contributions and would like to have you shouted out in the
release notes!
deprecations), or ``misc`` (for internal-only changes).
The content of the file is your changelog entry, which can contain Markdown
formatting. The entry should end with a full stop ('.') for consistency.
Adding credits to the changelog is encouraged, we value your
contributions and would like to have you shouted out in the release notes!
For example, a fix in PR #1234 would have its changelog entry in
``changelog.d/1234.bugfix``, and contain content like "The security levels of
Florbs are now validated when recieved over federation. Contributed by Jane
Matrix".
Matrix.".
Debian changelog
----------------
Changes which affect the debian packaging files (in ``debian``) are an
exception.
In this case, you will need to add an entry to the debian changelog for the
next release. For this, run the following command::
dch
This will make up a new version number (if there isn't already an unreleased
version in flight), and open an editor where you can add a new changelog entry.
(Our release process will ensure that the version number and maintainer name is
corrected for the release.)
If your change affects both the debian packaging *and* files outside the debian
directory, you will need both a regular newsfragment *and* an entry in the
debian changelog. (Though typically such changes should be submitted as two
separate pull requests.)
Attribution
~~~~~~~~~~~
@ -102,7 +125,7 @@ Sign off
In order to have a concrete record that your contribution is intentional
and you agree to license it under the same terms as the project's license, we've adopted the
same lightweight approach that the Linux Kernel
(https://www.kernel.org/doc/Documentation/SubmittingPatches), Docker
`submitting patches process <https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin>`_, Docker
(https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other
projects use: the DCO (Developer Certificate of Origin:
http://developercertificate.org/). This is a simple declaration that you wrote

430
INSTALL.md Normal file
View file

@ -0,0 +1,430 @@
* [Installing Synapse](#installing-synapse)
* [Installing from source](#installing-from-source)
* [Platform-Specific Instructions](#platform-specific-instructions)
* [Troubleshooting Installation](#troubleshooting-installation)
* [Prebuilt packages](#prebuilt-packages)
* [Setting up Synapse](#setting-up-synapse)
* [TLS certificates](#tls-certificates)
* [Registering a user](#registering-a-user)
* [Setting up a TURN server](#setting-up-a-turn-server)
* [URL previews](#url-previews)
# Installing Synapse
## Installing from source
(Prebuilt packages are available for some platforms - see [Prebuilt packages](#prebuilt-packages).)
System requirements:
- POSIX-compliant system (tested on Linux & OS X)
- Python 3.5, 3.6, 3.7, or 2.7
- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
Synapse is written in Python but some of the libraries it uses are written in
C. So before we can install Synapse itself we need a working C compiler and the
header files for Python C extensions. See [Platform-Specific
Instructions](#platform-specific-instructions) for information on installing
these on various platforms.
To install the Synapse homeserver run:
```
mkdir -p ~/synapse
virtualenv -p python3 ~/synapse/env
source ~/synapse/env/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
pip install matrix-synapse[all]
```
This will download Synapse from [PyPI](https://pypi.org/project/matrix-synapse)
and install it, along with the python libraries it uses, into a virtual environment
under `~/synapse/env`. Feel free to pick a different directory if you
prefer.
This Synapse installation can then be later upgraded by using pip again with the
update flag:
```
source ~/synapse/env/bin/activate
pip install -U matrix-synapse[all]
```
Before you can start Synapse, you will need to generate a configuration
file. To do this, run (in your virtualenv, as before)::
```
cd ~/synapse
python -m synapse.app.homeserver \
--server-name my.domain.name \
--config-path homeserver.yaml \
--generate-config \
--report-stats=[yes|no]
```
... substituting an appropriate value for `--server-name`. The server name
determines the "domain" part of user-ids for users on your server: these will
all be of the format `@user:my.domain.name`. It also determines how other
matrix servers will reach yours for Federation. For a test configuration,
set this to the hostname of your server. For a more production-ready setup, you
will probably want to specify your domain (`example.com`) rather than a
matrix-specific hostname here (in the same way that your email address is
probably `user@example.com` rather than `user@email.example.com`) - but
doing so may require more advanced setup: see [Setting up Federation](docs/federate.md).
Beware that the server name cannot be changed later.
This command will generate you a config file that you can then customise, but it will
also generate a set of keys for you. These keys will allow your Home Server to
identify itself to other Home Servers, so don't lose or delete them. It would be
wise to back them up somewhere safe. (If, for whatever reason, you do need to
change your Home Server's keys, you may find that other Home Servers have the
old key cached. If you update the signing key, you should change the name of the
key in the `<server name>.signing.key` file (the second word) to something
different. See the
[spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys)
for more information on key management.)
You will need to give Synapse a TLS certficate before it will start - see [TLS
certificates](#tls-certificates).
To actually run your new homeserver, pick a working directory for Synapse to
run (e.g. `~/synapse`), and::
cd ~/synapse
source env/bin/activate
synctl start
### Platform-Specific Instructions
#### Debian/Ubuntu/Raspbian
Installing prerequisites on Ubuntu or Debian:
```
sudo apt-get install build-essential python3-dev libffi-dev \
python-pip python-setuptools sqlite3 \
libssl-dev python-virtualenv libjpeg-dev libxslt1-dev
```
#### ArchLinux
Installing prerequisites on ArchLinux:
```
sudo pacman -S base-devel python python-pip \
python-setuptools python-virtualenv sqlite3
```
#### CentOS/Fedora
Installing prerequisites on CentOS 7 or Fedora 25:
```
sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
lcms2-devel libwebp-devel tcl-devel tk-devel redhat-rpm-config \
python-virtualenv libffi-devel openssl-devel
sudo yum groupinstall "Development Tools"
```
#### Mac OS X
Installing prerequisites on Mac OS X:
```
xcode-select --install
sudo easy_install pip
sudo pip install virtualenv
brew install pkg-config libffi
```
#### OpenSUSE
Installing prerequisites on openSUSE:
```
sudo zypper in -t pattern devel_basis
sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
python-devel libffi-devel libopenssl-devel libjpeg62-devel
```
#### OpenBSD
Installing prerequisites on OpenBSD:
```
doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
libxslt jpeg
```
There is currently no port for OpenBSD. Additionally, OpenBSD's security
settings require a slightly more difficult installation process.
XXX: I suspect this is out of date.
1. Create a new directory in `/usr/local` called `_synapse`. Also, create a
new user called `_synapse` and set that directory as the new user's home.
This is required because, by default, OpenBSD only allows binaries which need
write and execute permissions on the same memory space to be run from
`/usr/local`.
2. `su` to the new `_synapse` user and change to their home directory.
3. Create a new virtualenv: `virtualenv -p python2.7 ~/.synapse`
4. Source the virtualenv configuration located at
`/usr/local/_synapse/.synapse/bin/activate`. This is done in `ksh` by
using the `.` command, rather than `bash`'s `source`.
5. Optionally, use `pip` to install `lxml`, which Synapse needs to parse
webpages for their titles.
6. Use `pip` to install this repository: `pip install matrix-synapse`
7. Optionally, change `_synapse`'s shell to `/bin/false` to reduce the
chance of a compromised Synapse server being used to take over your box.
After this, you may proceed with the rest of the install directions.
#### Windows
If you wish to run or develop Synapse on Windows, the Windows Subsystem For
Linux provides a Linux environment on Windows 10 which is capable of using the
Debian, Fedora, or source installation methods. More information about WSL can
be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
for Windows Server.
### Troubleshooting Installation
XXX a bunch of this is no longer relevant.
Synapse requires pip 8 or later, so if your OS provides too old a version you
may need to manually upgrade it::
sudo pip install --upgrade pip
Installing may fail with `Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)`.
You can fix this by manually upgrading pip and virtualenv::
sudo pip install --upgrade virtualenv
You can next rerun `virtualenv -p python3 synapse` to update the virtual env.
Installing may fail during installing virtualenv with `InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.`
You can fix this by manually installing ndg-httpsclient::
pip install --upgrade ndg-httpsclient
Installing may fail with `mock requires setuptools>=17.1. Aborting installation`.
You can fix this by upgrading setuptools::
pip install --upgrade setuptools
If pip crashes mid-installation for reason (e.g. lost terminal), pip may
refuse to run until you remove the temporary installation directory it
created. To reset the installation::
rm -rf /tmp/pip_install_matrix
pip seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.::
pip install twisted
## Prebuilt packages
As an alternative to installing from source, prebuilt packages are available
for a number of platforms.
### Docker images and Ansible playbooks
There is an offical synapse image available at
https://hub.docker.com/r/matrixdotorg/synapse which can be used with
the docker-compose file available at [contrib/docker](contrib/docker). Further information on
this including configuration options is available in the README on
hub.docker.com.
Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
Dockerfile to automate a synapse server in a single Docker image, at
https://hub.docker.com/r/avhost/docker-matrix/tags/
Slavi Pantaleev has created an Ansible playbook,
which installs the offical Docker image of Matrix Synapse
along with many other Matrix-related services (Postgres database, riot-web, coturn, mxisd, SSL support, etc.).
For more details, see
https://github.com/spantaleev/matrix-docker-ansible-deploy
### Debian/Ubuntu
#### Matrix.org packages
Matrix.org provides Debian/Ubuntu packages of the latest stable version of
Synapse via https://matrix.org/packages/debian/. To use them:
```
sudo apt install -y lsb-release curl apt-transport-https
echo "deb https://matrix.org/packages/debian `lsb_release -cs` main" |
sudo tee /etc/apt/sources.list.d/matrix-org.list
curl "https://matrix.org/packages/debian/repo-key.asc" |
sudo apt-key add -
sudo apt update
sudo apt install matrix-synapse-py3
```
#### Downstream Debian/Ubuntu packages
For `buster` and `sid`, Synapse is available in the Debian repositories and
it should be possible to install it with simply:
```
sudo apt install matrix-synapse
```
There is also a version of `matrix-synapse` in `stretch-backports`. Please see
the [Debian documentation on
backports](https://backports.debian.org/Instructions/) for information on how
to use them.
We do not recommend using the packages in downstream Ubuntu at this time, as
they are old and suffer from known security vulnerabilities.
### Fedora
Synapse is in the Fedora repositories as `matrix-synapse`:
```
sudo dnf install matrix-synapse
```
Oleg Girko provides Fedora RPMs at
https://obs.infoserver.lv/project/monitor/matrix-synapse
### OpenSUSE
Synapse is in the OpenSUSE repositories as `matrix-synapse`:
```
sudo zypper install matrix-synapse
```
### SUSE Linux Enterprise Server
Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/
### ArchLinux
The quickest way to get up and running with ArchLinux is probably with the community package
https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of
the necessary dependencies.
pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):
```
sudo pip install --upgrade pip
```
If you encounter an error with lib bcrypt causing an Wrong ELF Class:
ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
compile it under the right architecture. (This should not be needed if
installing under virtualenv):
```
sudo pip uninstall py-bcrypt
sudo pip install py-bcrypt
```
### FreeBSD
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
- Packages: `pkg install py27-matrix-synapse`
### NixOS
Robin Lambertz has packaged Synapse for NixOS at:
https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix
# Setting up Synapse
Once you have installed synapse as above, you will need to configure it.
## TLS certificates
The default configuration exposes a single HTTP port: http://localhost:8008. It
is suitable for local testing, but for any practical use, you will either need
to enable a reverse proxy, or configure Synapse to expose an HTTPS port.
For information on using a reverse proxy, see
[docs/reverse_proxy.rst](docs/reverse_proxy.rst).
To configure Synapse to expose an HTTPS port, you will need to edit
`homeserver.yaml`, as follows:
* First, under the `listeners` section, uncomment the configuration for the
TLS-enabled listener. (Remove the hash sign (`#`) at the start of
each line). The relevant lines are like this:
```
- port: 8448
type: http
tls: true
resources:
- names: [client, federation]
```
* You will also need to uncomment the `tls_certificate_path` and
`tls_private_key_path` lines under the `TLS` section. You can either
point these settings at an existing certificate and key, or you can
enable Synapse's built-in ACME (Let's Encrypt) support. Instructions
for having Synapse automatically provision and renew federation
certificates through ACME can be found at [ACME.md](docs/ACME.md).
For those of you upgrading your TLS certificate in readiness for Synapse 1.0,
please take a look at `our guide <docs/MSC1711_certificates_FAQ.md#configuring-certificates-for-compatibility-with-synapse-100>`_.
## Registering a user
You will need at least one user on your server in order to use a Matrix
client. Users can be registered either via a Matrix client, or via a
commandline script.
To get started, it is easiest to use the command line to register new
users. This can be done as follows:
```
$ source ~/synapse/env/bin/activate
$ synctl start # if not already running
$ register_new_matrix_user -c homeserver.yaml http://localhost:8008
New user localpart: erikj
Password:
Confirm password:
Make admin [no]:
Success!
```
This process uses a setting `registration_shared_secret` in
`homeserver.yaml`, which is shared between Synapse itself and the
`register_new_matrix_user` script. It doesn't matter what it is (a random
value is generated by `--generate-config`), but it should be kept secret, as
anyone with knowledge of it can register users on your server even if
`enable_registration` is `false`.
## Setting up a TURN server
For reliable VoIP calls to be routed via this homeserver, you MUST configure
a TURN server. See [docs/turn-howto.rst](docs/turn-howto.rst) for details.
## URL previews
Synapse includes support for previewing URLs, which is disabled by default. To
turn it on you must enable the `url_preview_enabled: True` config parameter
and explicitly specify the IP ranges that Synapse is not allowed to spider for
previewing in the `url_preview_ip_range_blacklist` configuration parameter.
This is critical from a security perspective to stop arbitrary Matrix users
spidering 'internal' URLs on your network. At the very least we recommend that
your loopback and RFC1918 IP addresses are blacklisted.
This also requires the optional lxml and netaddr python dependencies to be
installed. This in turn requires the libxml2 library to be available - on
Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for
your OS.

5
MANIFEST.in
View file

@ -15,6 +15,7 @@ recursive-include docs *
recursive-include scripts *
recursive-include scripts-dev *
recursive-include synapse *.pyi
recursive-include tests *.pem
recursive-include tests *.py
recursive-include synapse/res *
@ -26,6 +27,7 @@ recursive-include synapse/static *.js
exclude Dockerfile
exclude .dockerignore
exclude test_postgresql.sh
exclude .editorconfig
include pyproject.toml
recursive-include changelog.d *
@ -34,6 +36,9 @@ prune .github
prune demo/etc
prune docker
prune .circleci
prune .coveragerc
prune debian
prune .codecov.yml
exclude jenkins*
recursive-exclude jenkins *.sh

701
README.rst
View file

@ -26,7 +26,6 @@ via IRC bridge at irc://irc.freenode.net/matrix.
Synapse is currently in rapid development, but as of version 0.5 we believe it
is sufficiently stable to be run as an internet-facing service for real usage!
About Matrix
============
@ -81,221 +80,34 @@ Thanks for using Matrix!
Synapse Installation
====================
Synapse is the reference Python/Twisted Matrix homeserver implementation.
.. _federation:
System requirements:
- POSIX-compliant system (tested on Linux & OS X)
- Python 2.7
- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
Installing from source
----------------------
(Prebuilt packages are available for some platforms - see `Platform-Specific
Instructions`_.)
Synapse is written in Python but some of the libraries it uses are written in
C. So before we can install Synapse itself we need a working C compiler and the
header files for Python C extensions.
Installing prerequisites on Ubuntu or Debian::
sudo apt-get install build-essential python2.7-dev libffi-dev \
python-pip python-setuptools sqlite3 \
libssl-dev python-virtualenv libjpeg-dev libxslt1-dev
Installing prerequisites on ArchLinux::
sudo pacman -S base-devel python2 python-pip \
python-setuptools python-virtualenv sqlite3
Installing prerequisites on CentOS 7 or Fedora 25::
sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
lcms2-devel libwebp-devel tcl-devel tk-devel redhat-rpm-config \
python-virtualenv libffi-devel openssl-devel
sudo yum groupinstall "Development Tools"
Installing prerequisites on Mac OS X::
xcode-select --install
sudo easy_install pip
sudo pip install virtualenv
brew install pkg-config libffi
Installing prerequisites on Raspbian::
sudo apt-get install build-essential python2.7-dev libffi-dev \
python-pip python-setuptools sqlite3 \
libssl-dev python-virtualenv libjpeg-dev
sudo pip install --upgrade pip
sudo pip install --upgrade ndg-httpsclient
sudo pip install --upgrade virtualenv
Installing prerequisites on openSUSE::
sudo zypper in -t pattern devel_basis
sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
python-devel libffi-devel libopenssl-devel libjpeg62-devel
Installing prerequisites on OpenBSD::
doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
libxslt
To install the Synapse homeserver run::
virtualenv -p python2.7 ~/.synapse
source ~/.synapse/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
pip install matrix-synapse
This installs Synapse, along with the libraries it uses, into a virtual
environment under ``~/.synapse``. Feel free to pick a different directory
if you prefer.
This Synapse installation can then be later upgraded by using pip again with the
update flag::
source ~/.synapse/bin/activate
pip install -U matrix-synapse
In case of problems, please see the _`Troubleshooting` section below.
There is an offical synapse image available at
https://hub.docker.com/r/matrixdotorg/synapse/tags/ which can be used with
the docker-compose file available at `contrib/docker <contrib/docker>`_. Further information on
this including configuration options is available in the README on
hub.docker.com.
Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
Dockerfile to automate a synapse server in a single Docker image, at
https://hub.docker.com/r/avhost/docker-matrix/tags/
Slavi Pantaleev has created an Ansible playbook,
which installs the offical Docker image of Matrix Synapse
along with many other Matrix-related services (Postgres database, riot-web, coturn, mxisd, SSL support, etc.).
For more details, see
https://github.com/spantaleev/matrix-docker-ansible-deploy
Configuring Synapse
-------------------
Before you can start Synapse, you will need to generate a configuration
file. To do this, run (in your virtualenv, as before)::
cd ~/.synapse
python -m synapse.app.homeserver \
--server-name my.domain.name \
--config-path homeserver.yaml \
--generate-config \
--report-stats=[yes|no]
... substituting an appropriate value for ``--server-name``. The server name
determines the "domain" part of user-ids for users on your server: these will
all be of the format ``@user:my.domain.name``. It also determines how other
matrix servers will reach yours for `Federation`_. For a test configuration,
set this to the hostname of your server. For a more production-ready setup, you
will probably want to specify your domain (``example.com``) rather than a
matrix-specific hostname here (in the same way that your email address is
probably ``user@example.com`` rather than ``user@email.example.com``) - but
doing so may require more advanced setup - see `Setting up
Federation`_. Beware that the server name cannot be changed later.
This command will generate you a config file that you can then customise, but it will
also generate a set of keys for you. These keys will allow your Home Server to
identify itself to other Home Servers, so don't lose or delete them. It would be
wise to back them up somewhere safe. (If, for whatever reason, you do need to
change your Home Server's keys, you may find that other Home Servers have the
old key cached. If you update the signing key, you should change the name of the
key in the ``<server name>.signing.key`` file (the second word) to something
different. See `the spec`__ for more information on key management.)
.. __: `key_management`_
The default configuration exposes two HTTP ports: 8008 and 8448. Port 8008 is
configured without TLS; it should be behind a reverse proxy for TLS/SSL
termination on port 443 which in turn should be used for clients. Port 8448
is configured to use TLS with a self-signed certificate. If you would like
to do initial test with a client without having to setup a reverse proxy,
you can temporarly use another certificate. (Note that a self-signed
certificate is fine for `Federation`_). You can do so by changing
``tls_certificate_path``, ``tls_private_key_path`` and ``tls_dh_params_path``
in ``homeserver.yaml``; alternatively, you can use a reverse-proxy, but be sure
to read `Using a reverse proxy with Synapse`_ when doing so.
Apart from port 8448 using TLS, both ports are the same in the default
configuration.
Registering a user
------------------
You will need at least one user on your server in order to use a Matrix
client. Users can be registered either `via a Matrix client`__, or via a
commandline script.
.. __: `client-user-reg`_
To get started, it is easiest to use the command line to register new users::
$ source ~/.synapse/bin/activate
$ synctl start # if not already running
$ register_new_matrix_user -c homeserver.yaml https://localhost:8448
New user localpart: erikj
Password:
Confirm password:
Make admin [no]:
Success!
This process uses a setting ``registration_shared_secret`` in
``homeserver.yaml``, which is shared between Synapse itself and the
``register_new_matrix_user`` script. It doesn't matter what it is (a random
value is generated by ``--generate-config``), but it should be kept secret, as
anyone with knowledge of it can register users on your server even if
``enable_registration`` is ``false``.
Setting up a TURN server
------------------------
For reliable VoIP calls to be routed via this homeserver, you MUST configure
a TURN server. See `<docs/turn-howto.rst>`_ for details.
Running Synapse
===============
To actually run your new homeserver, pick a working directory for Synapse to
run (e.g. ``~/.synapse``), and::
cd ~/.synapse
source ./bin/activate
synctl start
* For details on how to install synapse, see `<INSTALL.md>`_.
* For specific details on how to configure Synapse for federation see `docs/federate.md <docs/federate.md>`_
Connecting to Synapse from a client
===================================
The easiest way to try out your new Synapse installation is by connecting to it
from a web client. The easiest option is probably the one at
https://riot.im/app. You will need to specify a "Custom server" when you log on
or register: set this to ``https://domain.tld`` if you setup a reverse proxy
following the recommended setup, or ``https://localhost:8448`` - remember to specify the
port (``:8448``) if not ``:443`` unless you changed the configuration. (Leave the identity
server as the default - see `Identity servers`_.)
from a web client.
If using port 8448 you will run into errors until you accept the self-signed
certificate. You can easily do this by going to ``https://localhost:8448``
directly with your browser and accept the presented certificate. You can then
go back in your web client and proceed further.
Unless you are running a test instance of Synapse on your local machine, in
general, you will need to enable TLS support before you can successfully
connect from a client: see `<INSTALL.md#tls-certificates>`_.
An easy way to get started is to login or register via Riot at
https://riot.im/app/#/login or https://riot.im/app/#/register respectively.
You will need to change the server you are logging into from ``matrix.org``
and instead specify a Homeserver URL of ``https://<server_name>:8448``
(or just ``https://<server_name>`` if you are using a reverse proxy).
(Leave the identity server as the default - see `Identity servers`_.)
If you prefer to use another client, refer to our
`client breakdown <https://matrix.org/docs/projects/clients-matrix>`_.
If all goes well you should at least be able to log in, create a room, and
start sending messages.
(The homeserver runs a web client by default at https://localhost:8448/, though
as of the time of writing it is somewhat outdated and not really recommended -
https://github.com/matrix-org/synapse/issues/1527).
.. _`client-user-reg`:
Registering a new user from a client
@ -319,6 +131,12 @@ create the account. Your name will take the form of::
As when logging in, you will need to specify a "Custom server". Specify your
desired ``localpart`` in the 'User name' box.
ACME setup
==========
For details on having Synapse manage your federation TLS certificates
automatically, please see `<docs/ACME.md>`_.
Security Note
=============
@ -333,208 +151,10 @@ content served to web browsers a matrix API from being able to attack webapps ho
on the same domain. This is particularly true of sharing a matrix webclient and
server on the same domain.
See https://github.com/vector-im/vector-web/issues/1977 and
See https://github.com/vector-im/riot-web/issues/1977 and
https://developer.github.com/changes/2014-04-25-user-content-security for more details.
Platform-Specific Instructions
==============================
Debian
------
Matrix provides official Debian packages via apt from https://matrix.org/packages/debian/.
Note that these packages do not include a client - choose one from
https://matrix.org/docs/projects/try-matrix-now.html (or build your own with one of our SDKs :)
Fedora
------
Synapse is in the Fedora repositories as ``matrix-synapse``::
sudo dnf install matrix-synapse
Oleg Girko provides Fedora RPMs at
https://obs.infoserver.lv/project/monitor/matrix-synapse
OpenSUSE
--------
Synapse is in the OpenSUSE repositories as ``matrix-synapse``::
sudo zypper install matrix-synapse
SUSE Linux Enterprise Server
----------------------------
Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/
ArchLinux
---------
The quickest way to get up and running with ArchLinux is probably with the community package
https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of
the necessary dependencies. If the default web client is to be served (enabled by default in
the generated config),
https://www.archlinux.org/packages/community/any/python2-matrix-angular-sdk/ will also need to
be installed.
Alternatively, to install using pip a few changes may be needed as ArchLinux
defaults to python 3, but synapse currently assumes python 2.7 by default:
pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 )::
sudo pip2.7 install --upgrade pip
You also may need to explicitly specify python 2.7 again during the install
request::
pip2.7 install https://github.com/matrix-org/synapse/tarball/master
If you encounter an error with lib bcrypt causing an Wrong ELF Class:
ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
compile it under the right architecture. (This should not be needed if
installing under virtualenv)::
sudo pip2.7 uninstall py-bcrypt
sudo pip2.7 install py-bcrypt
During setup of Synapse you need to call python2.7 directly again::
cd ~/.synapse
python2.7 -m synapse.app.homeserver \
--server-name machine.my.domain.name \
--config-path homeserver.yaml \
--generate-config
...substituting your host and domain name as appropriate.
FreeBSD
-------
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: ``cd /usr/ports/net-im/py-matrix-synapse && make install clean``
- Packages: ``pkg install py27-matrix-synapse``
OpenBSD
-------
There is currently no port for OpenBSD. Additionally, OpenBSD's security
settings require a slightly more difficult installation process.
1) Create a new directory in ``/usr/local`` called ``_synapse``. Also, create a
new user called ``_synapse`` and set that directory as the new user's home.
This is required because, by default, OpenBSD only allows binaries which need
write and execute permissions on the same memory space to be run from
``/usr/local``.
2) ``su`` to the new ``_synapse`` user and change to their home directory.
3) Create a new virtualenv: ``virtualenv -p python2.7 ~/.synapse``
4) Source the virtualenv configuration located at
``/usr/local/_synapse/.synapse/bin/activate``. This is done in ``ksh`` by
using the ``.`` command, rather than ``bash``'s ``source``.
5) Optionally, use ``pip`` to install ``lxml``, which Synapse needs to parse
webpages for their titles.
6) Use ``pip`` to install this repository: ``pip install matrix-synapse``
7) Optionally, change ``_synapse``'s shell to ``/bin/false`` to reduce the
chance of a compromised Synapse server being used to take over your box.
After this, you may proceed with the rest of the install directions.
NixOS
-----
Robin Lambertz has packaged Synapse for NixOS at:
https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix
Windows Install
---------------
If you wish to run or develop Synapse on Windows, the Windows Subsystem For
Linux provides a Linux environment on Windows 10 which is capable of using the
Debian, Fedora, or source installation methods. More information about WSL can
be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
for Windows Server.
Troubleshooting
===============
Troubleshooting Installation
----------------------------
Synapse requires pip 8 or later, so if your OS provides too old a version you
may need to manually upgrade it::
sudo pip install --upgrade pip
Installing may fail with ``Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)``.
You can fix this by manually upgrading pip and virtualenv::
sudo pip install --upgrade virtualenv
You can next rerun ``virtualenv -p python2.7 synapse`` to update the virtual env.
Installing may fail during installing virtualenv with ``InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.``
You can fix this by manually installing ndg-httpsclient::
pip install --upgrade ndg-httpsclient
Installing may fail with ``mock requires setuptools>=17.1. Aborting installation``.
You can fix this by upgrading setuptools::
pip install --upgrade setuptools
If pip crashes mid-installation for reason (e.g. lost terminal), pip may
refuse to run until you remove the temporary installation directory it
created. To reset the installation::
rm -rf /tmp/pip_install_matrix
pip seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.::
pip install twisted
Running out of File Handles
~~~~~~~~~~~~~~~~~~~~~~~~~~~
If synapse runs out of filehandles, it typically fails badly - live-locking
at 100% CPU, and/or failing to accept new TCP connections (blocking the
connecting client). Matrix currently can legitimately use a lot of file handles,
thanks to busy rooms like #matrix:matrix.org containing hundreds of participating
servers. The first time a server talks in a room it will try to connect
simultaneously to all participating servers, which could exhaust the available
file descriptors between DNS queries & HTTPS sockets, especially if DNS is slow
to respond. (We need to improve the routing algorithm used to be better than
full mesh, but as of June 2017 this hasn't happened yet).
If you hit this failure mode, we recommend increasing the maximum number of
open file handles to be at least 4096 (assuming a default of 1024 or 256).
This is typically done by editing ``/etc/security/limits.conf``
Separately, Synapse may leak file handles if inbound HTTP requests get stuck
during processing - e.g. blocked behind a lock or talking to a remote server etc.
This is best diagnosed by matching up the 'Received request' and 'Processed request'
log lines and looking for any 'Processed request' lines which take more than
a few seconds to execute. Please let us know at #matrix-dev:matrix.org if
you see this failure mode so we can help debug it, however.
ArchLinux
~~~~~~~~~
If running `$ synctl start` fails with 'returned non-zero exit status 1',
you will need to explicitly call Python2.7 - either running as::
python2.7 -m synapse.app.homeserver --daemonize -c homeserver.yaml
...or by editing synctl with the correct python executable.
Upgrading an existing Synapse
=============================
@ -544,100 +164,19 @@ versions of synapse.
.. _UPGRADE.rst: UPGRADE.rst
.. _federation:
Setting up Federation
=====================
Federation is the process by which users on different servers can participate
in the same room. For this to work, those other servers must be able to contact
yours to send messages.
As explained in `Configuring synapse`_, the ``server_name`` in your
``homeserver.yaml`` file determines the way that other servers will reach
yours. By default, they will treat it as a hostname and try to connect to
port 8448. This is easy to set up and will work with the default configuration,
provided you set the ``server_name`` to match your machine's public DNS
hostname.
For a more flexible configuration, you can set up a DNS SRV record. This allows
you to run your server on a machine that might not have the same name as your
domain name. For example, you might want to run your server at
``synapse.example.com``, but have your Matrix user-ids look like
``@user:example.com``. (A SRV record also allows you to change the port from
the default 8448. However, if you are thinking of using a reverse-proxy on the
federation port, which is not recommended, be sure to read
`Reverse-proxying the federation port`_ first.)
To use a SRV record, first create your SRV record and publish it in DNS. This
should have the format ``_matrix._tcp.<yourdomain.com> <ttl> IN SRV 10 0 <port>
<synapse.server.name>``. The DNS record should then look something like::
$ dig -t srv _matrix._tcp.example.com
_matrix._tcp.example.com. 3600 IN SRV 10 0 8448 synapse.example.com.
Note that the server hostname cannot be an alias (CNAME record): it has to point
directly to the server hosting the synapse instance.
You can then configure your homeserver to use ``<yourdomain.com>`` as the domain in
its user-ids, by setting ``server_name``::
python -m synapse.app.homeserver \
--server-name <yourdomain.com> \
--config-path homeserver.yaml \
--generate-config
python -m synapse.app.homeserver --config-path homeserver.yaml
If you've already generated the config file, you need to edit the ``server_name``
in your ``homeserver.yaml`` file. If you've already started Synapse and a
database has been created, you will have to recreate the database.
If all goes well, you should be able to `connect to your server with a client`__,
and then join a room via federation. (Try ``#matrix-dev:matrix.org`` as a first
step. "Matrix HQ"'s sheer size and activity level tends to make even the
largest boxes pause for thought.)
.. __: `Connecting to Synapse from a client`_
Troubleshooting
---------------
You can use the federation tester to check if your homeserver is all set:
``https://matrix.org/federationtester/api/report?server_name=<your_server_name>``
If any of the attributes under "checks" is false, federation won't work.
The typical failure mode with federation is that when you try to join a room,
it is rejected with "401: Unauthorized". Generally this means that other
servers in the room couldn't access yours. (Joining a room over federation is a
complicated dance which requires connections in both directions).
So, things to check are:
* If you are trying to use a reverse-proxy, read `Reverse-proxying the
federation port`_.
* If you are not using a SRV record, check that your ``server_name`` (the part
of your user-id after the ``:``) matches your hostname, and that port 8448 on
that hostname is reachable from outside your network.
* If you *are* using a SRV record, check that it matches your ``server_name``
(it should be ``_matrix._tcp.<server_name>``), and that the port and hostname
it specifies are reachable from outside your network.
Running a Demo Federation of Synapses
-------------------------------------
If you want to get up and running quickly with a trio of homeservers in a
private federation, there is a script in the ``demo`` directory. This is mainly
useful just for development purposes. See `<demo/README>`_.
Using PostgreSQL
================
As of Synapse 0.9, `PostgreSQL <https://www.postgresql.org>`_ is supported as an
alternative to the `SQLite <https://sqlite.org/>`_ database that Synapse has
traditionally used for convenience and simplicity.
Synapse offers two database engines:
* `SQLite <https://sqlite.org/>`_
* `PostgreSQL <https://www.postgresql.org>`_
The advantages of Postgres include:
By default Synapse uses SQLite in and doing so trades performance for convenience.
SQLite is only recommended in Synapse for testing purposes or for servers with
light workloads.
Almost all installations should opt to use PostreSQL. Advantages include:
* significant performance improvements due to the superior threading and
caching model, smarter query optimiser
@ -649,7 +188,6 @@ The advantages of Postgres include:
For information on how to install and use PostgreSQL, please see
`docs/postgres.rst <docs/postgres.rst>`_.
.. _reverse-proxy:
Using a reverse proxy with Synapse
@ -663,117 +201,7 @@ It is recommended to put a reverse proxy such as
doing so is that it means that you can expose the default https port (443) to
Matrix clients without needing to run Synapse with root privileges.
The most important thing to know here is that Matrix clients and other Matrix
servers do not necessarily need to connect to your server via the same
port. Indeed, clients will use port 443 by default, whereas servers default to
port 8448. Where these are different, we refer to the 'client port' and the
'federation port'.
The next most important thing to know is that using a reverse-proxy on the
federation port has a number of pitfalls. It is possible, but be sure to read
`Reverse-proxying the federation port`_.
The recommended setup is therefore to configure your reverse-proxy on port 443
to port 8008 of synapse for client connections, but to also directly expose port
8448 for server-server connections. All the Matrix endpoints begin ``/_matrix``,
so an example nginx configuration might look like::
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name matrix.example.com;
location /_matrix {
proxy_pass http://localhost:8008;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
an example Caddy configuration might look like::
matrix.example.com {
proxy /_matrix http://localhost:8008 {
transparent
}
}
and an example Apache configuration might look like::
<VirtualHost *:443>
SSLEngine on
ServerName matrix.example.com;
<Location /_matrix>
ProxyPass http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse http://127.0.0.1:8008/_matrix
</Location>
</VirtualHost>
You will also want to set ``bind_addresses: ['127.0.0.1']`` and ``x_forwarded: true``
for port 8008 in ``homeserver.yaml`` to ensure that client IP addresses are
recorded correctly.
Having done so, you can then use ``https://matrix.example.com`` (instead of
``https://matrix.example.com:8448``) as the "Custom server" when `Connecting to
Synapse from a client`_.
Reverse-proxying the federation port
------------------------------------
There are two issues to consider before using a reverse-proxy on the federation
port:
* Due to the way SSL certificates are managed in the Matrix federation protocol
(see `spec`__), Synapse needs to be configured with the path to the SSL
certificate, *even if you do not terminate SSL at Synapse*.
.. __: `key_management`_
* Synapse does not currently support SNI on the federation protocol
(`bug #1491 <https://github.com/matrix-org/synapse/issues/1491>`_), which
means that using name-based virtual hosting is unreliable.
Furthermore, a number of the normal reasons for using a reverse-proxy do not
apply:
* Other servers will connect on port 8448 by default, so there is no need to
listen on port 443 (for federation, at least), which avoids the need for root
privileges and virtual hosting.
* A self-signed SSL certificate is fine for federation, so there is no need to
automate renewals. (The certificate generated by ``--generate-config`` is
valid for 10 years.)
If you want to set up a reverse-proxy on the federation port despite these
caveats, you will need to do the following:
* In ``homeserver.yaml``, set ``tls_certificate_path`` to the path to the SSL
certificate file used by your reverse-proxy, and set ``no_tls`` to ``True``.
(``tls_private_key_path`` will be ignored if ``no_tls`` is ``True``.)
* In your reverse-proxy configuration:
* If there are other virtual hosts on the same port, make sure that the
*default* one uses the certificate configured above.
* Forward ``/_matrix`` to Synapse.
* If your reverse-proxy is not listening on port 8448, publish a SRV record to
tell other servers how to find you. See `Setting up Federation`_.
When updating the SSL certificate, just update the file pointed to by
``tls_certificate_path``: there is no need to restart synapse. (You may like to
use a symbolic link to help make this process atomic.)
The most common mistake when setting up federation is not to tell Synapse about
your SSL certificate. To check it, you can visit
``https://matrix.org/federationtester/api/report?server_name=<your_server_name>``.
Unfortunately, there is no UI for this yet, but, you should see
``"MatchingTLSFingerprint": true``. If not, check that
``Certificates[0].SHA256Fingerprint`` (the fingerprint of the certificate
presented by your reverse-proxy) matches ``Keys.tls_fingerprints[0].sha256``
(the fingerprint of the certificate Synapse is using).
For information on configuring one, see `<docs/reverse_proxy.rst>`_.
Identity Servers
================
@ -805,36 +233,17 @@ an email address with your account, or send an invite to another user via their
email address.
URL Previews
============
Synapse 0.15.0 introduces a new API for previewing URLs at
``/_matrix/media/r0/preview_url``. This is disabled by default. To turn it on
you must enable the ``url_preview_enabled: True`` config parameter and
explicitly specify the IP ranges that Synapse is not allowed to spider for
previewing in the ``url_preview_ip_range_blacklist`` configuration parameter.
This is critical from a security perspective to stop arbitrary Matrix users
spidering 'internal' URLs on your network. At the very least we recommend that
your loopback and RFC1918 IP addresses are blacklisted.
This also requires the optional lxml and netaddr python dependencies to be
installed. This in turn requires the libxml2 library to be available - on
Debian/Ubuntu this means ``apt-get install libxml2-dev``, or equivalent for
your OS.
Password reset
==============
If a user has registered an email address to their account using an identity
server, they can request a password-reset token via clients such as Vector.
server, they can request a password-reset token via clients such as Riot.
A manual password reset can be done via direct database access as follows.
First calculate the hash of the new password::
$ source ~/.synapse/bin/activate
$ ./scripts/hash_password
$ ~/synapse/env/bin/hash_password
Password:
Confirm password:
$2a$12$xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
@ -850,7 +259,7 @@ Synapse Development
Before setting up a development environment for synapse, make sure you have the
system dependencies (such as the python header files) installed - see
`Installing from source`_.
`Installing from source <INSTALL.md#installing-from-source>`_.
To check out a synapse for development, clone the git repo into a working
directory of your choice::
@ -861,10 +270,9 @@ directory of your choice::
Synapse has a number of external dependencies, that are easiest
to install using pip and a virtualenv::
virtualenv -p python2.7 env
virtualenv -p python3 env
source env/bin/activate
python -m synapse.python_dependencies | xargs pip install
pip install lxml mock
python -m pip install -e .[all]
This will run a process of downloading and installing all the needed
dependencies into a virtual env.
@ -872,7 +280,7 @@ dependencies into a virtual env.
Once this is done, you may wish to run Synapse's unit tests, to
check that everything is installed as it should be::
PYTHONPATH="." trial tests
python -m twisted.trial tests
This should end with a 'PASSED' result::
@ -905,13 +313,39 @@ Building internal API documentation::
python setup.py build_sphinx
Troubleshooting
===============
Running out of File Handles
---------------------------
If synapse runs out of file handles, it typically fails badly - live-locking
at 100% CPU, and/or failing to accept new TCP connections (blocking the
connecting client). Matrix currently can legitimately use a lot of file handles,
thanks to busy rooms like #matrix:matrix.org containing hundreds of participating
servers. The first time a server talks in a room it will try to connect
simultaneously to all participating servers, which could exhaust the available
file descriptors between DNS queries & HTTPS sockets, especially if DNS is slow
to respond. (We need to improve the routing algorithm used to be better than
full mesh, but as of March 2019 this hasn't happened yet).
If you hit this failure mode, we recommend increasing the maximum number of
open file handles to be at least 4096 (assuming a default of 1024 or 256).
This is typically done by editing ``/etc/security/limits.conf``
Separately, Synapse may leak file handles if inbound HTTP requests get stuck
during processing - e.g. blocked behind a lock or talking to a remote server etc.
This is best diagnosed by matching up the 'Received request' and 'Processed request'
log lines and looking for any 'Processed request' lines which take more than
a few seconds to execute. Please let us know at #synapse:matrix.org if
you see this failure mode so we can help debug it, however.
Help!! Synapse eats all my RAM!
===============================
-------------------------------
Synapse's architecture is quite RAM hungry currently - we deliberately
cache a lot of recent room data and metadata in RAM in order to speed up
common requests. We'll improve this in future, but for now the easiest
common requests. We'll improve this in the future, but for now the easiest
way to either reduce the RAM usage (at the risk of slowing things down)
is to set the almost-undocumented ``SYNAPSE_CACHE_FACTOR`` environment
variable. The default is 0.5, which can be decreased to reduce RAM usage
@ -927,4 +361,5 @@ by installing the ``libjemalloc1`` package and adding this line to
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
.. _`key_management`: https://matrix.org/docs/spec/server_server/unstable.html#retrieving-server-keys
This can make a significant difference on Python 2.7 - it's unclear how
much of an improvement it provides on Python 3.x.

105
UPGRADE.rst
View file

@ -5,20 +5,20 @@ Before upgrading check if any special steps are required to upgrade from the
what you currently have installed to current version of synapse. The extra
instructions that may be required are listed later in this document.
1. If synapse was installed in a virtualenv then active that virtualenv before
upgrading. If synapse is installed in a virtualenv in ``~/.synapse/`` then
1. If synapse was installed in a virtualenv then activate that virtualenv before
upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then
run:
.. code:: bash
source ~/.synapse/bin/activate
source ~/synapse/env/bin/activate
2. If synapse was installed using pip then upgrade to the latest version by
running:
.. code:: bash
pip install --upgrade --process-dependency-links matrix-synapse
pip install --upgrade matrix-synapse[all]
# restart synapse
synctl restart
@ -31,14 +31,15 @@ instructions that may be required are listed later in this document.
# Pull the latest version of the master branch.
git pull
# Update the versions of synapse's python dependencies.
python synapse/python_dependencies.py | xargs pip install --upgrade
# Update synapse and its python dependencies.
pip install --upgrade .[all]
# restart synapse
./synctl restart
To check whether your update was sucessful, you can check the Server header
To check whether your update was successful, you can check the Server header
returned by the Client-Server API:
.. code:: bash
@ -48,6 +49,96 @@ returned by the Client-Server API:
# configured on port 443.
curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
Upgrading to v0.99.0
====================
Please be aware that, before Synapse v1.0 is released around March 2019, you
will need to replace any self-signed certificates with those verified by a
root CA. Information on how to do so can be found at `the ACME docs
<docs/ACME.md>`_.
For more information on configuring TLS certificates see the `FAQ <docs/MSC1711_certificates_FAQ.md>`_.
Upgrading to v0.34.0
====================
1. This release is the first to fully support Python 3. Synapse will now run on
Python versions 3.5, or 3.6 (as well as 2.7). We recommend switching to
Python 3, as it has been shown to give performance improvements.
For users who have installed Synapse into a virtualenv, we recommend doing
this by creating a new virtualenv. For example::
virtualenv -p python3 ~/synapse/env3
source ~/synapse/env3/bin/activate
pip install matrix-synapse
You can then start synapse as normal, having activated the new virtualenv::
cd ~/synapse
source env3/bin/activate
synctl start
Users who have installed from distribution packages should see the relevant
package documentation. See below for notes on Debian packages.
* When upgrading to Python 3, you **must** make sure that your log files are
configured as UTF-8, by adding ``encoding: utf8`` to the
``RotatingFileHandler`` configuration (if you have one) in your
``<server>.log.config`` file. For example, if your ``log.config`` file
contains::
handlers:
file:
class: logging.handlers.RotatingFileHandler
formatter: precise
filename: homeserver.log
maxBytes: 104857600
backupCount: 10
filters: [context]
console:
class: logging.StreamHandler
formatter: precise
filters: [context]
Then you should update this to be::
handlers:
file:
class: logging.handlers.RotatingFileHandler
formatter: precise
filename: homeserver.log
maxBytes: 104857600
backupCount: 10
filters: [context]
encoding: utf8
console:
class: logging.StreamHandler
formatter: precise
filters: [context]
There is no need to revert this change if downgrading to Python 2.
We are also making available Debian packages which will run Synapse on
Python 3. You can switch to these packages with ``apt-get install
matrix-synapse-py3``, however, please read `debian/NEWS
<https://github.com/matrix-org/synapse/blob/release-v0.34.0/debian/NEWS>`_
before doing so. The existing ``matrix-synapse`` packages will continue to
use Python 2 for the time being.
2. This release removes the ``riot.im`` from the default list of trusted
identity servers.
If ``riot.im`` is in your homeserver's list of
``trusted_third_party_id_servers``, you should remove it. It was added in
case a hypothetical future identity server was put there. If you don't
remove it, users may be unable to deactivate their accounts.
3. This release no longer installs the (unmaintained) Matrix Console web client
as part of the default installation. It is possible to re-enable it by
installing it separately and setting the ``web_client_location`` config
option, but please consider switching to another client.
Upgrading to v0.33.7
====================

1
changelog.d/4832.misc Normal file
View file

@ -0,0 +1 @@
Improve federation documentation, specifically .well-known support. Many thanks to @vaab.

10
contrib/docker/docker-compose.yml
View file

@ -6,9 +6,11 @@ version: '3'
services:
synapse:
build: ../..
build:
context: ../..
dockerfile: docker/Dockerfile
image: docker.io/matrixdotorg/synapse:latest
# Since snyapse does not retry to connect to the database, restart upon
# Since synapse does not retry to connect to the database, restart upon
# failure
restart: unless-stopped
# See the readme for a full documentation of the environment settings
@ -35,7 +37,7 @@ services:
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:my.matrix.Host
- traefik.port=8448
- traefik.port=8008
db:
image: docker.io/postgres:10-alpine
@ -47,4 +49,4 @@ services:
# You may store the database tables in a local folder..
- ./schemas:/var/lib/postgresql/data
# .. or store them on some high performance storage for better results
# - /path/to/ssd/storage:/var/lib/postfesql/data
# - /path/to/ssd/storage:/var/lib/postgresql/data

7
contrib/prometheus/README → contrib/prometheus/README.md
View file

@ -6,8 +6,10 @@ To use it, first install prometheus by following the instructions at
http://prometheus.io/
### for Prometheus v1
Add a new job to the main prometheus.conf file:
```yaml
job: {
name: "synapse"
@ -15,10 +17,12 @@ Add a new job to the main prometheus.conf file:
target: "http://SERVER.LOCATION.HERE:PORT/_synapse/metrics"
}
}
```
### for Prometheus v2
Add a new job to the main prometheus.yml file:
```yaml
- job_name: "synapse"
metrics_path: "/_synapse/metrics"
# when endpoint uses https:
@ -26,11 +30,14 @@ Add a new job to the main prometheus.yml file:
static_configs:
- targets: ['SERVER.LOCATION:PORT']
```
To use `synapse.rules` add
```yaml
rule_files:
- "/PATH/TO/synapse-v2.rules"
```
Metrics are disabled by default when running synapse; they must be enabled
with the 'enable-metrics' option, either in the synapse config file or as a

16
contrib/purge_api/README.md Normal file
View file

@ -0,0 +1,16 @@
Purge history API examples
==========================
# `purge_history.sh`
A bash file, that uses the [purge history API](/docs/admin_api/README.rst) to
purge all messages in a list of rooms up to a certain event. You can select a
timeframe or a number of messages that you want to keep in the room.
Just configure the variables DOMAIN, ADMIN, ROOMS_ARRAY and TIME at the top of
the script.
# `purge_remote_media.sh`
A bash file, that uses the [purge history API](/docs/admin_api/README.rst) to
purge all old cached remote media.

141
contrib/purge_api/purge_history.sh Normal file
View file

@ -0,0 +1,141 @@
#!/bin/bash
# this script will use the api:
# https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.rst
#
# It will purge all messages in a list of rooms up to a cetrain event
###################################################################################################
# define your domain and admin user
###################################################################################################
# add this user as admin in your home server:
DOMAIN=yourserver.tld
# add this user as admin in your home server:
ADMIN="@you_admin_username:$DOMAIN"
API_URL="$DOMAIN:8008/_matrix/client/r0"
###################################################################################################
#choose the rooms to prune old messages from (add a free comment at the end)
###################################################################################################
# the room_id's you can get e.g. from your Riot clients "View Source" button on each message
ROOMS_ARRAY=(
'!DgvjtOljKujDBrxyHk:matrix.org#riot:matrix.org'
'!QtykxKocfZaZOUrTwp:matrix.org#Matrix HQ'
)
# ALTERNATIVELY:
# you can select all the rooms that are not encrypted and loop over the result:
# SELECT room_id FROM rooms WHERE room_id NOT IN (SELECT DISTINCT room_id FROM events WHERE type ='m.room.encrypted')
# or
# select all rooms with at least 100 members:
# SELECT q.room_id FROM (select count(*) as numberofusers, room_id FROM current_state_events WHERE type ='m.room.member'
# GROUP BY room_id) AS q LEFT JOIN room_aliases a ON q.room_id=a.room_id WHERE q.numberofusers > 100 ORDER BY numberofusers desc
###################################################################################################
# evaluate the EVENT_ID before which should be pruned
###################################################################################################
# choose a time before which the messages should be pruned:
TIME='12 months ago'
# ALTERNATIVELY:
# a certain time:
# TIME='2016-08-31 23:59:59'
# creates a timestamp from the given time string:
UNIX_TIMESTAMP=$(date +%s%3N --date='TZ="UTC+2" '"$TIME")
# ALTERNATIVELY:
# prune all messages that are older than 1000 messages ago:
# LAST_MESSAGES=1000
# SQL_GET_EVENT="SELECT event_id from events WHERE type='m.room.message' AND room_id ='$ROOM' ORDER BY received_ts DESC LIMIT 1 offset $(($LAST_MESSAGES - 1))"
# ALTERNATIVELY:
# select the EVENT_ID manually:
#EVENT_ID='$1471814088343495zpPNI:matrix.org' # an example event from 21st of Aug 2016 by Matthew
###################################################################################################
# make the admin user a server admin in the database with
###################################################################################################
# psql -A -t --dbname=synapse -c "UPDATE users SET admin=1 WHERE name LIKE '$ADMIN'"
###################################################################################################
# database function
###################################################################################################
sql (){
# for sqlite3:
#sqlite3 homeserver.db "pragma busy_timeout=20000;$1" | awk '{print $2}'
# for postgres:
psql -A -t --dbname=synapse -c "$1" | grep -v 'Pager'
}
###################################################################################################
# get an access token
###################################################################################################
# for example externally by watching Riot in your browser's network inspector
# or internally on the server locally, use this:
TOKEN=$(sql "SELECT token FROM access_tokens WHERE user_id='$ADMIN' ORDER BY id DESC LIMIT 1")
AUTH="Authorization: Bearer $TOKEN"
###################################################################################################
# check, if your TOKEN works. For example this works:
###################################################################################################
# $ curl --header "$AUTH" "$API_URL/rooms/$ROOM/state/m.room.power_levels"
###################################################################################################
# finally start pruning the room:
###################################################################################################
POSTDATA='{"delete_local_events":"true"}' # this will really delete local events, so the messages in the room really disappear unless they are restored by remote federation
for ROOM in "${ROOMS_ARRAY[@]}"; do
echo "########################################### $(date) ################# "
echo "pruning room: $ROOM ..."
ROOM=${ROOM%#*}
#set -x
echo "check for alias in db..."
# for postgres:
sql "SELECT * FROM room_aliases WHERE room_id='$ROOM'"
echo "get event..."
# for postgres:
EVENT_ID=$(sql "SELECT event_id FROM events WHERE type='m.room.message' AND received_ts<'$UNIX_TIMESTAMP' AND room_id='$ROOM' ORDER BY received_ts DESC LIMIT 1;")
if [ "$EVENT_ID" == "" ]; then
echo "no event $TIME"
else
echo "event: $EVENT_ID"
SLEEP=2
set -x
# call purge
OUT=$(curl --header "$AUTH" -s -d $POSTDATA POST "$API_URL/admin/purge_history/$ROOM/$EVENT_ID")
PURGE_ID=$(echo "$OUT" |grep purge_id|cut -d'"' -f4 )
if [ "$PURGE_ID" == "" ]; then
# probably the history purge is already in progress for $ROOM
: "continuing with next room"
else
while : ; do
# get status of purge and sleep longer each time if still active
sleep $SLEEP
STATUS=$(curl --header "$AUTH" -s GET "$API_URL/admin/purge_history_status/$PURGE_ID" |grep status|cut -d'"' -f4)
: "$ROOM --> Status: $STATUS"
[[ "$STATUS" == "active" ]] || break
SLEEP=$((SLEEP + 1))
done
fi
set +x
sleep 1
fi
done
###################################################################################################
# additionally
###################################################################################################
# to benefit from pruning large amounts of data, you need to call VACUUM to free the unused space.
# This can take a very long time (hours) and the client have to be stopped while you do so:
# $ synctl stop
# $ sqlite3 -line homeserver.db "vacuum;"
# $ synctl start
# This could be set, so you don't need to prune every time after deleting some rows:
# $ sqlite3 homeserver.db "PRAGMA auto_vacuum = FULL;"
# be cautious, it could make the database somewhat slow if there are a lot of deletions
exit

54
contrib/purge_api/purge_remote_media.sh Normal file
View file

@ -0,0 +1,54 @@
#!/bin/bash
DOMAIN=yourserver.tld
# add this user as admin in your home server:
ADMIN="@you_admin_username:$DOMAIN"
API_URL="$DOMAIN:8008/_matrix/client/r0"
# choose a time before which the messages should be pruned:
# TIME='2016-08-31 23:59:59'
TIME='12 months ago'
# creates a timestamp from the given time string:
UNIX_TIMESTAMP=$(date +%s%3N --date='TZ="UTC+2" '"$TIME")
###################################################################################################
# database function
###################################################################################################
sql (){
# for sqlite3:
#sqlite3 homeserver.db "pragma busy_timeout=20000;$1" | awk '{print $2}'
# for postgres:
psql -A -t --dbname=synapse -c "$1" | grep -v 'Pager'
}
###############################################################################
# make the admin user a server admin in the database with
###############################################################################
# sql "UPDATE users SET admin=1 WHERE name LIKE '$ADMIN'"
###############################################################################
# get an access token
###############################################################################
# for example externally by watching Riot in your browser's network inspector
# or internally on the server locally, use this:
TOKEN=$(sql "SELECT token FROM access_tokens WHERE user_id='$ADMIN' ORDER BY id DESC LIMIT 1")
###############################################################################
# check, if your TOKEN works. For example this works:
###############################################################################
# curl --header "Authorization: Bearer $TOKEN" "$API_URL/rooms/$ROOM/state/m.room.power_levels"
###############################################################################
# optional check size before
###############################################################################
# echo calculate used storage before ...
# du -shc ../.synapse/media_store/*
###############################################################################
# finally start pruning media:
###############################################################################
set -x # for debugging the generated string
curl --header "Authorization: Bearer $TOKEN" -v POST "$API_URL/admin/purge_media_cache/?before_ts=$UNIX_TIMESTAMP"

31
contrib/systemd/matrix-synapse.service Normal file
View file

@ -0,0 +1,31 @@
# Example systemd configuration file for synapse. Copy into
# /etc/systemd/system/, update the paths if necessary, then:
#
# systemctl enable matrix-synapse
# systemctl start matrix-synapse
#
# This assumes that Synapse has been installed in a virtualenv in
# /opt/synapse/env.
#
# **NOTE:** This is an example service file that may change in the future. If you
# wish to use this please copy rather than symlink it.
[Unit]
Description=Synapse Matrix homeserver
[Service]
Type=simple
Restart=on-abort
User=synapse
Group=nogroup
WorkingDirectory=/opt/synapse
ExecStart=/opt/synapse/env/bin/python -m synapse.app.homeserver --config-path=/opt/synapse/homeserver.yaml
# adjust the cache factor if necessary
# Environment=SYNAPSE_CACHE_FACTOR=2.0
[Install]
WantedBy=multi-user.target

22
contrib/systemd/synapse.service
View file

@ -1,22 +0,0 @@
# This assumes that Synapse has been installed as a system package
# (e.g. https://www.archlinux.org/packages/community/any/matrix-synapse/ for ArchLinux)
# rather than in a user home directory or similar under virtualenv.
# **NOTE:** This is an example service file that may change in the future. If you
# wish to use this please copy rather than symlink it.
[Unit]
Description=Synapse Matrix homeserver
[Service]
Type=simple
User=synapse
Group=synapse
WorkingDirectory=/var/lib/synapse
ExecStart=/usr/bin/python2.7 -m synapse.app.homeserver --config-path=/etc/synapse/homeserver.yaml
ExecStop=/usr/bin/synctl stop /etc/synapse/homeserver.yaml
# EnvironmentFile=-/etc/sysconfig/synapse # Can be used to e.g. set SYNAPSE_CACHE_FACTOR
[Install]
WantedBy=multi-user.target

7
debian/.gitignore vendored Normal file
View file

@ -0,0 +1,7 @@
/matrix-synapse-py3.*.debhelper
/matrix-synapse-py3.debhelper.log
/matrix-synapse-py3.substvars
/matrix-synapse-*/
/files
/debhelper-build-stamp
/.debhelper

32
debian/NEWS vendored Normal file
View file

@ -0,0 +1,32 @@
matrix-synapse-py3 (0.34.0) stable; urgency=medium
matrix-synapse-py3 is intended as a drop-in replacement for the existing
matrix-synapse package. When the package is installed, matrix-synapse will be
automatically uninstalled. The replacement should be relatively seamless,
however, please note the following important differences to matrix-synapse:
* Most importantly, the matrix-synapse service now runs under Python 3 rather
than Python 2.7.
* Synapse is installed into its own virtualenv (in /opt/venvs/matrix-synapse)
instead of using the system python libraries. (This may mean that you can
remove a number of old dependencies with `apt autoremove`).
* If you have previously manually installed any custom python extensions
(such as matrix-synapse-rest-auth) into the system python directories, you
will need to reinstall them in the new virtualenv. Please consult the
documentation of the relevant extensions for further details.
matrix-synapse-py3 will take over responsibility for the existing
configuration files, including the matrix-synapse systemd service.
Beware, however, that `apt purge matrix-synapse` will *disable* the
matrix-synapse service (so that it will not be started on reboot), even
though that service is no longer being provided by the matrix-synapse
package. It can be re-enabled with `systemctl enable matrix-synapse`.
The matrix.org team will continue to provide Python 2 `matrix-synapse`
packages for the next couple of releases, to allow time for system
administrators to test the new packages.
-- Richard van der Hoff <richard@matrix.org> Wed, 19 Dec 2018 14:00:00 +0000

91
debian/build_virtualenv vendored Executable file
View file

@ -0,0 +1,91 @@
#!/bin/bash
#
# runs dh_virtualenv to build the virtualenv in the build directory,
# and then runs the trial tests against the installed synapse.
set -e
export DH_VIRTUALENV_INSTALL_ROOT=/opt/venvs
# make sure that the virtualenv links to the specific version of python, by
# dereferencing the python3 symlink.
#
# Otherwise, if somebody tries to install (say) the stretch package on buster,
# they will get a confusing error about "No module named 'synapse'", because
# python won't look in the right directory. At least this way, the error will
# be a *bit* more obvious.
#
SNAKE=`readlink -e /usr/bin/python3`
# try to set the CFLAGS so any compiled C extensions are compiled with the most
# generic as possible x64 instructions, so that compiling it on a new Intel chip
# doesn't enable features not available on older ones or AMD.
#
# TODO: add similar things for non-amd64, or figure out a more generic way to
# do this.
case `dpkg-architecture -q DEB_HOST_ARCH` in
amd64)
export CFLAGS=-march=x86-64
;;
esac
# Use --builtin-venv to use the better `venv` module from CPython 3.4+ rather
# than the 2/3 compatible `virtualenv`.
dh_virtualenv \
--install-suffix "matrix-synapse" \
--builtin-venv \
--setuptools \
--python "$SNAKE" \
--upgrade-pip \
--preinstall="lxml" \
--preinstall="mock" \
--extra-pip-arg="--no-cache-dir" \
--extra-pip-arg="--compile" \
--extras="all"
PACKAGE_BUILD_DIR="debian/matrix-synapse-py3"
VIRTUALENV_DIR="${PACKAGE_BUILD_DIR}${DH_VIRTUALENV_INSTALL_ROOT}/matrix-synapse"
TARGET_PYTHON="${VIRTUALENV_DIR}/bin/python"
# we copy the tests to a temporary directory so that we can put them on the
# PYTHONPATH without putting the uninstalled synapse on the pythonpath.
tmpdir=`mktemp -d`
trap "rm -r $tmpdir" EXIT
cp -r tests "$tmpdir"
PYTHONPATH="$tmpdir" \
"${TARGET_PYTHON}" -B -m twisted.trial --reporter=text -j2 tests
# build the config file
"${TARGET_PYTHON}" -B "${VIRTUALENV_DIR}/bin/generate_config" \
--config-dir="/etc/matrix-synapse" \
--data-dir="/var/lib/matrix-synapse" |
perl -pe '
# tweak the paths to the tls certs and signing keys
/^tls_.*_path:/ and s/SERVERNAME/homeserver/;
/^signing_key_path:/ and s/SERVERNAME/homeserver/;
# tweak the pid file location
/^pid_file:/ and s#:.*#: "/var/run/matrix-synapse.pid"#;
# tweak the path to the log config
/^log_config:/ and s/SERVERNAME\.log\.config/log.yaml/;
# tweak the path to the media store
/^media_store_path:/ and s#/media_store#/media#;
# remove the server_name setting, which is set in a separate file
/^server_name:/ and $_ = "#\n# This is set in /etc/matrix-synapse/conf.d/server_name.yaml for Debian installations.\n# $_";
# remove the report_stats setting, which is set in a separate file
/^# report_stats:/ and $_ = "";
' > "${PACKAGE_BUILD_DIR}/etc/matrix-synapse/homeserver.yaml"
# add a dependency on the right version of python to substvars.
PYPKG=`basename $SNAKE`
echo "synapse:pydepends=$PYPKG" >> debian/matrix-synapse-py3.substvars

698
debian/changelog vendored Normal file
View file

@ -0,0 +1,698 @@
matrix-synapse-py3 (0.99.2) stable; urgency=medium
* Fix overwriting of config settings on upgrade.
* New synapse release 0.99.2.
-- Synapse Packaging team <packages@matrix.org> Fri, 01 Mar 2019 10:55:08 +0000
matrix-synapse-py3 (0.99.1.1) stable; urgency=medium
* New synapse release 0.99.1.1
-- Synapse Packaging team <packages@matrix.org> Thu, 14 Feb 2019 17:19:44 +0000
matrix-synapse-py3 (0.99.1) stable; urgency=medium
[ Damjan Georgievski ]
* Added ExecReload= in service unit file to send a HUP signal
[ Synapse Packaging team ]
* New synapse release 0.99.1
-- Synapse Packaging team <packages@matrix.org> Thu, 14 Feb 2019 14:12:26 +0000
matrix-synapse-py3 (0.99.0) stable; urgency=medium
* New synapse release 0.99.0
-- Synapse Packaging team <packages@matrix.org> Tue, 5 Feb 2019 18:25:00 +0000
matrix-synapse-py3 (0.34.1.1++1) stable; urgency=medium
* Update conflicts specifications to allow smoother transition from matrix-synapse.
-- Synapse Packaging team <packages@matrix.org> Sat, 12 Jan 2019 12:58:35 +0000
matrix-synapse-py3 (0.34.1.1) stable; urgency=high
* New synapse release 0.34.1.1
-- Synapse Packaging team <packages@matrix.org> Thu, 10 Jan 2019 15:04:52 +0000
matrix-synapse-py3 (0.34.1+1) stable; urgency=medium
* Remove 'Breaks: matrix-synapse-ldap3'. (matrix-synapse-py3 includes
the matrix-synapse-ldap3 python files, which makes the
matrix-synapse-ldap3 debian package redundant but not broken.
-- Synapse Packaging team <packages@matrix.org> Wed, 09 Jan 2019 15:30:00 +0000
matrix-synapse-py3 (0.34.1) stable; urgency=medium
* New synapse release 0.34.1.
* Update Conflicts specifications to allow installation alongside our
matrix-synapse transitional package.
-- Synapse Packaging team <packages@matrix.org> Wed, 09 Jan 2019 14:52:24 +0000
matrix-synapse-py3 (0.34.0) stable; urgency=medium
* New synapse release 0.34.0.
* Synapse is now installed into a Python 3 virtual environment with
up-to-date dependencies.
* The matrix-synapse service will now be restarted when the package is
upgraded.
(Fixes https://github.com/matrix-org/package-synapse-debian/issues/18)
-- Synapse packaging team <packages@matrix.org> Wed, 19 Dec 2018 14:00:00 +0000
matrix-synapse (0.33.9-1matrix1) stretch; urgency=medium
[ Erik Johnston ]
* Remove dependency on python-pydenticon
[ Richard van der Hoff ]
* New upstream version 0.33.9
* Refresh patches for 0.33.9
-- Richard van der Hoff <richard@matrix.org> Tue, 20 Nov 2018 10:26:05 +0000
matrix-synapse (0.33.8-1) stretch; urgency=medium
* New upstream version 0.33.8
-- Erik Johnston <erik@matrix.org> Thu, 01 Nov 2018 14:33:26 +0000
matrix-synapse (0.33.7-1matrix1) stretch; urgency=medium
* New upstream version 0.33.7
-- Richard van der Hoff <richard@matrix.org> Thu, 18 Oct 2018 16:18:26 +0100
matrix-synapse (0.33.6-1matrix1) stretch; urgency=medium
* Imported Upstream version 0.33.6
* Remove redundant explicit dep on python-bcrypt
* Run the tests during build
* Add dependency on python-attr 16.0
* Refresh patches for 0.33.6
-- Richard van der Hoff <richard@matrix.org> Thu, 04 Oct 2018 14:40:29 +0100
matrix-synapse (0.33.5.1-1matrix1) stretch; urgency=medium
* Imported Upstream version 0.33.5.1
-- Richard van der Hoff <richard@matrix.org> Mon, 24 Sep 2018 18:20:51 +0100
matrix-synapse (0.33.5-1matrix1) stretch; urgency=medium
* Imported Upstream version 0.33.5
-- Richard van der Hoff <richard@matrix.org> Mon, 24 Sep 2018 16:06:23 +0100
matrix-synapse (0.33.4-1mx1) stretch; urgency=medium
* Imported Upstream version 0.33.4
* Avoid telling people to install packages with pip
(fixes https://github.com/matrix-org/synapse/issues/3743)
-- Richard van der Hoff <richard@matrix.org> Fri, 07 Sep 2018 14:06:17 +0100
matrix-synapse (0.33.3.1-1mx1) stretch; urgency=critical
[ Richard van der Hoff ]
* Imported Upstream version 0.33.3.1
-- Richard van der Hoff <richard@matrix.org> Thu, 06 Sep 2018 11:20:37 +0100
matrix-synapse (0.33.3-2) stretch; urgency=medium
* We now require python-twisted 17.1.0 or later
* Add recommendations for python-psycopg2 and python-lxml
-- Richard van der Hoff <richard@matrix.org> Thu, 23 Aug 2018 19:04:08 +0100
matrix-synapse (0.33.3-1) jessie; urgency=medium
* New upstream version 0.33.3
-- Richard van der Hoff <richard@matrix.org> Wed, 22 Aug 2018 14:50:30 +0100
matrix-synapse (0.33.2-1) jessie; urgency=medium
* New upstream version 0.33.2
-- Richard van der Hoff <richard@matrix.org> Thu, 09 Aug 2018 15:40:42 +0100
matrix-synapse (0.33.1-1) jessie; urgency=medium
* New upstream version 0.33.1
-- Erik Johnston <erik@matrix.org> Thu, 02 Aug 2018 15:52:19 +0100
matrix-synapse (0.33.0-1) jessie; urgency=medium
* New upstream version 0.33.0
-- Richard van der Hoff <richard@matrix.org> Thu, 19 Jul 2018 13:38:41 +0100
matrix-synapse (0.32.1-1) jessie; urgency=medium
* New upstream version 0.32.1
-- Richard van der Hoff <richard@matrix.org> Fri, 06 Jul 2018 17:16:29 +0100
matrix-synapse (0.32.0-1) jessie; urgency=medium
* New upstream version 0.32.0
-- Erik Johnston <erik@matrix.org> Fri, 06 Jul 2018 15:34:06 +0100
matrix-synapse (0.31.2-1) jessie; urgency=high
* New upstream version 0.31.2
-- Richard van der Hoff <richard@matrix.org> Thu, 14 Jun 2018 16:49:07 +0100
matrix-synapse (0.31.1-1) jessie; urgency=medium
* New upstream version 0.31.1
* Require python-prometheus-client >= 0.0.14
-- Richard van der Hoff <richard@matrix.org> Fri, 08 Jun 2018 16:11:55 +0100
matrix-synapse (0.31.0-1) jessie; urgency=medium
* New upstream version 0.31.0
-- Richard van der Hoff <richard@matrix.org> Wed, 06 Jun 2018 17:23:10 +0100
matrix-synapse (0.30.0-1) jessie; urgency=medium
[ Michael Kaye ]
* update homeserver.yaml to be somewhat more modern.
[ Erik Johnston ]
* New upstream version 0.30.0
-- Erik Johnston <erik@matrix.org> Thu, 24 May 2018 16:43:16 +0100
matrix-synapse (0.29.0-1) jessie; urgency=medium
* New upstream version 0.29.0
-- Erik Johnston <erik@matrix.org> Wed, 16 May 2018 17:43:06 +0100
matrix-synapse (0.28.1-1) jessie; urgency=medium
* New upstream version 0.28.1
-- Erik Johnston <erik@matrix.org> Tue, 01 May 2018 19:21:39 +0100
matrix-synapse (0.28.0-1) jessie; urgency=medium
* New upstream 0.28.0
-- Erik Johnston <erik@matrix.org> Fri, 27 Apr 2018 13:15:49 +0100
matrix-synapse (0.27.4-1) jessie; urgency=medium
* Bump canonicaljson version
* New upstream 0.27.4
-- Erik Johnston <erik@matrix.org> Fri, 13 Apr 2018 13:37:47 +0100
matrix-synapse (0.27.3-1) jessie; urgency=medium
* Report stats should default to off
* Refresh patches
* New upstream 0.27.3
-- Erik Johnston <erik@matrix.org> Wed, 11 Apr 2018 11:43:47 +0100
matrix-synapse (0.27.2-1) jessie; urgency=medium
* New upstream version 0.27.2
-- Erik Johnston <erik@matrix.org> Mon, 26 Mar 2018 16:41:57 +0100
matrix-synapse (0.27.1-1) jessie; urgency=medium
* New upstream version 0.27.1
-- Erik Johnston <erik@matrix.org> Mon, 26 Mar 2018 16:22:03 +0100
matrix-synapse (0.27.0-2) jessie; urgency=medium
* Fix bcrypt dependency
-- Erik Johnston <erik@matrix.org> Mon, 26 Mar 2018 16:00:26 +0100
matrix-synapse (0.27.0-1) jessie; urgency=medium
* New upstream version 0.27.0
-- Erik Johnston <erik@matrix.org> Mon, 26 Mar 2018 15:07:52 +0100
matrix-synapse (0.26.1-1) jessie; urgency=medium
* Ignore RC
* New upstream version 0.26.1
-- Erik Johnston <erik@matrix.org> Fri, 16 Mar 2018 00:40:08 +0000
matrix-synapse (0.26.0-1) jessie; urgency=medium
[ Richard van der Hoff ]
* Remove `level` for `file` log handler
[ Erik Johnston ]
-- Erik Johnston <erik@matrix.org> Fri, 05 Jan 2018 11:21:26 +0000
matrix-synapse (0.25.1-1) jessie; urgency=medium
* New upstream version 0.25.1
-- Erik Johnston <erik@matrix.org> Mon, 20 Nov 2017 10:05:37 +0000
matrix-synapse (0.25.0-1) jessie; urgency=medium
* New upstream version 0.25.0
-- Erik Johnston <erik@matrix.org> Wed, 15 Nov 2017 11:36:32 +0000
matrix-synapse (0.24.1-1) jessie; urgency=medium
* New upstream version 0.24.1
-- Erik Johnston <erik@matrix.org> Tue, 24 Oct 2017 15:05:03 +0100
matrix-synapse (0.24.0-1) jessie; urgency=medium
* New upstream version 0.24.0
-- Erik Johnston <erik@matrix.org> Mon, 23 Oct 2017 14:11:46 +0100
matrix-synapse (0.23.1-1) xenial; urgency=medium
* Imported upstream version 0.23.1
-- Erik Johnston <erikj@matrix.org> Thu, 05 Oct 2017 15:28:25 +0100
matrix-synapse (0.23.0-1) jessie; urgency=medium
* Fix patch after refactor
* Add patch to remove requirement on affinity package
* refresh webclient patch
-- Erik Johnston <erikj@matrix.org> Mon, 02 Oct 2017 15:34:57 +0100
matrix-synapse (0.22.1-1) jessie; urgency=medium
* Imported Upstream version 0.22.1
-- Erik Johnston <erikj@matrix.org> Thu, 06 Jul 2017 18:14:13 +0100
matrix-synapse (0.22.0-1) jessie; urgency=medium
* Imported upstream version 0.22.0
-- Erik Johnston <erikj@matrix.org> Thu, 06 Jul 2017 10:47:45 +0100
matrix-synapse (0.21.1-1) jessie; urgency=medium
* Imported upstream version 0.21.1
-- Erik Johnston <erikj@matrix.org> Thu, 15 Jun 2017 13:31:13 +0100
matrix-synapse (0.21.0-1) jessie; urgency=medium
* Imported upstream version 0.21.0
* Update patches
-- Erik Johnston <erikj@matrix.org> Thu, 18 May 2017 14:16:54 +0100
matrix-synapse (0.20.0-2) jessie; urgency=medium
* Depend on python-jsonschema
-- Erik Johnston <erikj@matrix.org> Wed, 12 Apr 2017 10:41:46 +0100
matrix-synapse (0.20.0-1) jessie; urgency=medium
* Imported upstream version 0.20.0
-- Erik Johnston <erikj@matrix.org> Tue, 11 Apr 2017 12:58:26 +0100
matrix-synapse (0.19.3-1) jessie; urgency=medium
* Imported upstream version 0.19.3
-- Erik Johnston <erikj@matrix.org> Tue, 21 Mar 2017 13:45:41 +0000
matrix-synapse (0.19.2-1) jessie; urgency=medium
[ Sunil Mohan Adapa ]
* Bump standards version to 3.9.8
* Add debian/copyright file
* Don't ignore errors in debian/config
* Reformat depenedencies in debian/control
* Internationalize strings in template file
* Update package description
* Add lsb-base as dependency
* Update questions for debconf style
* Add man pages for all binaries
[ Erik Johnston ]
* Imported upstream version 0.19.2
-- Erik Johnston <erikj@matrix.org> Tue, 21 Feb 2017 13:55:00 +0000
matrix-synapse (0.19.1-1) jessie; urgency=medium
* Imported upstream version 0.19.1
-- Erik Johnston <erikj@matrix.org> Thu, 09 Feb 2017 11:53:27 +0000
matrix-synapse (0.19.0-1) jessie; urgency=medium
This build requires python-twisted 0.19.0, which may need to be installed
from backports.
[ Bryce Chidester ]
* Add EnvironmentFile to the systemd service
* Create matrix-synapse.default
[ Erik Johnston ]
* Imported upstream version 0.19.0
-- Erik Johnston <erikj@matrix.org> Sat, 04 Feb 2017 09:58:29 +0000
matrix-synapse (0.18.7-1) trusty; urgency=medium
* Imported Upstream version 0.18.4
-- Erik Johnston <erikj@matrix.org> Mon, 09 Jan 2017 15:10:21 +0000
matrix-synapse (0.18.5-1) trusty; urgency=medium
* Imported Upstream version 0.18.5
-- Erik Johnston <erikj@matrix.org> Fri, 16 Dec 2016 10:51:59 +0000
matrix-synapse (0.18.4-1) trusty; urgency=medium
* Imported Upstream version 0.18.4
-- Erik Johnston <erikj@matrix.org> Tue, 22 Nov 2016 10:33:41 +0000
matrix-synapse (0.18.3-1) trusty; urgency=medium
* Imported Upstream version 0.18.3
* Remove upstreamed ldap3 patch
-- Erik Johnston <erikj@matrix.org> Tue, 08 Nov 2016 15:01:49 +0000
matrix-synapse (0.18.2-2) trusty; urgency=high
* Patch ldap3 support to workaround differences in python-ldap3 0.9,
bug allowed unauthorized logins if ldap3 0.9 was used.
-- Erik Johnston <erikj@matrix.org> Tue, 08 Nov 2016 13:48:09 +0000
matrix-synapse (0.18.2-1) trusty; urgency=medium
* Imported Upstream version 0.18.2
-- Erik Johnston <erikj@matrix.org> Tue, 01 Nov 2016 13:30:45 +0000
matrix-synapse (0.18.1-1) trusty; urgency=medium
* Imported Upstream version 0.18.1
-- Erik Johnston <erikj@matrix.org> Wed, 05 Oct 2016 14:52:53 +0100
matrix-synapse (0.18.0-1) trusty; urgency=medium
* Imported Upstream version 0.18.0
-- Erik Johnston <erikj@matrix.org> Mon, 19 Sep 2016 17:38:48 +0100
matrix-synapse (0.17.3-1) trusty; urgency=medium
* Imported Upstream version 0.17.3
-- Erik Johnston <erikj@matrix.org> Fri, 09 Sep 2016 11:18:18 +0100
matrix-synapse (0.17.2-1) trusty; urgency=medium
* Imported Upstream version 0.17.2
-- Erik Johnston <erikj@matrix.org> Thu, 08 Sep 2016 15:37:14 +0100
matrix-synapse (0.17.1-1) trusty; urgency=medium
* Imported Upstream version 0.17.1
-- Erik Johnston <erikj@matrix.org> Wed, 24 Aug 2016 15:11:29 +0100
matrix-synapse (0.17.0-1) trusty; urgency=medium
* Imported Upstream version 0.17.0
-- Erik Johnston <erikj@matrix.org> Mon, 08 Aug 2016 13:56:15 +0100
matrix-synapse (0.16.1-r1-1) trusty; urgency=medium
* Imported Upstream version 0.16.1-r1
-- Erik Johnston <erikj@matrix.org> Fri, 08 Jul 2016 16:47:35 +0100
matrix-synapse (0.16.1-2) trusty; urgency=critical
* Apply security patch
-- Erik Johnston <erikj@matrix.org> Fri, 08 Jul 2016 11:05:27 +0100
matrix-synapse (0.16.1-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Tue, 21 Jun 2016 14:56:48 +0100
matrix-synapse (0.16.0-3) trusty; urgency=medium
* Don't require strict nacl==0.3.0 requirement
-- Erik Johnston <erikj@matrix.org> Mon, 20 Jun 2016 13:24:22 +0100
matrix-synapse (0.16.0-2) trusty; urgency=medium
* Also change the permissions of /etc/matrix-synapse
* Add apt webclient instructions
* Fix up patches
* Update default homeserver.yaml
* Add patch
-- Erik Johnston <erikj@matrix.org> Fri, 10 Jun 2016 14:06:20 +0100
matrix-synapse (0.16.0-1) trusty; urgency=medium
[ David A Roberts ]
* systemd
[ Erik Johnston ]
* Fixup postinst and matrix-synapse.service
* Handle email optional deps
* New upstream release
-- Erik Johnston <erikj@matrix.org> Thu, 09 Jun 2016 16:17:01 +0100
matrix-synapse (0.14.0-1) trusty; urgency=medium
* Remove saml2 module requirements
-- Erik Johnston <erikj@matrix.org> Wed, 30 Mar 2016 14:31:17 +0100
matrix-synapse (0.13.3-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Thu, 11 Feb 2016 16:35:39 +0000
matrix-synapse (0.13.2-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Thu, 11 Feb 2016 11:01:16 +0000
matrix-synapse (0.13.0-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Wed, 10 Feb 2016 16:34:39 +0000
matrix-synapse (0.12.0-2) trusty; urgency=medium
* Don't default `registerion_shared_secret` config option
-- Erik Johnston <erikj@matrix.org> Wed, 06 Jan 2016 16:34:02 +0000
matrix-synapse (0.12.0-1) stable; urgency=medium
* Imported Upstream version 0.12.0
-- Mark Haines <mark@matrix.org> Mon, 04 Jan 2016 15:38:33 +0000
matrix-synapse (0.11.1-1) unstable; urgency=medium
* Imported Upstream version 0.11.1
-- Erik Johnston <erikj@matrix.org> Fri, 20 Nov 2015 17:56:52 +0000
matrix-synapse (0.11.0-r2-1) stable; urgency=medium
* Imported Upstream version 0.11.0-r2
* Add gbp.conf
-- Erik Johnston <erikj@matrix.org> Thu, 19 Nov 2015 13:52:36 +0000
matrix-synapse (0.11.0-1) wheezy; urgency=medium
* Fix dependencies.
-- Erik Johnston <erikj@matrix.org> Tue, 17 Nov 2015 16:28:06 +0000
matrix-synapse (0.11.0-0) wheezy; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Tue, 17 Nov 2015 16:03:01 +0000
matrix-synapse (0.10.0-2) wheezy; urgency=medium
* Rebuild for wheezy.
-- Erik Johnston <erikj@matrix.org> Fri, 04 Sep 2015 14:21:03 +0100
matrix-synapse (0.10.0-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Thu, 03 Sep 2015 10:08:34 +0100
matrix-synapse (0.10.0~rc6-3) trusty; urgency=medium
* Create log directory.
-- Erik Johnston <erikj@matrix.org> Wed, 02 Sep 2015 17:49:07 +0100
matrix-synapse (0.10.0~rc6-2) trusty; urgency=medium
* Add patch to work around upstream bug in config directory handling.
-- Erik Johnston <erikj@matrix.org> Wed, 02 Sep 2015 17:42:42 +0100
matrix-synapse (0.10.0~rc6-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Wed, 02 Sep 2015 17:21:21 +0100
matrix-synapse (0.10.0~rc5-3) trusty; urgency=medium
* Update init script to work.
-- Erik Johnston <erikj@matrix.org> Fri, 28 Aug 2015 10:51:56 +0100
matrix-synapse (0.10.0~rc5-2) trusty; urgency=medium
* Fix where python files are installed.
-- Erik Johnston <erikj@matrix.org> Thu, 27 Aug 2015 11:55:39 +0100
matrix-synapse (0.10.0~rc5-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Thu, 27 Aug 2015 11:26:54 +0100
matrix-synapse (0.10.0~rc4-1) trusty; urgency=medium
* New upstream version.
-- Erik Johnston <erikj@matrix.org> Thu, 27 Aug 2015 10:29:31 +0100
matrix-synapse (0.10.0~rc3-7) trusty; urgency=medium
* Add debian/watch
-- Erik Johnston <erikj@matrix.org> Wed, 26 Aug 2015 17:57:08 +0100
matrix-synapse (0.10.0~rc3-6) trusty; urgency=medium
* Deps.
-- Erik Johnston <erikj@matrix.org> Wed, 26 Aug 2015 17:07:13 +0100
matrix-synapse (0.10.0~rc3-5) trusty; urgency=medium
* Deps.
-- Erik Johnston <erikj@matrix.org> Wed, 26 Aug 2015 16:18:02 +0100
matrix-synapse (0.10.0~rc3-4) trusty; urgency=medium
* More deps.
-- Erik Johnston <erikj@matrix.org> Wed, 26 Aug 2015 14:09:27 +0100
matrix-synapse (0.10.0~rc3-3) trusty; urgency=medium
* Update deps.
-- Erik Johnston <erikj@matrix.org> Wed, 26 Aug 2015 13:49:20 +0100
matrix-synapse (0.10.0~rc3-2) trusty; urgency=medium
* Add more deps.
-- Erik Johnston <erikj@matrix.org> Wed, 26 Aug 2015 13:25:45 +0100
matrix-synapse (0.10.0~rc3-1) trusty; urgency=medium
* New upstream release
-- Erik Johnston <erikj@matrix.org> Tue, 25 Aug 2015 17:52:33 +0100
matrix-synapse (0.9.3-1~trusty1) trusty; urgency=medium
* Rebuild for trusty.
-- Erik Johnston <erikj@matrix.org> Thu, 20 Aug 2015 15:05:43 +0100
matrix-synapse (0.9.3-1) wheezy; urgency=medium
* New upstream release
* Create a user, "matrix-synapse", to run as
* Log to /var/log/matrix-synapse/ directory
* Override the way synapse looks for the angular SDK (syweb) so it finds the
packaged one
-- Paul "LeoNerd" Evans <paul@matrix.org> Fri, 07 Aug 2015 15:32:12 +0100
matrix-synapse (0.9.2-2) wheezy; urgency=medium
* Supply a default config file
* Create directory in /var/lib
* Use debconf to ask the user for the server name at installation time
-- Paul "LeoNerd" Evans <paul@matrix.org> Thu, 06 Aug 2015 15:28:00 +0100
matrix-synapse (0.9.2-1) wheezy; urgency=low
* source package automatically created by stdeb 0.8.2
-- Paul "LeoNerd" Evans <paul@matrix.org> Fri, 12 Jun 2015 14:32:03 +0100

1
debian/compat vendored Normal file
View file

@ -0,0 +1 @@
9

40
debian/control vendored Normal file
View file

@ -0,0 +1,40 @@
Source: matrix-synapse-py3
Section: contrib/python
Priority: extra
Maintainer: Synapse Packaging team <packages@matrix.org>
Build-Depends:
debhelper (>= 9),
dh-systemd,
dh-virtualenv (>= 1.1),
lsb-release,
python3-dev,
python3,
python3-setuptools,
python3-pip,
python3-venv,
tar,
Standards-Version: 3.9.8
Homepage: https://github.com/matrix-org/synapse
Package: matrix-synapse-py3
Architecture: amd64
Provides: matrix-synapse
Conflicts:
matrix-synapse (<< 0.34.0.1-0matrix2),
matrix-synapse (>= 0.34.0.1-1),
Pre-Depends: dpkg (>= 1.16.1)
Depends:
adduser,
debconf,
python3-distutils|libpython3-stdlib (<< 3.6),
${misc:Depends},
${synapse:pydepends},
# some of our scripts use perl, but none of them are important,
# so we put perl:Depends in Suggests rather than Depends.
Suggests:
sqlite3,
${perl:Depends},
Description: Open federated Instant Messaging and VoIP server
Matrix is an ambitious new ecosystem for open federated Instant
Messaging and VoIP. Synapse is a reference Matrix server
implementation.

118
debian/copyright vendored Normal file
View file

@ -0,0 +1,118 @@
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: synapse
Source: https://github.com/matrix-org/synapse
Files: *
Copyright: 2014-2017, OpenMarket Ltd, 2017-2018 New Vector Ltd
License: Apache-2.0
Files: synapse/config/saml2.py
Copyright: 2015, Ericsson
License: Apache-2.0
Files: synapse/config/jwt.py
Copyright: 2015, Niklas Riekenbrauck
License: Apache-2.0
Files: synapse/config/workers.py
Copyright: 2016, matrix.org
License: Apache-2.0
Files: synapse/config/repository.py
Copyright: 2014-2015, matrix.org
License: Apache-2.0
Files: contrib/jitsimeetbridge/unjingle/strophe/base64.js
Copyright: Public Domain (Tyler Akins http://rumkin.com)
License: public-domain
This code was written by Tyler Akins and has been placed in the
public domain. It would be nice if you left this header intact.
Base64 code from Tyler Akins -- http://rumkin.com
Files: contrib/jitsimeetbridge/unjingle/strophe/md5.js
Copyright: 1999-2002, Paul Johnston & Contributors
License: BSD-3-clause
Files: contrib/jitsimeetbridge/unjingle/strophe/strophe.js
Copyright: 2006-2008, OGG, LLC
License: Expat
Files: contrib/jitsimeetbridge/unjingle/strophe/XMLHttpRequest.js
Copyright: 2010 passive.ly LLC
License: Expat
Files: contrib/jitsimeetbridge/unjingle/*.js
Copyright: 2014 Jitsi
License: Apache-2.0
Files: debian/*
Copyright: 2016-2017, Erik Johnston <erik@matrix.org>
2017, Rahul De <rahulde@swecha.net>
2017, Sunil Mohan Adapa <sunil@medhas.org>
License: Apache-2.0
License: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian systems, the full text of the Apache License version
2.0 can be found in the file
`/usr/share/common-licenses/Apache-2.0'.
License: BSD-3-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
.
Redistributions of source code must retain the above copyright
notice, this list of conditions and the following
disclaimer. Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided with
the distribution.
.
Neither the name of the author nor the names of its contributors may
be used to endorse or promote products derived from this software
without specific prior written permission.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License: Expat
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
.
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

3
debian/dirs vendored Normal file
View file

@ -0,0 +1,3 @@
etc/matrix-synapse
var/lib/matrix-synapse
var/log/matrix-synapse

90
debian/hash_password.1 vendored Normal file
View file

@ -0,0 +1,90 @@
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "HASH_PASSWORD" "1" "February 2017" "" ""
.
.SH "NAME"
\fBhash_password\fR \- Calculate the hash of a new password, so that passwords can be reset
.
.SH "SYNOPSIS"
\fBhash_password\fR [\fB\-p\fR|\fB\-\-password\fR [password]] [\fB\-c\fR|\fB\-\-config\fR \fIfile\fR]
.
.SH "DESCRIPTION"
\fBhash_password\fR calculates the hash of a supplied password using bcrypt\.
.
.P
\fBhash_password\fR takes a password as an parameter either on the command line or the \fBSTDIN\fR if not supplied\.
.
.P
It accepts an YAML file which can be used to specify parameters like the number of rounds for bcrypt and password_config section having the pepper value used for the hashing\. By default \fBbcrypt_rounds\fR is set to \fB10\fR\.
.
.P
The hashed password is written on the \fBSTDOUT\fR\.
.
.SH "FILES"
A sample YAML file accepted by \fBhash_password\fR is described below:
.
.P
bcrypt_rounds: 17 password_config: pepper: "random hashing pepper"
.
.SH "OPTIONS"
.
.TP
\fB\-p\fR, \fB\-\-password\fR
Read the password form the command line if [password] is supplied\. If not, prompt the user and read the password form the \fBSTDIN\fR\. It is not recommended to type the password on the command line directly\. Use the STDIN instead\.
.
.TP
\fB\-c\fR, \fB\-\-config\fR
Read the supplied YAML \fIfile\fR containing the options \fBbcrypt_rounds\fR and the \fBpassword_config\fR section containing the \fBpepper\fR value\.
.
.SH "EXAMPLES"
Hash from the command line:
.
.IP "" 4
.
.nf
$ hash_password \-p "p@ssw0rd"
$2b$12$VJNqWQYfsWTEwcELfoSi4Oa8eA17movHqqi8\.X8fWFpum7SxZ9MFe
.
.fi
.
.IP "" 0
.
.P
Hash from the STDIN:
.
.IP "" 4
.
.nf
$ hash_password
Password:
Confirm password:
$2b$12$AszlvfmJl2esnyhmn8m/kuR2tdXgROWtWxnX\.rcuAbM8ErLoUhybG
.
.fi
.
.IP "" 0
.
.P
Using a config file:
.
.IP "" 4
.
.nf
$ hash_password \-c config\.yml
Password:
Confirm password:
$2b$12$CwI\.wBNr\.w3kmiUlV3T5s\.GT2wH7uebDCovDrCOh18dFedlANK99O
.
.fi
.
.IP "" 0
.
.SH "COPYRIGHT"
This man page was written by Rahul De <\fIrahulde@swecha\.net\fR> for Debian GNU/Linux distribution\.
.
.SH "SEE ALSO"
synctl(1), synapse_port_db(1), register_new_matrix_user(1)

69
debian/hash_password.ronn vendored Normal file
View file

@ -0,0 +1,69 @@
hash_password(1) -- Calculate the hash of a new password, so that passwords can be reset
========================================================================================
## SYNOPSIS
`hash_password` [`-p`|`--password` [password]] [`-c`|`--config` <file>]
## DESCRIPTION
**hash_password** calculates the hash of a supplied password using bcrypt.
`hash_password` takes a password as an parameter either on the command line
or the `STDIN` if not supplied.
It accepts an YAML file which can be used to specify parameters like the
number of rounds for bcrypt and password_config section having the pepper
value used for the hashing. By default `bcrypt_rounds` is set to **10**.
The hashed password is written on the `STDOUT`.
## FILES
A sample YAML file accepted by `hash_password` is described below:
bcrypt_rounds: 17
password_config:
pepper: "random hashing pepper"
## OPTIONS
* `-p`, `--password`:
Read the password form the command line if [password] is supplied.
If not, prompt the user and read the password form the `STDIN`.
It is not recommended to type the password on the command line
directly. Use the STDIN instead.
* `-c`, `--config`:
Read the supplied YAML <file> containing the options `bcrypt_rounds`
and the `password_config` section containing the `pepper` value.
## EXAMPLES
Hash from the command line:
$ hash_password -p "p@ssw0rd"
$2b$12$VJNqWQYfsWTEwcELfoSi4Oa8eA17movHqqi8.X8fWFpum7SxZ9MFe
Hash from the STDIN:
$ hash_password
Password:
Confirm password:
$2b$12$AszlvfmJl2esnyhmn8m/kuR2tdXgROWtWxnX.rcuAbM8ErLoUhybG
Using a config file:
$ hash_password -c config.yml
Password:
Confirm password:
$2b$12$CwI.wBNr.w3kmiUlV3T5s.GT2wH7uebDCovDrCOh18dFedlANK99O
## COPYRIGHT
This man page was written by Rahul De <<rahulde@swecha.net>>
for Debian GNU/Linux distribution.
## SEE ALSO
synctl(1), synapse_port_db(1), register_new_matrix_user(1)

2
debian/install vendored Normal file
View file

@ -0,0 +1,2 @@
debian/log.yaml etc/matrix-synapse
debian/manage_debconf.pl /opt/venvs/matrix-synapse/lib/

36
debian/log.yaml vendored Normal file
View file

@ -0,0 +1,36 @@
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s- %(message)s'
filters:
context:
(): synapse.util.logcontext.LoggingContextFilter
request: ""
handlers:
file:
class: logging.handlers.RotatingFileHandler
formatter: precise
filename: /var/log/matrix-synapse/homeserver.log
maxBytes: 104857600
backupCount: 10
filters: [context]
encoding: utf8
console:
class: logging.StreamHandler
formatter: precise
level: WARN
loggers:
synapse:
level: INFO
synapse.storage.SQL:
level: INFO
root:
level: INFO
handlers: [file, console]

130
debian/manage_debconf.pl vendored Executable file
View file

@ -0,0 +1,130 @@
#!/usr/bin/perl
#
# Interface between our config files and the debconf database.
#
# Usage:
#
# manage_debconf.pl <action>
#
# where <action> can be:
#
# read: read the configuration from the yaml into debconf
# update: update the yaml config according to the debconf database
use strict;
use warnings;
use Debconf::Client::ConfModule (qw/get set/);
# map from the name of a setting in our .yaml file to the relevant debconf
# setting.
my %MAPPINGS=(
server_name => 'matrix-synapse/server-name',
report_stats => 'matrix-synapse/report-stats',
);
# enable debug if dpkg --debug
my $DEBUG = $ENV{DPKG_MAINTSCRIPT_DEBUG};
sub read_config {
my @files = @_;
foreach my $file (@files) {
print STDERR "reading $file\n" if $DEBUG;
open my $FH, "<", $file or next;
# rudimentary parsing which (a) avoids having to depend on a yaml library,
# and (b) is tolerant of yaml errors
while($_ = <$FH>) {
while (my ($setting, $debconf) = each %MAPPINGS) {
$setting = quotemeta $setting;
if(/^${setting}\s*:(.*)$/) {
my $val = $1;
# remove leading/trailing whitespace
$val =~ s/^\s*//;
$val =~ s/\s*$//;
# remove surrounding quotes
if ($val =~ /^"(.*)"$/ || $val =~ /^'(.*)'$/) {
$val = $1;
}
print STDERR ">> $debconf = $val\n" if $DEBUG;
set($debconf, $val);
}
}
}
close $FH;
}
}
sub update_config {
my @files = @_;
my %substs = ();
while (my ($setting, $debconf) = each %MAPPINGS) {
my @res = get($debconf);
$substs{$setting} = $res[1] if $res[0] == 0;
}
foreach my $file (@files) {
print STDERR "checking $file\n" if $DEBUG;
open my $FH, "<", $file or next;
my $updated = 0;
# read the whole file into memory
my @lines = <$FH>;
while (my ($setting, $val) = each %substs) {
$setting = quotemeta $setting;
map {
if (/^${setting}\s*:\s*(.*)\s*$/) {
my $current = $1;
if ($val ne $current) {
$_ = "${setting}: $val\n";
$updated = 1;
}
}
} @lines;
}
close $FH;
next unless $updated;
print STDERR "updating $file\n" if $DEBUG;
open $FH, ">", $file or die "unable to update $file";
print $FH @lines;
close $FH;
}
}
my $cmd = $ARGV[0];
my $read = 0;
my $update = 0;
if (not $cmd) {
die "must specify a command to perform\n";
} elsif ($cmd eq 'read') {
$read = 1;
} elsif ($cmd eq 'update') {
$update = 1;
} else {
die "unknown command '$cmd'\n";
}
my @files = (
"/etc/matrix-synapse/homeserver.yaml",
glob("/etc/matrix-synapse/conf.d/*.yaml"),
);
if ($read) {
read_config(@files);
} elsif ($update) {
update_config(@files);
}

4
debian/manpages vendored Normal file
View file

@ -0,0 +1,4 @@
debian/hash_password.1
debian/register_new_matrix_user.1
debian/synapse_port_db.1
debian/synctl.1

12
debian/matrix-synapse-py3.config vendored Executable file
View file

@ -0,0 +1,12 @@
#!/bin/sh
set -e
. /usr/share/debconf/confmodule
# try to update the debconf db according to whatever is in the config files
/opt/venvs/matrix-synapse/lib/manage_debconf.pl read || true
db_input high matrix-synapse/server-name || true
db_input high matrix-synapse/report-stats || true
db_go

4
debian/matrix-synapse-py3.links vendored Normal file
View file

@ -0,0 +1,4 @@
opt/venvs/matrix-synapse/bin/hash_password usr/bin/hash_password
opt/venvs/matrix-synapse/bin/register_new_matrix_user usr/bin/register_new_matrix_user
opt/venvs/matrix-synapse/bin/synapse_port_db usr/bin/synapse_port_db
opt/venvs/matrix-synapse/bin/synctl usr/bin/synctl

56
debian/matrix-synapse-py3.postinst vendored Normal file
View file

@ -0,0 +1,56 @@
#!/bin/sh -e
. /usr/share/debconf/confmodule
CONFIGFILE_SERVERNAME="/etc/matrix-synapse/conf.d/server_name.yaml"
CONFIGFILE_REPORTSTATS="/etc/matrix-synapse/conf.d/report_stats.yaml"
USER="matrix-synapse"
case "$1" in
configure|reconfigure)
# generate template config files if they don't exist
mkdir -p "/etc/matrix-synapse/conf.d/"
if [ ! -e "$CONFIGFILE_SERVERNAME" ]; then
cat > "$CONFIGFILE_SERVERNAME" <<EOF
# This file is autogenerated, and will be recreated on upgrade if it is deleted.
# Any changes you make will be preserved.
# The domain name of the server, with optional explicit port.
# This is used by remote servers to connect to this server,
# e.g. matrix.org, localhost:8080, etc.
# This is also the last part of your UserID.
#
server_name: ''
EOF
fi
if [ ! -e "$CONFIGFILE_REPORTSTATS" ]; then
cat > "$CONFIGFILE_REPORTSTATS" <<EOF
# This file is autogenerated, and will be recreated on upgrade if it is deleted.
# Any changes you make will be preserved.
# Whether to report anonymized homeserver usage statistics.
report_stats: false
EOF
fi
# update the config files according to whatever is in the debconf database
/opt/venvs/matrix-synapse/lib/manage_debconf.pl update
if ! getent passwd $USER >/dev/null; then
adduser --quiet --system --no-create-home --home /var/lib/matrix-synapse $USER
fi
for DIR in /var/lib/matrix-synapse /var/log/matrix-synapse /etc/matrix-synapse; do
if ! dpkg-statoverride --list --quiet $DIR >/dev/null; then
dpkg-statoverride --force --quiet --update --add $USER nogroup 0755 $DIR
fi
done
;;
esac
#DEBHELPER#
exit 0

31
debian/matrix-synapse-py3.preinst vendored Normal file
View file

@ -0,0 +1,31 @@
#!/bin/sh -e
# Attempt to undo some of the braindamage caused by
# https://github.com/matrix-org/package-synapse-debian/issues/18.
#
# Due to reasons [1], the old python2 matrix-synapse package will not stop the
# service when the package is uninstalled. Our maintainer scripts will do the
# right thing in terms of ensuring the service is enabled and unmasked, but
# then do a `systemctl start matrix-synapse`, which of course does nothing -
# leaving the old (py2) service running.
#
# There should normally be no reason for the service to be running during our
# preinst, so we assume that if it *is* running, it's due to that situation,
# and stop it.
#
# [1] dh_systemd_start doesn't do anything because it sees that there is an
# init.d script with the same name, so leaves it to dh_installinit.
#
# dh_installinit doesn't do anything because somebody gave it a --no-start
# for unknown reasons.
if [ -x /bin/systemctl ]; then
if /bin/systemctl --quiet is-active -- matrix-synapse; then
echo >&2 "stopping existing matrix-synapse service"
/bin/systemctl stop matrix-synapse || true
fi
fi
#DEBHELPER#
exit 0

9
debian/matrix-synapse-py3.triggers vendored Normal file
View file

@ -0,0 +1,9 @@
# Register interest in Python interpreter changes and
# don't make the Python package dependent on the virtualenv package
# processing (noawait)
interest-noawait /usr/bin/python3.5
interest-noawait /usr/bin/python3.6
interest-noawait /usr/bin/python3.7
# Also provide a symbolic trigger for all dh-virtualenv packages
interest dh-virtualenv-interpreter-update

2
debian/matrix-synapse.default vendored Normal file
View file

@ -0,0 +1,2 @@
# Specify environment variables used when running Synapse
# SYNAPSE_CACHE_FACTOR=1 (default)

16
debian/matrix-synapse.service vendored Normal file
View file

@ -0,0 +1,16 @@
[Unit]
Description=Synapse Matrix homeserver
[Service]
Type=simple
User=matrix-synapse
WorkingDirectory=/var/lib/matrix-synapse
EnvironmentFile=/etc/default/matrix-synapse
ExecStartPre=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --generate-keys
ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target

1
debian/po/POTFILES.in vendored Normal file
View file

@ -0,0 +1 @@
[type: gettext/rfc822deb] templates

56
debian/po/templates.pot vendored Normal file
View file

@ -0,0 +1,56 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the matrix-synapse package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: matrix-synapse\n"
"Report-Msgid-Bugs-To: matrix-synapse@packages.debian.org\n"
"POT-Creation-Date: 2017-02-21 07:51+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=CHARSET\n"
"Content-Transfer-Encoding: 8bit\n"
#. Type: string
#. Description
#: ../templates:1001
msgid "Name of the server:"
msgstr ""
#. Type: string
#. Description
#: ../templates:1001
msgid ""
"The name that this homeserver will appear as, to clients and other servers "
"via federation. This name should match the SRV record published in DNS."
msgstr ""
#. Type: boolean
#. Description
#: ../templates:2001
msgid "Report anonymous statistics?"
msgstr ""
#. Type: boolean
#. Description
#: ../templates:2001
msgid ""
"Developers of Matrix and Synapse really appreciate helping the project out "
"by reporting anonymized usage statistics from this homeserver. Only very "
"basic aggregate data (e.g. number of users) will be reported, but it helps "
"track the growth of the Matrix community, and helps in making Matrix a "
"success, as well as to convince other networks that they should peer with "
"Matrix."
msgstr ""
#. Type: boolean
#. Description
#: ../templates:2001
msgid "Thank you."
msgstr ""

72
debian/register_new_matrix_user.1 vendored Normal file
View file

@ -0,0 +1,72 @@
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "REGISTER_NEW_MATRIX_USER" "1" "February 2017" "" ""
.
.SH "NAME"
\fBregister_new_matrix_user\fR \- Used to register new users with a given home server when registration has been disabled
.
.SH "SYNOPSIS"
\fBregister_new_matrix_user\fR options\.\.\.
.
.SH "DESCRIPTION"
\fBregister_new_matrix_user\fR registers new users with a given home server when registration has been disabled\. For this to work, the home server must be configured with the \'registration_shared_secret\' option set\.
.
.P
This accepts the user credentials like the username, password, is user an admin or not and registers the user onto the homeserver database\. Also, a YAML file containing the shared secret can be provided\. If not, the shared secret can be provided via the command line\.
.
.P
By default it assumes the home server URL to be \fBhttps://localhost:8448\fR\. This can be changed via the \fBserver_url\fR command line option\.
.
.SH "FILES"
A sample YAML file accepted by \fBregister_new_matrix_user\fR is described below:
.
.IP "" 4
.
.nf
registration_shared_secret: "s3cr3t"
.
.fi
.
.IP "" 0
.
.SH "OPTIONS"
.
.TP
\fB\-u\fR, \fB\-\-user\fR
Local part of the new user\. Will prompt if omitted\.
.
.TP
\fB\-p\fR, \fB\-\-password\fR
New password for user\. Will prompt if omitted\. Supplying the password on the command line is not recommended\. Use the STDIN instead\.
.
.TP
\fB\-a\fR, \fB\-\-admin\fR
Register new user as an admin\. Will prompt if omitted\.
.
.TP
\fB\-c\fR, \fB\-\-config\fR
Path to server config file containing the shared secret\.
.
.TP
\fB\-k\fR, \fB\-\-shared\-secret\fR
Shared secret as defined in server config file\. This is an optional parameter as it can be also supplied via the YAML file\.
.
.TP
\fBserver_url\fR
URL of the home server\. Defaults to \'https://localhost:8448\'\.
.
.SH "EXAMPLES"
.
.nf
$ register_new_matrix_user \-u user1 \-p p@ssword \-a \-c config\.yaml
.
.fi
.
.SH "COPYRIGHT"
This man page was written by Rahul De <\fIrahulde@swecha\.net\fR> for Debian GNU/Linux distribution\.
.
.SH "SEE ALSO"
synctl(1), synapse_port_db(1), hash_password(1)

61
debian/register_new_matrix_user.ronn vendored Normal file
View file

@ -0,0 +1,61 @@
register_new_matrix_user(1) -- Used to register new users with a given home server when registration has been disabled
======================================================================================================================
## SYNOPSIS
`register_new_matrix_user` options...
## DESCRIPTION
**register_new_matrix_user** registers new users with a given home server when
registration has been disabled. For this to work, the home server must be
configured with the 'registration_shared_secret' option set.
This accepts the user credentials like the username, password, is user an
admin or not and registers the user onto the homeserver database. Also,
a YAML file containing the shared secret can be provided. If not, the
shared secret can be provided via the command line.
By default it assumes the home server URL to be `https://localhost:8448`.
This can be changed via the `server_url` command line option.
## FILES
A sample YAML file accepted by `register_new_matrix_user` is described below:
registration_shared_secret: "s3cr3t"
## OPTIONS
* `-u`, `--user`:
Local part of the new user. Will prompt if omitted.
* `-p`, `--password`:
New password for user. Will prompt if omitted. Supplying the password
on the command line is not recommended. Use the STDIN instead.
* `-a`, `--admin`:
Register new user as an admin. Will prompt if omitted.
* `-c`, `--config`:
Path to server config file containing the shared secret.
* `-k`, `--shared-secret`:
Shared secret as defined in server config file. This is an optional
parameter as it can be also supplied via the YAML file.
* `server_url`:
URL of the home server. Defaults to 'https://localhost:8448'.
## EXAMPLES
$ register_new_matrix_user -u user1 -p p@ssword -a -c config.yaml
## COPYRIGHT
This man page was written by Rahul De <<rahulde@swecha.net>>
for Debian GNU/Linux distribution.
## SEE ALSO
synctl(1), synapse_port_db(1), hash_password(1)

22
debian/rules vendored Executable file
View file

@ -0,0 +1,22 @@
#!/usr/bin/make -f
#
# Build Debian package using https://github.com/spotify/dh-virtualenv
#
override_dh_systemd_enable:
dh_systemd_enable --name=matrix-synapse
override_dh_installinit:
dh_installinit --name=matrix-synapse
override_dh_strip:
override_dh_shlibdeps:
override_dh_virtualenv:
./debian/build_virtualenv
# We are restricted to compat level 9 (because xenial), so have to
# enable the systemd bits manually.
%:
dh $@ --with python-virtualenv --with systemd

1
debian/source/format vendored Normal file
View file

@ -0,0 +1 @@
3.0 (native)

98
debian/synapse_port_db.1 vendored Normal file
View file

@ -0,0 +1,98 @@
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "SYNAPSE_PORT_DB" "1" "February 2017" "" ""
.
.SH "NAME"
\fBsynapse_port_db\fR \- A script to port an existing synapse SQLite database to a new PostgreSQL database\.
.
.SH "SYNOPSIS"
\fBsynapse_port_db\fR [\-v] \-\-sqlite\-database=\fIdbfile\fR \-\-postgres\-config=\fIyamlconfig\fR [\-\-curses] [\-\-batch\-size=\fIbatch\-size\fR]
.
.SH "DESCRIPTION"
\fBsynapse_port_db\fR ports an existing synapse SQLite database to a new PostgreSQL database\.
.
.P
SQLite database is specified with \fB\-\-sqlite\-database\fR option and PostgreSQL configuration required to connect to PostgreSQL database is provided using \fB\-\-postgres\-config\fR configuration\. The configuration is specified in YAML format\.
.
.SH "OPTIONS"
.
.TP
\fB\-v\fR
Print log messages in \fBdebug\fR level instead of \fBinfo\fR level\.
.
.TP
\fB\-\-sqlite\-database\fR
The snapshot of the SQLite database file\. This must not be currently used by a running synapse server\.
.
.TP
\fB\-\-postgres\-config\fR
The database config file for the PostgreSQL database\.
.
.TP
\fB\-\-curses\fR
Display a curses based progress UI\.
.
.SH "CONFIG FILE"
The postgres configuration file must be a valid YAML file with the following options\.
.
.IP "\(bu" 4
\fBdatabase\fR: Database configuration section\. This section header can be ignored and the options below may be specified as top level keys\.
.
.IP "\(bu" 4
\fBname\fR: Connector to use when connecting to the database\. This value must be \fBpsycopg2\fR\.
.
.IP "\(bu" 4
\fBargs\fR: DB API 2\.0 compatible arguments to send to the \fBpsycopg2\fR module\.
.
.IP "\(bu" 4
\fBdbname\fR \- the database name
.
.IP "\(bu" 4
\fBuser\fR \- user name used to authenticate
.
.IP "\(bu" 4
\fBpassword\fR \- password used to authenticate
.
.IP "\(bu" 4
\fBhost\fR \- database host address (defaults to UNIX socket if not provided)
.
.IP "\(bu" 4
\fBport\fR \- connection port number (defaults to 5432 if not provided)
.
.IP "" 0
.
.IP "\(bu" 4
\fBsynchronous_commit\fR: Optional\. Default is True\. If the value is \fBFalse\fR, enable asynchronous commit and don\'t wait for the server to call fsync before ending the transaction\. See: https://www\.postgresql\.org/docs/current/static/wal\-async\-commit\.html
.
.IP "" 0
.
.IP "" 0
.
.P
Following example illustrates the configuration file format\.
.
.IP "" 4
.
.nf
database:
name: psycopg2
args:
dbname: synapsedb
user: synapseuser
password: ORohmi9Eet=ohphi
host: localhost
synchronous_commit: false
.
.fi
.
.IP "" 0
.
.SH "COPYRIGHT"
This man page was written by Sunil Mohan Adapa <\fIsunil@medhas\.org\fR> for Debian GNU/Linux distribution\.
.
.SH "SEE ALSO"
synctl(1), hash_password(1), register_new_matrix_user(1)

87
debian/synapse_port_db.ronn vendored Normal file
View file

@ -0,0 +1,87 @@
synapse_port_db(1) -- A script to port an existing synapse SQLite database to a new PostgreSQL database.
=============================================
## SYNOPSIS
`synapse_port_db` [-v] --sqlite-database=<dbfile> --postgres-config=<yamlconfig> [--curses] [--batch-size=<batch-size>]
## DESCRIPTION
**synapse_port_db** ports an existing synapse SQLite database to a new
PostgreSQL database.
SQLite database is specified with `--sqlite-database` option and
PostgreSQL configuration required to connect to PostgreSQL database is
provided using `--postgres-config` configuration. The configuration
is specified in YAML format.
## OPTIONS
* `-v`:
Print log messages in `debug` level instead of `info` level.
* `--sqlite-database`:
The snapshot of the SQLite database file. This must not be
currently used by a running synapse server.
* `--postgres-config`:
The database config file for the PostgreSQL database.
* `--curses`:
Display a curses based progress UI.
## CONFIG FILE
The postgres configuration file must be a valid YAML file with the
following options.
* `database`:
Database configuration section. This section header can be
ignored and the options below may be specified as top level
keys.
* `name`:
Connector to use when connecting to the database. This value must
be `psycopg2`.
* `args`:
DB API 2.0 compatible arguments to send to the `psycopg2` module.
* `dbname` - the database name
* `user` - user name used to authenticate
* `password` - password used to authenticate
* `host` - database host address (defaults to UNIX socket if not
provided)
* `port` - connection port number (defaults to 5432 if not
provided)
* `synchronous_commit`:
Optional. Default is True. If the value is `False`, enable
asynchronous commit and don't wait for the server to call fsync
before ending the transaction. See:
https://www.postgresql.org/docs/current/static/wal-async-commit.html
Following example illustrates the configuration file format.
database:
name: psycopg2
args:
dbname: synapsedb
user: synapseuser
password: ORohmi9Eet=ohphi
host: localhost
synchronous_commit: false
## COPYRIGHT
This man page was written by Sunil Mohan Adapa <<sunil@medhas.org>> for
Debian GNU/Linux distribution.
## SEE ALSO
synctl(1), hash_password(1), register_new_matrix_user(1)

63
debian/synctl.1 vendored Normal file
View file

@ -0,0 +1,63 @@
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "SYNCTL" "1" "February 2017" "" ""
.
.SH "NAME"
\fBsynctl\fR \- Synapse server control interface
.
.SH "SYNOPSIS"
Start, stop or restart synapse server\.
.
.P
\fBsynctl\fR {start|stop|restart} [configfile] [\-w|\-\-worker=\fIWORKERCONFIG\fR] [\-a|\-\-all\-processes=\fIWORKERCONFIGDIR\fR]
.
.SH "DESCRIPTION"
\fBsynctl\fR can be used to start, stop or restart Synapse server\. The control operation can be done on all processes or a single worker process\.
.
.SH "OPTIONS"
.
.TP
\fBaction\fR
The value of action should be one of \fBstart\fR, \fBstop\fR or \fBrestart\fR\.
.
.TP
\fBconfigfile\fR
Optional path of the configuration file to use\. Default value is \fBhomeserver\.yaml\fR\. The configuration file must exist for the operation to succeed\.
.
.TP
\fB\-w\fR, \fB\-\-worker\fR:
.
.IP
Perform start, stop or restart operations on a single worker\. Incompatible with \fB\-a\fR|\fB\-\-all\-processes\fR\. Value passed must be a valid worker\'s configuration file\.
.
.TP
\fB\-a\fR, \fB\-\-all\-processes\fR:
.
.IP
Perform start, stop or restart operations on all the workers in the given directory and the main synapse process\. Incompatible with \fB\-w\fR|\fB\-\-worker\fR\. Value passed must be a directory containing valid work configuration files\. All files ending with \fB\.yaml\fR extension shall be considered as configuration files and all other files in the directory are ignored\.
.
.SH "CONFIGURATION FILE"
Configuration file may be generated as follows:
.
.IP "" 4
.
.nf
$ python \-B \-m synapse\.app\.homeserver \-c config\.yaml \-\-generate\-config \-\-server\-name=<server name>
.
.fi
.
.IP "" 0
.
.SH "ENVIRONMENT"
.
.TP
\fBSYNAPSE_CACHE_FACTOR\fR
Synapse\'s architecture is quite RAM hungry currently \- a lot of recent room data and metadata is deliberately cached in RAM in order to speed up common requests\. This will be improved in future, but for now the easiest way to either reduce the RAM usage (at the risk of slowing things down) is to set the SYNAPSE_CACHE_FACTOR environment variable\. Roughly speaking, a SYNAPSE_CACHE_FACTOR of 1\.0 will max out at around 3\-4GB of resident memory \- this is what we currently run the matrix\.org on\. The default setting is currently 0\.1, which is probably around a ~700MB footprint\. You can dial it down further to 0\.02 if desired, which targets roughly ~512MB\. Conversely you can dial it up if you need performance for lots of users and have a box with a lot of RAM\.
.
.SH "COPYRIGHT"
This man page was written by Sunil Mohan Adapa <\fIsunil@medhas\.org\fR> for Debian GNU/Linux distribution\.
.
.SH "SEE ALSO"
synapse_port_db(1), hash_password(1), register_new_matrix_user(1)

70
debian/synctl.ronn vendored Normal file
View file

@ -0,0 +1,70 @@
synctl(1) -- Synapse server control interface
=============================================
## SYNOPSIS
Start, stop or restart synapse server.
`synctl` {start|stop|restart} [configfile] [-w|--worker=<WORKERCONFIG>] [-a|--all-processes=<WORKERCONFIGDIR>]
## DESCRIPTION
**synctl** can be used to start, stop or restart Synapse server. The
control operation can be done on all processes or a single worker
process.
## OPTIONS
* `action`:
The value of action should be one of `start`, `stop` or `restart`.
* `configfile`:
Optional path of the configuration file to use. Default value is
`homeserver.yaml`. The configuration file must exist for the
operation to succeed.
* `-w`, `--worker`:
Perform start, stop or restart operations on a single worker.
Incompatible with `-a`|`--all-processes`. Value passed must be a
valid worker's configuration file.
* `-a`, `--all-processes`:
Perform start, stop or restart operations on all the workers in
the given directory and the main synapse process. Incompatible
with `-w`|`--worker`. Value passed must be a directory containing
valid work configuration files. All files ending with `.yaml`
extension shall be considered as configuration files and all other
files in the directory are ignored.
## CONFIGURATION FILE
Configuration file may be generated as follows:
$ python -B -m synapse.app.homeserver -c config.yaml --generate-config --server-name=<server name>
## ENVIRONMENT
* `SYNAPSE_CACHE_FACTOR`:
Synapse's architecture is quite RAM hungry currently - a lot of
recent room data and metadata is deliberately cached in RAM in
order to speed up common requests. This will be improved in
future, but for now the easiest way to either reduce the RAM usage
(at the risk of slowing things down) is to set the
SYNAPSE_CACHE_FACTOR environment variable. Roughly speaking, a
SYNAPSE_CACHE_FACTOR of 1.0 will max out at around 3-4GB of
resident memory - this is what we currently run the matrix.org
on. The default setting is currently 0.1, which is probably around
a ~700MB footprint. You can dial it down further to 0.02 if
desired, which targets roughly ~512MB. Conversely you can dial it
up if you need performance for lots of users and have a box with a
lot of RAM.
## COPYRIGHT
This man page was written by Sunil Mohan Adapa <<sunil@medhas.org>> for
Debian GNU/Linux distribution.
## SEE ALSO
synapse_port_db(1), hash_password(1), register_new_matrix_user(1)

19
debian/templates vendored Normal file
View file

@ -0,0 +1,19 @@
Template: matrix-synapse/server-name
Type: string
_Description: Name of the server:
The name that this homeserver will appear as, to clients and other
servers via federation. This name should match the SRV record
published in DNS.
Template: matrix-synapse/report-stats
Type: boolean
Default: false
_Description: Report anonymous statistics?
Developers of Matrix and Synapse really appreciate helping the
project out by reporting anonymized usage statistics from this
homeserver. Only very basic aggregate data (e.g. number of users)
will be reported, but it helps track the growth of the Matrix
community, and helps in making Matrix a success, as well as to
convince other networks that they should peer with Matrix.
.
Thank you.

7
demo/.gitignore vendored Normal file
View file

@ -0,0 +1,7 @@
*.db
*.log
*.log.*
*.pid
/media_store.*
/etc

9
demo/demo.tls.dh
View file

@ -1,9 +0,0 @@
2048-bit DH parameters taken from rfc3526
-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEA///////////JD9qiIWjCNMTGYouA3BzRKQJOCIpnzHQCC76mOxOb
IlFKCHmONATd75UZs806QxswKwpt8l8UN0/hNW1tUcJF5IW1dmJefsb0TELppjft
awv/XLb0Brft7jhr+1qJn6WunyQRfEsf5kkoZlHs5Fs9wgB8uKFjvwWY2kg2HFXT
mmkWP6j9JM9fg2VdI9yjrZYcYvNWIIVSu57VKQdwlpZtZww1Tkq8mATxdGwIyhgh
fDKQXkYuNs474553LBgOhgObJ4Oi7Aeij7XFXfBvTFLJ3ivL9pVYFxg5lUl86pVq
5RXSJhiY+gUQFXKOWoqsqmj//////////wIBAg==
-----END DH PARAMETERS-----

24
docker/Dockerfile
View file

@ -1,3 +1,16 @@
# Dockerfile to build the matrixdotorg/synapse docker images.
#
# To build the image, run `docker build` command from the root of the
# synapse repository:
#
# docker build -f docker/Dockerfile .
#
# There is an optional PYTHON_VERSION build argument which sets the
# version of python to build against: for example:
#
# docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.6 .
#
ARG PYTHON_VERSION=2
###
@ -31,11 +44,12 @@ RUN pip install --prefix="/install" --no-warn-script-location \
# now install synapse and all of the python deps to /install.
COPY . /synapse
COPY synapse /synapse/synapse/
COPY scripts /synapse/scripts/
COPY MANIFEST.in README.rst setup.py synctl /synapse/
RUN pip install --prefix="/install" --no-warn-script-location \
lxml \
psycopg2 \
/synapse
/synapse[all]
###
### Stage 1: runtime
@ -58,6 +72,6 @@ COPY ./docker/conf /conf
VOLUME ["/data"]
EXPOSE 8008/tcp 8448/tcp
EXPOSE 8008/tcp 8009/tcp 8448/tcp
ENTRYPOINT ["/start.py"]

68
docker/Dockerfile-dhvirtualenv Normal file
View file

@ -0,0 +1,68 @@
# A dockerfile which builds a docker image for building a debian package for
# synapse. The distro to build for is passed as a docker build var.
#
# The default entrypoint expects the synapse source to be mounted as a
# (read-only) volume at /synapse/source, and an output directory at /debs.
#
# A pair of environment variables (TARGET_USERID and TARGET_GROUPID) can be
# passed to the docker container; if these are set, the build script will chown
# the build products accordingly, to avoid ending up with things owned by root
# in the host filesystem.
# Get the distro we want to pull from as a dynamic build variable
ARG distro=""
###
### Stage 0: build a dh-virtualenv
###
FROM ${distro} as builder
RUN apt-get update -qq -o Acquire::Languages=none
RUN env DEBIAN_FRONTEND=noninteractive apt-get install \
-yqq --no-install-recommends \
build-essential \
ca-certificates \
devscripts \
equivs \
wget
# fetch and unpack the package
RUN wget -q -O /dh-virtuenv-1.1.tar.gz https://github.com/spotify/dh-virtualenv/archive/1.1.tar.gz
RUN tar xvf /dh-virtuenv-1.1.tar.gz
# install its build deps
RUN cd dh-virtualenv-1.1/ \
&& env DEBIAN_FRONTEND=noninteractive mk-build-deps -ri -t "apt-get -yqq --no-install-recommends"
# build it
RUN cd dh-virtualenv-1.1 && dpkg-buildpackage -us -uc -b
###
### Stage 1
###
FROM ${distro}
# Install the build dependencies
RUN apt-get update -qq -o Acquire::Languages=none \
&& env DEBIAN_FRONTEND=noninteractive apt-get install \
-yqq --no-install-recommends -o Dpkg::Options::=--force-unsafe-io \
build-essential \
debhelper \
devscripts \
dh-systemd \
lsb-release \
python3-dev \
python3-pip \
python3-setuptools \
python3-venv \
sqlite3
COPY --from=builder /dh-virtualenv_1.1-1_all.deb /
# install dhvirtualenv. Update the apt cache again first, in case we got a
# cached cache from docker the first time.
RUN apt-get update -qq -o Acquire::Languages=none \
&& apt-get install -yq /dh-virtualenv_1.1-1_all.deb
WORKDIR /synapse/source
ENTRYPOINT ["bash","/synapse/source/docker/build_debian.sh"]

88
docker/README.md
View file

@ -1,22 +1,21 @@
# Synapse Docker
This Docker image will run Synapse as a single process. It does not provide a database
server or a TURN server, you should run these separately.
This Docker image will run Synapse as a single process. By default it uses a
sqlite database; for production use you should connect it to a separate
postgres database.
The image also does *not* provide a TURN server.
## Run
We do not currently offer a `latest` image, as this has somewhat undefined semantics.
We instead release only tagged versions so upgrading between releases is entirely
within your control.
### Using docker-compose (easier)
This image is designed to run either with an automatically generated configuration
file or with a custom configuration that requires manual editing.
This image is designed to run either with an automatically generated
configuration file or with a custom configuration that requires manual editing.
An easy way to make use of this image is via docker-compose. See the
[contrib/docker](../contrib/docker)
section of the synapse project for examples.
[contrib/docker](../contrib/docker) section of the synapse project for
examples.
### Without Compose (harder)
@ -32,7 +31,7 @@ docker run \
-v ${DATA_PATH}:/data \
-e SYNAPSE_SERVER_NAME=my.matrix.host \
-e SYNAPSE_REPORT_STATS=yes \
docker.io/matrixdotorg/synapse:latest
matrixdotorg/synapse:latest
```
## Volumes
@ -53,6 +52,28 @@ In order to setup an application service, simply create an ``appservices``
directory in the data volume and write the application service Yaml
configuration file there. Multiple application services are supported.
## TLS certificates
Synapse requires a valid TLS certificate. You can do one of the following:
* Provide your own certificate and key (as
`${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt` and
`${DATA_PATH}/${SYNAPSE_SERVER_NAME}.key`, or elsewhere by providing an
entire config as `${SYNAPSE_CONFIG_PATH}`).
* Use a reverse proxy to terminate incoming TLS, and forward the plain http
traffic to port 8008 in the container. In this case you should set `-e
SYNAPSE_NO_TLS=1`.
* Use the ACME (Let's Encrypt) support built into Synapse. This requires
`${SYNAPSE_SERVER_NAME}` port 80 to be forwarded to port 8009 in the
container, for example with `-p 80:8009`. To enable it in the docker
container, set `-e SYNAPSE_ACME=1`.
If you don't do any of these, Synapse will fail to start with an error similar to:
synapse.config._base.ConfigError: Error accessing file '/data/<server_name>.tls.crt' (config for tls_certificate): No such file or directory
## Environment
Unless you specify a custom path for the configuration file, a very generic
@ -71,7 +92,7 @@ then customize it manually. No other environment variable is required.
Otherwise, a dynamic configuration file will be used. The following environment
variables are available for configuration:
* ``SYNAPSE_SERVER_NAME`` (mandatory), the current server public hostname.
* ``SYNAPSE_SERVER_NAME`` (mandatory), the server public hostname.
* ``SYNAPSE_REPORT_STATS``, (mandatory, ``yes`` or ``no``), enable anonymous
statistics reporting back to the Matrix project which helps us to get funding.
* ``SYNAPSE_NO_TLS``, set this variable to disable TLS in Synapse (use this if
@ -80,7 +101,6 @@ variables are available for configuration:
the Synapse instance.
* ``SYNAPSE_ALLOW_GUEST``, set this variable to allow guest joining this server.
* ``SYNAPSE_EVENT_CACHE_SIZE``, the event cache size [default `10K`].
* ``SYNAPSE_CACHE_FACTOR``, the cache factor [default `0.5`].
* ``SYNAPSE_RECAPTCHA_PUBLIC_KEY``, set this variable to the recaptcha public
key in order to enable recaptcha upon registration.
* ``SYNAPSE_RECAPTCHA_PRIVATE_KEY``, set this variable to the recaptcha private
@ -88,7 +108,9 @@ variables are available for configuration:
* ``SYNAPSE_TURN_URIS``, set this variable to the coma-separated list of TURN
uris to enable TURN for this homeserver.
* ``SYNAPSE_TURN_SECRET``, set this to the TURN shared secret if required.
* ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size [default `10M`].
* ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size
[default `10M`].
* ``SYNAPSE_ACME``: set this to enable the ACME certificate renewal support.
Shared secrets, that will be initialized to random values if not set:
@ -99,27 +121,25 @@ Shared secrets, that will be initialized to random values if not set:
Database specific values (will use SQLite if not set):
* `POSTGRES_DB` - The database name for the synapse postgres database. [default: `synapse`]
* `POSTGRES_HOST` - The host of the postgres database if you wish to use postgresql instead of sqlite3. [default: `db` which is useful when using a container on the same docker network in a compose file where the postgres service is called `db`]
* `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If this is set then postgres will be used instead of sqlite3.** [default: none] **NOTE**: You are highly encouraged to use postgresql! Please use the compose file to make it easier to deploy.
* `POSTGRES_USER` - The user for the synapse postgres database. [default: `matrix`]
* `POSTGRES_DB` - The database name for the synapse postgres
database. [default: `synapse`]
* `POSTGRES_HOST` - The host of the postgres database if you wish to use
postgresql instead of sqlite3. [default: `db` which is useful when using a
container on the same docker network in a compose file where the postgres
service is called `db`]
* `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If
this is set then postgres will be used instead of sqlite3.** [default: none]
**NOTE**: You are highly encouraged to use postgresql! Please use the compose
file to make it easier to deploy.
* `POSTGRES_USER` - The user for the synapse postgres database. [default:
`matrix`]
Mail server specific values (will not send emails if not set):
* ``SYNAPSE_SMTP_HOST``, hostname to the mail server.
* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default ``25``].
* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if any.
* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail server if any.
## Build
Build the docker image with the `docker build` command from the root of the synapse repository.
```
docker build -t docker.io/matrixdotorg/synapse . -f docker/Dockerfile
```
The `-t` option sets the image tag. Official images are tagged `matrixdotorg/synapse:<version>` where `<version>` is the same as the release tag in the synapse git repository.
You may have a local Python wheel cache available, in which case copy the relevant
packages in the ``cache/`` directory at the root of the project.
* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default
``25``].
* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if
any.
* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail
server if any.

27
docker/build_debian.sh Normal file
View file

@ -0,0 +1,27 @@
#!/bin/bash
# The script to build the Debian package, as ran inside the Docker image.
set -ex
DIST=`lsb_release -c -s`
# we get a read-only copy of the source: make a writeable copy
cp -aT /synapse/source /synapse/build
cd /synapse/build
# add an entry to the changelog for this distribution
dch -M -l "+$DIST" "build for $DIST"
dch -M -r "" --force-distribution --distribution "$DIST"
dpkg-buildpackage -us -uc
ls -l ..
# copy the build results out, setting perms if necessary
shopt -s nullglob
for i in ../*.deb ../*.dsc ../*.tar.xz ../*.changes ../*.buildinfo; do
[ -z "$TARGET_USERID" ] || chown "$TARGET_USERID" "$i"
[ -z "$TARGET_GROUPID" ] || chgrp "$TARGET_GROUPID" "$i"
mv "$i" /debs
done

21
docker/conf/homeserver.yaml
View file

@ -2,11 +2,18 @@
## TLS ##
{% if not SYNAPSE_NO_TLS %}
tls_certificate_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.crt"
tls_private_key_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.key"
tls_dh_params_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.dh"
no_tls: {{ "True" if SYNAPSE_NO_TLS else "False" }}
tls_fingerprints: []
{% if SYNAPSE_ACME %}
acme:
enabled: true
port: 8009
{% endif %}
{% endif %}
## Server ##
@ -14,6 +21,7 @@ server_name: "{{ SYNAPSE_SERVER_NAME }}"
pid_file: /homeserver.pid
web_client: False
soft_file_limit: 0
log_config: "/compiled/log.config"
## Ports ##
@ -67,9 +75,6 @@ database:
## Performance ##
event_cache_size: "{{ SYNAPSE_EVENT_CACHE_SIZE or "10K" }}"
verbose: 0
log_file: "/data/homeserver.log"
log_config: "/compiled/log.config"
## Ratelimiting ##
@ -150,10 +155,12 @@ enable_group_creation: true
# The list of identity servers trusted to verify third party
# identifiers by this server.
#
# Also defines the ID server which will be called when an account is
# deactivated (one will be picked arbitrarily).
trusted_third_party_id_servers:
- matrix.org
- vector.im
- riot.im
## Metrics ###

18
docker/start.py
View file

@ -47,9 +47,8 @@ if mode == "generate":
# In normal mode, generate missing keys if any, then run synapse
else:
# Parse the configuration file
if "SYNAPSE_CONFIG_PATH" in environ:
args += ["--config-path", environ["SYNAPSE_CONFIG_PATH"]]
config_path = environ["SYNAPSE_CONFIG_PATH"]
else:
check_arguments(environ, ("SYNAPSE_SERVER_NAME", "SYNAPSE_REPORT_STATS"))
generate_secrets(environ, {
@ -58,10 +57,21 @@ else:
})
environ["SYNAPSE_APPSERVICES"] = glob.glob("/data/appservices/*.yaml")
if not os.path.exists("/compiled"): os.mkdir("/compiled")
convert("/conf/homeserver.yaml", "/compiled/homeserver.yaml", environ)
config_path = "/compiled/homeserver.yaml"
convert("/conf/homeserver.yaml", config_path, environ)
convert("/conf/log.config", "/compiled/log.config", environ)
subprocess.check_output(["chown", "-R", ownership, "/data"])
args += ["--config-path", "/compiled/homeserver.yaml"]
args += [
"--config-path", config_path,
# tell synapse to put any generated keys in /data rather than /compiled
"--keys-directory", "/data",
]
# Generate missing keys and start synapse
subprocess.check_output(args + ["--generate-keys"])
os.execv("/sbin/su-exec", ["su-exec", ownership] + args)

129
docs/ACME.md Normal file
View file

@ -0,0 +1,129 @@
# ACME
Synapse v1.0 will require valid TLS certificates for communication between
servers (port `8448` by default) in addition to those that are client-facing
(port `443`). If you do not already have a valid certificate for your domain,
the easiest way to get one is with Synapse's new ACME support, which will use
the ACME protocol to provision a certificate automatically. Synapse v0.99.0+
will provision server-to-server certificates automatically for you for free
through [Let's Encrypt](https://letsencrypt.org/) if you tell it to.
In the case that your `server_name` config variable is the same as
the hostname that the client connects to, then the same certificate can be
used between client and federation ports without issue.
If your configuration file does not already have an `acme` section, you can
generate an example config by running the `generate_config` executable. For
example:
```
~/synapse/env3/bin/generate_config
```
You will need to provide Let's Encrypt (or another ACME provider) access to
your Synapse ACME challenge responder on port 80, at the domain of your
homeserver. This requires you to either change the port of the ACME listener
provided by Synapse to a high port and reverse proxy to it, or use a tool
like `authbind` to allow Synapse to listen on port 80 without root access.
(Do not run Synapse with root permissions!) Detailed instructions are
available under "ACME setup" below.
If you already have certificates, you will need to back up or delete them
(files `example.com.tls.crt` and `example.com.tls.key` in Synapse's root
directory), Synapse's ACME implementation will not overwrite them.
You may wish to use alternate methods such as Certbot to obtain a certificate
from Let's Encrypt, depending on your server configuration. Of course, if you
already have a valid certificate for your homeserver's domain, that can be
placed in Synapse's config directory without the need for any ACME setup.
## ACME setup
The main steps for enabling ACME support in short summary are:
1. Allow Synapse to listen for incoming ACME challenges.
1. Enable ACME support in `homeserver.yaml`.
1. Move your old certificates (files `example.com.tls.crt` and `example.com.tls.key` out of the way if they currently exist at the paths specified in `homeserver.yaml`.
1. Restart Synapse.
Detailed instructions for each step are provided below.
### Listening on port 80
In order for Synapse to complete the ACME challenge to provision a
certificate, it needs access to port 80. Typically listening on port 80 is
only granted to applications running as root. There are thus two solutions to
this problem.
#### Using a reverse proxy
A reverse proxy such as Apache or nginx allows a single process (the web
server) to listen on port 80 and proxy traffic to the appropriate program
running on your server. It is the recommended method for setting up ACME as
it allows you to use your existing webserver while also allowing Synapse to
provision certificates as needed.
For nginx users, add the following line to your existing `server` block:
```
location /.well-known/acme-challenge {
proxy_pass http://localhost:8009/;
}
```
For Apache, add the following to your existing webserver config:
```
ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge
```
Make sure to restart/reload your webserver after making changes.
Now make the relevant changes in `homeserver.yaml` to enable ACME support:
```
acme:
enabled: true
port: 8009
```
#### Authbind
`authbind` allows a program which does not run as root to bind to
low-numbered ports in a controlled way. The setup is simpler, but requires a
webserver not to already be running on port 80. **This includes every time
Synapse renews a certificate**, which may be cumbersome if you usually run a
web server on port 80. Nevertheless, if you're sure port 80 is not being used
for any other purpose then all that is necessary is the following:
Install `authbind`. For example, on Debian/Ubuntu:
```
sudo apt-get install authbind
```
Allow `authbind` to bind port 80:
```
sudo touch /etc/authbind/byport/80
sudo chmod 777 /etc/authbind/byport/80
```
When Synapse is started, use the following syntax:
```
authbind --deep <synapse start command>
```
Make the relevant changes in `homeserver.yaml` to enable ACME support:
```
acme:
enabled: true
```
### (Re)starting synapse
Ensure that the certificate paths specified in `homeserver.yaml` (`tls_certificate_path` and `tls_private_key_path`) do not currently point to any files. Synapse will not provision certificates if files exist, as it does not want to overwrite existing certificates.
Finally, start/restart Synapse.

338
docs/MSC1711_certificates_FAQ.md Normal file
View file

@ -0,0 +1,338 @@
# MSC1711 Certificates FAQ
The goal of Synapse 0.99.0 is to act as a stepping stone to Synapse 1.0.0. It
supports the r0.1 release of the server to server specification, but is
compatible with both the legacy Matrix federation behaviour (pre-r0.1) as well
as post-r0.1 behaviour, in order to allow for a smooth upgrade across the
federation.
The most important thing to know is that Synapse 1.0.0 will require a valid TLS
certificate on federation endpoints. Self signed certificates will not be
sufficient.
Synapse 0.99.0 makes it easy to configure TLS certificates and will
interoperate with both >= 1.0.0 servers as well as existing servers yet to
upgrade.
**It is critical that all admins upgrade to 0.99.0 and configure a valid TLS
certificate.** Admins will have 1 month to do so, after which 1.0.0 will be
released and those servers without a valid certificate will not longer be able
to federate with >= 1.0.0 servers.
Full details on how to carry out this configuration change is given
[below](#configuring-certificates-for-compatibility-with-synapse-100). A
timeline and some frequently asked questions are also given below.
For more details and context on the release of the r0.1 Server/Server API and
imminent Matrix 1.0 release, you can also see our
[main talk from FOSDEM 2019](https://matrix.org/blog/2019/02/04/matrix-at-fosdem-2019/).
## Contents
* Timeline
* Configuring certificates for compatibility with Synapse 1.0
* FAQ
* Synapse 0.99.0 has just been released, what do I need to do right now?
* How do I upgrade?
* What will happen if I do not set up a valid federation certificate
immediately?
* What will happen if I do nothing at all?
* When do I need a SRV record or .well-known URI?
* Can I still use an SRV record?
* I have created a .well-known URI. Do I still need an SRV record?
* It used to work just fine, why are you breaking everything?
* Can I manage my own certificates rather than having Synapse renew
certificates itself?
* Do you still recommend against using a reverse proxy on the federation port?
* Do I still need to give my TLS certificates to Synapse if I am using a
reverse proxy?
* Do I need the same certificate for the client and federation port?
* How do I tell Synapse to reload my keys/certificates after I replace them?
## Timeline
**5th Feb 2019 - Synapse 0.99.0 is released.**
All server admins are encouraged to upgrade.
0.99.0:
- provides support for ACME to make setting up Let's Encrypt certs easy, as
well as .well-known support.
- does not enforce that a valid CA cert is present on the federation API, but
rather makes it easy to set one up.
- provides support for .well-known
Admins should upgrade and configure a valid CA cert. Homeservers that require a
.well-known entry (see below), should retain their SRV record and use it
alongside their .well-known record.
**>= 5th March 2019 - Synapse 1.0.0 is released**
1.0.0 will land no sooner than 1 month after 0.99.0, leaving server admins one
month after 5th February to upgrade to 0.99.0 and deploy their certificates. In
accordance with the the [S2S spec](https://matrix.org/docs/spec/server_server/r0.1.0.html)
1.0.0 will enforce certificate validity. This means that any homeserver without a
valid certificate after this point will no longer be able to federate with
1.0.0 servers.
## Configuring certificates for compatibility with Synapse 1.0.0
### If you do not currently have an SRV record
In this case, your `server_name` points to the host where your Synapse is
running. There is no need to create a `.well-known` URI or an SRV record, but
you will need to give Synapse a valid, signed, certificate.
The easiest way to do that is with Synapse's built-in ACME (Let's Encrypt)
support. Full details are in [ACME.md](./ACME.md) but, in a nutshell:
1. Allow Synapse to listen on port 80 with `authbind`, or forward it from a
reverse proxy.
2. Enable acme support in `homeserver.yaml`.
3. Move your old certificates out of the way.
4. Restart Synapse.
### If you do have an SRV record currently
If you are using an SRV record, your matrix domain (`server_name`) may not
point to the same host that your Synapse is running on (the 'target
domain'). (If it does, you can follow the recommendation above; otherwise, read
on.)
Let's assume that your `server_name` is `example.com`, and your Synapse is
hosted at a target domain of `customer.example.net`. Currently you should have
an SRV record which looks like:
```
_matrix._tcp.example.com. IN SRV 10 5 8000 customer.example.net.
```
In this situation, you have three choices for how to proceed:
#### Option 1: give Synapse a certificate for your matrix domain
Synapse 1.0 will expect your server to present a TLS certificate for your
`server_name` (`example.com` in the above example). You can achieve this by
doing one of the following:
* Acquire a certificate for the `server_name` yourself (for example, using
`certbot`), and give it and the key to Synapse via `tls_certificate_path`
and `tls_private_key_path`, or:
* Use Synapse's [ACME support](./ACME.md), and forward port 80 on the
`server_name` domain to your Synapse instance.
#### Option 2: run Synapse behind a reverse proxy
If you have an existing reverse proxy set up with correct TLS certificates for
your domain, you can simply route all traffic through the reverse proxy by
updating the SRV record appropriately (or removing it, if the proxy listens on
8448).
See [reverse_proxy.rst](reverse_proxy.rst) for information on setting up a
reverse proxy.
#### Option 3: add a .well-known file to delegate your matrix traffic
This will allow you to keep Synapse on a separate domain, without having to
give it a certificate for the matrix domain.
You can do this with a `.well-known` file as follows:
1. Keep the SRV record in place - it is needed for backwards compatibility
with Synapse 0.34 and earlier.
2. Give synapse a certificate corresponding to the target domain
(`customer.example.net` in the above example). Currently Synapse's ACME
support [does not support
this](https://github.com/matrix-org/synapse/issues/4552), so you will have
to acquire a certificate yourself and give it to Synapse via
`tls_certificate_path` and `tls_private_key_path`.
3. Restart Synapse to ensure the new certificate is loaded.
4. Arrange for a `.well-known` file at
`https://<server_name>/.well-known/matrix/server` with contents:
```json
{"m.server": "<target server name>"}
```
where the target server name is resolved as usual (i.e. SRV lookup, falling
back to talking to port 8448).
In the above example, where synapse is listening on port 8000,
`https://example.com/.well-known/matrix/server` should have `m.server` set to one of:
1. `customer.example.net` ─ with a SRV record on
`_matrix._tcp.customer.example.com` pointing to port 8000, or:
2. `customer.example.net` ─ updating synapse to listen on the default port
8448, or:
3. `customer.example.net:8000` ─ ensuring that if there is a reverse proxy
on `customer.example.net:8000` it correctly handles HTTP requests with
Host header set to `customer.example.net:8000`.
## FAQ
### Synapse 0.99.0 has just been released, what do I need to do right now?
Upgrade as soon as you can in preparation for Synapse 1.0.0, and update your
TLS certificates as [above](#configuring-certificates-for-compatibility-with-synapse-100).
### What will happen if I do not set up a valid federation certificate immediately?
Nothing initially, but once 1.0.0 is in the wild it will not be possible to
federate with 1.0.0 servers.
### What will happen if I do nothing at all?
If the admin takes no action at all, and remains on a Synapse < 0.99.0 then the
homeserver will be unable to federate with those who have implemented
.well-known. Then, as above, once the month upgrade window has expired the
homeserver will not be able to federate with any Synapse >= 1.0.0
### When do I need a SRV record or .well-known URI?
If your homeserver listens on the default federation port (8448), and your
`server_name` points to the host that your homeserver runs on, you do not need an
SRV record or `.well-known/matrix/server` URI.
For instance, if you registered `example.com` and pointed its DNS A record at a
fresh Upcloud VPS or similar, you could install Synapse 0.99 on that host,
giving it a server_name of `example.com`, and it would automatically generate a
valid TLS certificate for you via Let's Encrypt and no SRV record or
`.well-known` URI would be needed.
This is the common case, although you can add an SRV record or
`.well-known/matrix/server` URI for completeness if you wish.
**However**, if your server does not listen on port 8448, or if your `server_name`
does not point to the host that your homeserver runs on, you will need to let
other servers know how to find it.
In this case, you should see ["If you do have an SRV record
currently"](#if-you-do-have-an-srv-record-currently) above.
### Can I still use an SRV record?
Firstly, if you didn't need an SRV record before (because your server is
listening on port 8448 of your server_name), you certainly don't need one now:
the defaults are still the same.
If you previously had an SRV record, you can keep using it provided you are
able to give Synapse a TLS certificate corresponding to your server name. For
example, suppose you had the following SRV record, which directs matrix traffic
for example.com to matrix.example.com:443:
```
_matrix._tcp.example.com. IN SRV 10 5 443 matrix.example.com
```
In this case, Synapse must be given a certificate for example.com - or be
configured to acquire one from Let's Encrypt.
If you are unable to give Synapse a certificate for your server_name, you will
also need to use a .well-known URI instead. However, see also "I have created a
.well-known URI. Do I still need an SRV record?".
### I have created a .well-known URI. Do I still need an SRV record?
As of Synapse 0.99, Synapse will first check for the existence of a `.well-known`
URI and follow any delegation it suggests. It will only then check for the
existence of an SRV record.
That means that the SRV record will often be redundant. However, you should
remember that there may still be older versions of Synapse in the federation
which do not understand `.well-known` URIs, so if you removed your SRV record you
would no longer be able to federate with them.
It is therefore best to leave the SRV record in place for now. Synapse 0.34 and
earlier will follow the SRV record (and not care about the invalid
certificate). Synapse 0.99 and later will follow the .well-known URI, with the
correct certificate chain.
### It used to work just fine, why are you breaking everything?
We have always wanted Matrix servers to be as easy to set up as possible, and
so back when we started federation in 2014 we didn't want admins to have to go
through the cumbersome process of buying a valid TLS certificate to run a
server. This was before Let's Encrypt came along and made getting a free and
valid TLS certificate straightforward. So instead, we adopted a system based on
[Perspectives](https://en.wikipedia.org/wiki/Convergence_(SSL)): an approach
where you check a set of "notary servers" (in practice, homeservers) to vouch
for the validity of a certificate rather than having it signed by a CA. As long
as enough different notaries agree on the certificate's validity, then it is
trusted.
However, in practice this has never worked properly. Most people only use the
default notary server (matrix.org), leading to inadvertent centralisation which
we want to eliminate. Meanwhile, we never implemented the full consensus
algorithm to query the servers participating in a room to determine consensus
on whether a given certificate is valid. This is fiddly to get right
(especially in face of sybil attacks), and we found ourselves questioning
whether it was worth the effort to finish the work and commit to maintaining a
secure certificate validation system as opposed to focusing on core Matrix
development.
Meanwhile, Let's Encrypt came along in 2016, and put the final nail in the
coffin of the Perspectives project (which was already pretty dead). So, the
Spec Core Team decided that a better approach would be to mandate valid TLS
certificates for federation alongside the rest of the Web. More details can be
found in
[MSC1711](https://github.com/matrix-org/matrix-doc/blob/master/proposals/1711-x509-for-federation.md#background-the-failure-of-the-perspectives-approach).
This results in a breaking change, which is disruptive, but absolutely critical
for the security model. However, the existence of Let's Encrypt as a trivial
way to replace the old self-signed certificates with valid CA-signed ones helps
smooth things over massively, especially as Synapse can now automate Let's
Encrypt certificate generation if needed.
### Can I manage my own certificates rather than having Synapse renew certificates itself?
Yes, you are welcome to manage your certificates yourself. Synapse will only
attempt to obtain certificates from Let's Encrypt if you configure it to do
so.The only requirement is that there is a valid TLS cert present for
federation end points.
### Do you still recommend against using a reverse proxy on the federation port?
We no longer actively recommend against using a reverse proxy. Many admins will
find it easier to direct federation traffic to a reverse proxy and manage their
own TLS certificates, and this is a supported configuration.
See [reverse_proxy.rst](reverse_proxy.rst) for information on setting up a
reverse proxy.
### Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?
Practically speaking, this is no longer necessary.
If you are using a reverse proxy for all of your TLS traffic, then you can set
`no_tls: True`. In that case, the only reason Synapse needs the certificate is
to populate a legacy 'tls_fingerprints' field in the federation API. This is
ignored by Synapse 0.99.0 and later, and the only time pre-0.99 Synapses will
check it is when attempting to fetch the server keys - and generally this is
delegated via `matrix.org`, which is on 0.99.0.
However, there is a bug in Synapse 0.99.0
[4554](<https://github.com/matrix-org/synapse/issues/4554>) which prevents
Synapse from starting if you do not give it a TLS certificate. To work around
this, you can give it any TLS certificate at all. This will be fixed soon.
### Do I need the same certificate for the client and federation port?
No. There is nothing stopping you from using different certificates,
particularly if you are using a reverse proxy. However, Synapse will use the
same certificate on any ports where TLS is configured.
### How do I tell Synapse to reload my keys/certificates after I replace them?
Synapse will reload the keys and certificates when it receives a SIGHUP - for
example `kill -HUP $(cat homeserver.pid)`. Alternatively, simply restart
Synapse, though this will result in downtime while it restarts.

8
docs/admin_api/purge_history_api.rst
View file

@ -61,3 +61,11 @@ the following:
}
The status will be one of ``active``, ``complete``, or ``failed``.
Reclaim disk space (Postgres)
-----------------------------
To reclaim the disk space and return it to the operating system, you need to run
`VACUUM FULL;` on the database.
https://www.postgresql.org/docs/current/sql-vacuum.html

11
docs/admin_api/register_api.rst
View file

@ -39,13 +39,13 @@ As an example::
}
The MAC is the hex digest output of the HMAC-SHA1 algorithm, with the key being
the shared secret and the content being the nonce, user, password, and either
the string "admin" or "notadmin", each separated by NULs. For an example of
generation in Python::
the shared secret and the content being the nonce, user, password, either the
string "admin" or "notadmin", and optionally the user_type
each separated by NULs. For an example of generation in Python::
import hmac, hashlib
def generate_mac(nonce, user, password, admin=False):
def generate_mac(nonce, user, password, admin=False, user_type=None):
mac = hmac.new(
key=shared_secret,
@ -59,5 +59,8 @@ generation in Python::
mac.update(password.encode('utf8'))
mac.update(b"\x00")
mac.update(b"admin" if admin else b"notadmin")
if user_type:
mac.update(b"\x00")
mac.update(user_type.encode('utf8'))
return mac.hexdigest()

37
docs/consent_tracking.md
View file

@ -85,6 +85,37 @@ Once this is complete, and the server has been restarted, try visiting
an error "Missing string query parameter 'u'". It is now possible to manually
construct URIs where users can give their consent.
### Enabling consent tracking at registration
1. Add the following to your configuration:
```yaml
user_consent:
require_at_registration: true
policy_name: "Privacy Policy" # or whatever you'd like to call the policy
```
2. In your consent templates, make use of the `public_version` variable to
see if an unauthenticated user is viewing the page. This is typically
wrapped around the form that would be used to actually agree to the document:
```
{% if not public_version %}
<!-- The variables used here are only provided when the 'u' param is given to the homeserver -->
<form method="post" action="consent">
<input type="hidden" name="v" value="{{version}}"/>
<input type="hidden" name="u" value="{{user}}"/>
<input type="hidden" name="h" value="{{userhmac}}"/>
<input type="submit" value="Sure thing!"/>
</form>
{% endif %}
```
3. Restart Synapse to apply the changes.
Visiting `https://<server>/_matrix/consent` should now give you a view of the privacy
document. This is what users will be able to see when registering for accounts.
### Constructing the consent URI
It may be useful to manually construct the "consent URI" for a given user - for
@ -106,6 +137,12 @@ query parameters:
`https://<server>/_matrix/consent?u=<user>&h=68a152465a4d...`.
Note that not providing a `u` parameter will be interpreted as wanting to view
the document from an unauthenticated perspective, such as prior to registration.
Therefore, the `h` parameter is not required in this scenario. To enable this
behaviour, set `require_at_registration` to `true` in your `user_consent` config.
Sending users a server notice asking them to agree to the policy
----------------------------------------------------------------

123
docs/federate.md Normal file
View file

@ -0,0 +1,123 @@
Setting up Federation
=====================
Federation is the process by which users on different servers can participate
in the same room. For this to work, those other servers must be able to contact
yours to send messages.
The ``server_name`` configured in the Synapse configuration file (often
``homeserver.yaml``) defines how resources (users, rooms, etc.) will be
identified (eg: ``@user:example.com``, ``#room:example.com``). By
default, it is also the domain that other servers will use to
try to reach your server (via port 8448). This is easy to set
up and will work provided you set the ``server_name`` to match your
machine's public DNS hostname, and provide Synapse with a TLS certificate
which is valid for your ``server_name``.
Once you have completed the steps necessary to federate, you should be able to
join a room via federation. (A good place to start is ``#synapse:matrix.org`` - a
room for Synapse admins.)
## Delegation
For a more flexible configuration, you can have ``server_name``
resources (eg: ``@user:example.com``) served by a different host and
port (eg: ``synapse.example.com:443``). There are two ways to do this:
- adding a ``/.well-known/matrix/server`` URL served on ``https://example.com``.
- adding a DNS ``SRV`` record in the DNS zone of domain
``example.com``.
Without configuring delegation, the matrix federation will
expect to find your server via ``example.com:8448``. The following methods
allow you retain a `server_name` of `example.com` so that your user IDs, room
aliases, etc continue to look like `*:example.com`, whilst having your
federation traffic routed to a different server.
### .well-known delegation
To use this method, you need to be able to alter the
``server_name`` 's https server to serve the ``/.well-known/matrix/server``
URL. Having an active server (with a valid TLS certificate) serving your
``server_name`` domain is out of the scope of this documentation.
The URL ``https://<server_name>/.well-known/matrix/server`` should
return a JSON structure containing the key ``m.server`` like so:
{
"m.server": "<synapse.server.name>[:<yourport>]"
}
In our example, this would mean that URL ``https://example.com/.well-known/matrix/server``
should return:
{
"m.server": "synapse.example.com:443"
}
Note, specifying a port is optional. If a port is not specified an SRV lookup
is performed, as described below. If the target of the
delegation does not have an SRV record, then the port defaults to 8448.
Most installations will not need to configure .well-known. However, it can be
useful in cases where the admin is hosting on behalf of someone else and
therefore cannot gain access to the necessary certificate. With .well-known,
federation servers will check for a valid TLS certificate for the delegated
hostname (in our example: ``synapse.example.com``).
.well-known support first appeared in Synapse v0.99.0. To federate with older
servers you may need to additionally configure SRV delegation. Alternatively,
encourage the server admin in question to upgrade :).
### DNS SRV delegation
To use this delegation method, you need to have write access to your
``server_name`` 's domain zone DNS records (in our example it would be
``example.com`` DNS zone).
This method requires the target server to provide a
valid TLS certificate for the original ``server_name``.
You need to add a SRV record in your ``server_name`` 's DNS zone with
this format:
_matrix._tcp.<yourdomain.com> <ttl> IN SRV <priority> <weight> <port> <synapse.server.name>
In our example, we would need to add this SRV record in the
``example.com`` DNS zone:
_matrix._tcp.example.com. 3600 IN SRV 10 5 443 synapse.example.com.
Once done and set up, you can check the DNS record with ``dig -t srv
_matrix._tcp.<server_name>``. In our example, we would expect this:
$ dig -t srv _matrix._tcp.example.com
_matrix._tcp.example.com. 3600 IN SRV 10 0 443 synapse.example.com.
Note that the target of a SRV record cannot be an alias (CNAME record): it has to point
directly to the server hosting the synapse instance.
## Troubleshooting
You can use the [federation tester](
<https://matrix.org/federationtester>) to check if your homeserver is
configured correctly. Alternatively try the [JSON API used by the federation tester](https://matrix.org/federationtester/api/report?server_name=DOMAIN).
Note that you'll have to modify this URL to replace ``DOMAIN`` with your
``server_name``. Hitting the API directly provides extra detail.
The typical failure mode for federation is that when the server tries to join
a room, it is rejected with "401: Unauthorized". Generally this means that other
servers in the room could not access yours. (Joining a room over federation is
a complicated dance which requires connections in both directions).
Another common problem is that people on other servers can't join rooms that
you invite them to. This can be caused by an incorrectly-configured reverse
proxy: see [reverse_proxy.rst](<reverse_proxy.rst>) for instructions on how to correctly
configure a reverse proxy.
## Running a Demo Federation of Synapses
If you want to get up and running quickly with a trio of homeservers in a
private federation, there is a script in the ``demo`` directory. This is mainly
useful just for development purposes. See [demo/README](<../demo/README>).

58
docs/log_contexts.rst
View file

@ -163,7 +163,7 @@ the logcontext was set, this will make things work out ok: provided
It's all too easy to forget to ``yield``: for instance if we forgot that
``do_some_stuff`` returned a deferred, we might plough on regardless. This
leads to a mess; it will probably work itself out eventually, but not before
a load of stuff has been logged against the wrong content. (Normally, other
a load of stuff has been logged against the wrong context. (Normally, other
things will break, more obviously, if you forget to ``yield``, so this tends
not to be a major problem in practice.)
@ -440,3 +440,59 @@ To conclude: I think this scheme would have worked equally well, with less
danger of messing it up, and probably made some more esoteric code easier to
write. But again — changing the conventions of the entire Synapse codebase is
not a sensible option for the marginal improvement offered.
A note on garbage-collection of Deferred chains
-----------------------------------------------
It turns out that our logcontext rules do not play nicely with Deferred
chains which get orphaned and garbage-collected.
Imagine we have some code that looks like this:
.. code:: python
listener_queue = []
def on_something_interesting():
for d in listener_queue:
d.callback("foo")
@defer.inlineCallbacks
def await_something_interesting():
new_deferred = defer.Deferred()
listener_queue.append(new_deferred)
with PreserveLoggingContext():
yield new_deferred
Obviously, the idea here is that we have a bunch of things which are waiting
for an event. (It's just an example of the problem here, but a relatively
common one.)
Now let's imagine two further things happen. First of all, whatever was
waiting for the interesting thing goes away. (Perhaps the request times out,
or something *even more* interesting happens.)
Secondly, let's suppose that we decide that the interesting thing is never
going to happen, and we reset the listener queue:
.. code:: python
def reset_listener_queue():
listener_queue.clear()
So, both ends of the deferred chain have now dropped their references, and the
deferred chain is now orphaned, and will be garbage-collected at some point.
Note that ``await_something_interesting`` is a generator function, and when
Python garbage-collects generator functions, it gives them a chance to clean
up by making the ``yield`` raise a ``GeneratorExit`` exception. In our case,
that means that the ``__exit__`` handler of ``PreserveLoggingContext`` will
carefully restore the request context, but there is now nothing waiting for
its return, so the request context is never cleared.
To reiterate, this problem only arises when *both* ends of a deferred chain
are dropped. Dropping the the reference to a deferred you're supposed to be
calling is probably bad practice, so this doesn't actually happen too much.
Unfortunately, when it does happen, it will lead to leaked logcontexts which
are incredibly hard to track down.

3
docs/privacy_policy_templates/en/1.0.html
View file

@ -12,6 +12,8 @@
<p>
All your base are belong to us.
</p>
{% if not public_version %}
<!-- The variables used here are only provided when the 'u' param is given to the homeserver -->
<form method="post" action="consent">
<input type="hidden" name="v" value="{{version}}"/>
<input type="hidden" name="u" value="{{user}}"/>
@ -19,5 +21,6 @@
<input type="submit" value="Sure thing!"/>
</form>
{% endif %}
{% endif %}
</body>
</html>

112
docs/reverse_proxy.rst Normal file
View file

@ -0,0 +1,112 @@
Using a reverse proxy with Synapse
==================================
It is recommended to put a reverse proxy such as
`nginx <https://nginx.org/en/docs/http/ngx_http_proxy_module.html>`_,
`Apache <https://httpd.apache.org/docs/current/mod/mod_proxy_http.html>`_,
`Caddy <https://caddyserver.com/docs/proxy>`_ or
`HAProxy <https://www.haproxy.org/>`_ in front of Synapse. One advantage of
doing so is that it means that you can expose the default https port (443) to
Matrix clients without needing to run Synapse with root privileges.
**NOTE**: Your reverse proxy must not 'canonicalise' or 'normalise' the
requested URI in any way (for example, by decoding ``%xx`` escapes). Beware
that Apache *will* canonicalise URIs unless you specifify ``nocanon``.
When setting up a reverse proxy, remember that Matrix clients and other Matrix
servers do not necessarily need to connect to your server via the same server
name or port. Indeed, clients will use port 443 by default, whereas servers
default to port 8448. Where these are different, we refer to the 'client port'
and the 'federation port'. See `Setting up federation
<../README.rst#setting-up-federation>`_ for more details of the algorithm used for
federation connections.
Let's assume that we expect clients to connect to our server at
``https://matrix.example.com``, and other servers to connect at
``https://example.com:8448``. Here are some example configurations:
* nginx::
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name matrix.example.com;
location /_matrix {
proxy_pass http://localhost:8008;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
server {
listen 8448 ssl default_server;
listen [::]:8448 ssl default_server;
server_name example.com;
location / {
proxy_pass http://localhost:8008;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
* Caddy::
matrix.example.com {
proxy /_matrix http://localhost:8008 {
transparent
}
}
example.com:8448 {
proxy / http://localhost:8008 {
transparent
}
}
* Apache (note the ``nocanon`` options here!)::
<VirtualHost *:443>
SSLEngine on
ServerName matrix.example.com;
<Location /_matrix>
ProxyPass http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse http://127.0.0.1:8008/_matrix
</Location>
</VirtualHost>
<VirtualHost *:8448>
SSLEngine on
ServerName example.com;
<Location /_matrix>
ProxyPass http://127.0.0.1:8008/_matrix nocanon
ProxyPassReverse http://127.0.0.1:8008/_matrix
</Location>
</VirtualHost>
* HAProxy::
frontend https
bind 0.0.0.0:443 v4v6 ssl crt /etc/ssl/haproxy/ strict-sni alpn h2,http/1.1
bind :::443 ssl crt /etc/ssl/haproxy/ strict-sni alpn h2,http/1.1
# Matrix client traffic
acl matrix hdr(host) -i matrix.example.com
use_backend matrix if matrix
frontend matrix-federation
bind 0.0.0.0:8448 v4v6 ssl crt /etc/ssl/haproxy/synapse.pem alpn h2,http/1.1
bind :::8448 ssl crt /etc/ssl/haproxy/synapse.pem alpn h2,http/1.1
default_backend matrix
backend matrix
server matrix 127.0.0.1:8008
You will also want to set ``bind_addresses: ['127.0.0.1']`` and ``x_forwarded: true``
for port 8008 in ``homeserver.yaml`` to ensure that client IP addresses are
recorded correctly.
Having done so, you can then use ``https://matrix.example.com`` (instead of
``https://matrix.example.com:8448``) as the "Custom server" when connecting to
Synapse from a client.

26
docs/tcp_replication.rst
View file

@ -137,7 +137,6 @@ for each stream so that on reconneciton it can start streaming from the correct
place. Note: not all RDATA have valid tokens due to batching. See
``RdataCommand`` for more details.
Example
~~~~~~~
@ -221,3 +220,28 @@ SYNC (S, C)
See ``synapse/replication/tcp/commands.py`` for a detailed description and the
format of each command.
Cache Invalidation Stream
~~~~~~~~~~~~~~~~~~~~~~~~~
The cache invalidation stream is used to inform workers when they need to
invalidate any of their caches in the data store. This is done by streaming all
cache invalidations done on master down to the workers, assuming that any caches
on the workers also exist on the master.
Each individual cache invalidation results in a row being sent down replication,
which includes the cache name (the name of the function) and they key to
invalidate. For example::
> RDATA caches 550953771 ["get_user_by_id", ["@bob:example.com"], 1550574873251]
However, there are times when a number of caches need to be invalidated at the
same time with the same key. To reduce traffic we batch those invalidations into
a single poke by defining a special cache name that workers understand to mean
to expand to invalidate the correct caches.
Currently the special cache names are declared in ``synapse/storage/_base.py``
and are:
1. ``cs_cache_fake`` ─ invalidates caches that depend on the current state

3
docs/turn-howto.rst
View file

@ -40,7 +40,6 @@ You may be able to setup coturn via your package manager, or set it up manually
4. Create or edit the config file in ``/etc/turnserver.conf``. The relevant
lines, with example values, are::
lt-cred-mech
use-auth-secret
static-auth-secret=[your secret key here]
realm=turn.myserver.org
@ -52,7 +51,7 @@ You may be able to setup coturn via your package manager, or set it up manually
5. Consider your security settings. TURN lets users request a relay
which will connect to arbitrary IP addresses and ports. At the least
we recommend:
we recommend::
# VoIP traffic is all UDP. There is no reason to let users connect to arbitrary TCP endpoints via the relay.
no-tcp-relay

12
docs/workers.rst
View file

@ -26,9 +26,8 @@ Configuration
To make effective use of the workers, you will need to configure an HTTP
reverse-proxy such as nginx or haproxy, which will direct incoming requests to
the correct worker, or to the main synapse instance. Note that this includes
requests made to the federation port. The caveats regarding running a
reverse-proxy on the federation port still apply (see
https://github.com/matrix-org/synapse/blob/master/README.rst#reverse-proxying-the-federation-port).
requests made to the federation port. See `<reverse_proxy.rst>`_ for
information on setting up a reverse proxy.
To enable workers, you need to add two replication listeners to the master
synapse, e.g.::
@ -223,6 +222,13 @@ following regular expressions::
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/context/.*$
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/members$
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/state$
^/_matrix/client/(api/v1|r0|unstable)/login$
Additionally, the following REST endpoints can be handled, but all requests must
be routed to the same instance::
^/_matrix/client/(r0|unstable)/register$
``synapse.app.user_dir``
~~~~~~~~~~~~~~~~~~~~~~~~

19
jenkins/prepare_synapse.sh
View file

@ -14,22 +14,3 @@ fi
# set up the virtualenv
tox -e py27 --notest -v
TOX_BIN=$TOX_DIR/py27/bin
# cryptography 2.2 requires setuptools >= 18.5.
#
# older versions of virtualenv (?) give us a virtualenv with the same version
# of setuptools as is installed on the system python (and tox runs virtualenv
# under python3, so we get the version of setuptools that is installed on that).
#
# anyway, make sure that we have a recent enough setuptools.
$TOX_BIN/pip install 'setuptools>=18.5'
# we also need a semi-recent version of pip, because old ones fail to install
# the "enum34" dependency of cryptography.
$TOX_BIN/pip install 'pip>=10'
{ python synapse/python_dependencies.py
echo lxml
} | xargs $TOX_BIN/pip install

154
scripts-dev/build_debian_packages Executable file
View file

@ -0,0 +1,154 @@
#!/usr/bin/env python3
# Build the Debian packages using Docker images.
#
# This script builds the Docker images and then executes them sequentially, each
# one building a Debian package for the targeted operating system. It is
# designed to be a "single command" to produce all the images.
#
# By default, builds for all known distributions, but a list of distributions
# can be passed on the commandline for debugging.
import argparse
import os
import signal
import subprocess
import sys
import threading
from concurrent.futures import ThreadPoolExecutor
DISTS = (
"debian:stretch",
"debian:buster",
"debian:sid",
"ubuntu:xenial",
"ubuntu:bionic",
"ubuntu:cosmic",
)
DESC = '''\
Builds .debs for synapse, using a Docker image for the build environment.
By default, builds for all known distributions, but a list of distributions
can be passed on the commandline for debugging.
'''
class Builder(object):
def __init__(self, redirect_stdout=False):
self.redirect_stdout = redirect_stdout
self.active_containers = set()
self._lock = threading.Lock()
self._failed = False
def run_build(self, dist):
"""Build deb for a single distribution"""
if self._failed:
print("not building %s due to earlier failure" % (dist, ))
raise Exception("failed")
try:
self._inner_build(dist)
except Exception as e:
print("build of %s failed: %s" % (dist, e), file=sys.stderr)
self._failed = True
raise
def _inner_build(self, dist):
projdir = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
os.chdir(projdir)
tag = dist.split(":", 1)[1]
# Make the dir where the debs will live.
#
# Note that we deliberately put this outside the source tree, otherwise
# we tend to get source packages which are full of debs. (We could hack
# around that with more magic in the build_debian.sh script, but that
# doesn't solve the problem for natively-run dpkg-buildpakage).
debsdir = os.path.join(projdir, '../debs')
os.makedirs(debsdir, exist_ok=True)
if self.redirect_stdout:
logfile = os.path.join(debsdir, "%s.buildlog" % (tag, ))
print("building %s: directing output to %s" % (dist, logfile))
stdout = open(logfile, "w")
else:
stdout = None
# first build a docker image for the build environment
subprocess.check_call([
"docker", "build",
"--tag", "dh-venv-builder:" + tag,
"--build-arg", "distro=" + dist,
"-f", "docker/Dockerfile-dhvirtualenv",
"docker",
], stdout=stdout, stderr=subprocess.STDOUT)
container_name = "synapse_build_" + tag
with self._lock:
self.active_containers.add(container_name)
# then run the build itself
subprocess.check_call([
"docker", "run",
"--rm",
"--name", container_name,
"--volume=" + projdir + ":/synapse/source:ro",
"--volume=" + debsdir + ":/debs",
"-e", "TARGET_USERID=%i" % (os.getuid(), ),
"-e", "TARGET_GROUPID=%i" % (os.getgid(), ),
"dh-venv-builder:" + tag,
], stdout=stdout, stderr=subprocess.STDOUT)
with self._lock:
self.active_containers.remove(container_name)
if stdout is not None:
stdout.close()
print("Completed build of %s" % (dist, ))
def kill_containers(self):
with self._lock:
active = list(self.active_containers)
for c in active:
print("killing container %s" % (c,))
subprocess.run([
"docker", "kill", c,
], stdout=subprocess.DEVNULL)
with self._lock:
self.active_containers.remove(c)
def run_builds(dists, jobs=1):
builder = Builder(redirect_stdout=(jobs > 1))
def sig(signum, _frame):
print("Caught SIGINT")
builder.kill_containers()
signal.signal(signal.SIGINT, sig)
with ThreadPoolExecutor(max_workers=jobs) as e:
res = e.map(builder.run_build, dists)
# make sure we consume the iterable so that exceptions are raised.
for r in res:
pass
if __name__ == '__main__':
parser = argparse.ArgumentParser(
description=DESC,
)
parser.add_argument(
'-j', '--jobs', type=int, default=1,
help='specify the number of builds to run in parallel',
)
parser.add_argument(
'dist', nargs='*', default=DISTS,
help='a list of distributions to build for. Default: %(default)s',
)
args = parser.parse_args()
run_builds(dists=args.dist, jobs=args.jobs)

41
scripts-dev/check-newsfragment Executable file
View file

@ -0,0 +1,41 @@
#!/bin/bash
#
# A script which checks that an appropriate news file has been added on this
# branch.
set -e
# make sure that origin/develop is up to date
git remote set-branches --add origin develop
git fetch --depth=1 origin develop
UPSTREAM=origin/develop
# if there are changes in the debian directory, check that the debian changelog
# has been updated
if ! git diff --quiet $UPSTREAM... -- debian; then
if git diff --quiet $UPSTREAM... -- debian/changelog; then
echo "Updates to debian directory, but no update to the changelog." >&2
exit 1
fi
fi
# if there are changes *outside* the debian directory, check that the
# newsfragments have been updated.
if git diff --name-only $UPSTREAM... | grep -qv '^develop/'; then
tox -e check-newsfragment
fi
echo
echo "--------------------------"
echo
# check that any new newsfiles on this branch end with a full stop.
for f in `git diff --name-only $UPSTREAM... -- changelog.d`; do
lastchar=`tr -d '\n' < $f | tail -c 1`
if [ $lastchar != '.' ]; then
echo -e "\e[31mERROR: newsfragment $f does not end with a '.'\e[39m" >&2
exit 1
fi
done

9
scripts-dev/federation_client.py
View file

@ -154,10 +154,15 @@ def request_json(method, origin_name, origin_key, destination, path, content):
s = requests.Session()
s.mount("matrix://", MatrixConnectionAdapter())
headers = {"Host": destination, "Authorization": authorization_headers[0]}
if method == "POST":
headers["Content-Type"] = "application/json"
result = s.request(
method=method,
url=dest,
headers={"Host": destination, "Authorization": authorization_headers[0]},
headers=headers,
verify=False,
data=content,
)
@ -203,7 +208,7 @@ def main():
parser.add_argument(
"-X",
"--method",
help="HTTP method to use for the request. Defaults to GET if --data is"
help="HTTP method to use for the request. Defaults to GET if --body is"
"unspecified, POST if it is.",
)

39
scripts-dev/make_identicons.pl
View file

@ -1,39 +0,0 @@
#!/usr/bin/env perl
use strict;
use warnings;
use DBI;
use DBD::SQLite;
use JSON;
use Getopt::Long;
my $db; # = "homeserver.db";
my $server = "http://localhost:8008";
my $size = 320;
GetOptions("db|d=s", \$db,
"server|s=s", \$server,
"width|w=i", \$size) or usage();
usage() unless $db;
my $dbh = DBI->connect("dbi:SQLite:dbname=$db","","") || die $DBI::errstr;
my $res = $dbh->selectall_arrayref("select token, name from access_tokens, users where access_tokens.user_id = users.id group by user_id") || die $DBI::errstr;
foreach (@$res) {
my ($token, $mxid) = ($_->[0], $_->[1]);
my ($user_id) = ($mxid =~ m/@(.*):/);
my ($url) = $dbh->selectrow_array("select avatar_url from profiles where user_id=?", undef, $user_id);
if (!$url || $url =~ /#auto$/) {
`curl -s -o tmp.png "$server/_matrix/media/v1/identicon?name=${mxid}&width=$size&height=$size"`;
my $json = `curl -s -X POST -H "Content-Type: image/png" -T "tmp.png" $server/_matrix/media/v1/upload?access_token=$token`;
my $content_uri = from_json($json)->{content_uri};
`curl -X PUT -H "Content-Type: application/json" --data '{ "avatar_url": "${content_uri}#auto"}' $server/_matrix/client/api/v1/profile/${mxid}/avatar_url?access_token=$token`;
}
}
sub usage {
die "usage: ./make-identicons.pl\n\t-d database [e.g. homeserver.db]\n\t-s homeserver (default: http://localhost:8008)\n\t-w identicon size in pixels (default 320)";
}

67
scripts/generate_config Executable file
View file

@ -0,0 +1,67 @@
#!/usr/bin/env python
import argparse
import sys
from synapse.config.homeserver import HomeServerConfig
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"--config-dir",
default="CONFDIR",
help="The path where the config files are kept. Used to create filenames for "
"things like the log config and the signing key. Default: %(default)s",
)
parser.add_argument(
"--data-dir",
default="DATADIR",
help="The path where the data files are kept. Used to create filenames for "
"things like the database and media store. Default: %(default)s",
)
parser.add_argument(
"--server-name",
default="SERVERNAME",
help="The server name. Used to initialise the server_name config param, but also "
"used in the names of some of the config files. Default: %(default)s",
)
parser.add_argument(
"--report-stats",
action="store",
help="Whether the generated config reports anonymized usage statistics",
choices=["yes", "no"],
)
parser.add_argument(
"--generate-secrets",
action="store_true",
help="Enable generation of new secrets for things like the macaroon_secret_key."
"By default, these parameters will be left unset."
)
parser.add_argument(
"-o", "--output-file",
type=argparse.FileType('w'),
default=sys.stdout,
help="File to write the configuration to. Default: stdout",
)
args = parser.parse_args()
report_stats = args.report_stats
if report_stats is not None:
report_stats = report_stats == "yes"
conf = HomeServerConfig().generate_config(
config_dir_path=args.config_dir,
data_dir_path=args.data_dir,
server_name=args.server_name,
generate_secrets=args.generate_secrets,
report_stats=report_stats,
)
args.output_file.write(conf)

39
scripts/hash_password
View file

@ -3,6 +3,7 @@
import argparse
import getpass
import sys
import unicodedata
import bcrypt
import yaml
@ -10,6 +11,7 @@ import yaml
bcrypt_rounds = 12
password_pepper = ""
def prompt_for_pass():
password = getpass.getpass("Password: ")
@ -23,19 +25,27 @@ def prompt_for_pass():
return password
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Calculate the hash of a new password, so that passwords"
" can be reset")
description=(
"Calculate the hash of a new password, so that passwords can be reset"
)
)
parser.add_argument(
"-p", "--password",
"-p",
"--password",
default=None,
help="New password for user. Will prompt if omitted.",
)
parser.add_argument(
"-c", "--config",
"-c",
"--config",
type=argparse.FileType('r'),
help="Path to server config file. Used to read in bcrypt_rounds and password_pepper.",
help=(
"Path to server config file. "
"Used to read in bcrypt_rounds and password_pepper."
),
)
args = parser.parse_args()
@ -49,4 +59,21 @@ if __name__ == "__main__":
if not password:
password = prompt_for_pass()
print bcrypt.hashpw(password + password_pepper, bcrypt.gensalt(bcrypt_rounds))
# On Python 2, make sure we decode it to Unicode before we normalise it
if isinstance(password, bytes):
try:
password = password.decode(sys.stdin.encoding)
except UnicodeDecodeError:
print(
"ERROR! Your password is not decodable using your terminal encoding (%s)."
% (sys.stdin.encoding,)
)
pw = unicodedata.normalize("NFKC", password)
hashed = bcrypt.hashpw(
pw.encode('utf8') + password_pepper.encode("utf8"),
bcrypt.gensalt(bcrypt_rounds),
).decode('ascii')
print(hashed)

1
scripts/synapse_port_db
View file

@ -53,6 +53,7 @@ BOOLEAN_COLUMNS = {
"group_summary_users": ["is_public"],
"group_roles": ["is_public"],
"local_group_membership": ["is_publicised", "is_admin"],
"e2e_room_keys": ["is_verified"],
}

16
setup.py
View file

@ -84,13 +84,25 @@ version = exec_file(("synapse", "__init__.py"))["__version__"]
dependencies = exec_file(("synapse", "python_dependencies.py"))
long_description = read_file(("README.rst",))
REQUIREMENTS = dependencies['REQUIREMENTS']
CONDITIONAL_REQUIREMENTS = dependencies['CONDITIONAL_REQUIREMENTS']
# Make `pip install matrix-synapse[all]` install all the optional dependencies.
ALL_OPTIONAL_REQUIREMENTS = set()
for optional_deps in CONDITIONAL_REQUIREMENTS.values():
ALL_OPTIONAL_REQUIREMENTS = set(optional_deps) | ALL_OPTIONAL_REQUIREMENTS
CONDITIONAL_REQUIREMENTS["all"] = list(ALL_OPTIONAL_REQUIREMENTS)
setup(
name="matrix-synapse",
version=version,
packages=find_packages(exclude=["tests", "tests.*"]),
description="Reference homeserver for the Matrix decentralised comms protocol",
install_requires=dependencies['requirements'](include_conditional=True).keys(),
dependency_links=dependencies["DEPENDENCY_LINKS"].values(),
install_requires=REQUIREMENTS,
extras_require=CONDITIONAL_REQUIREMENTS,
include_package_data=True,
zip_safe=False,
long_description=long_description,

4
synapse/__init__.py
View file

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# Copyright 2014-2016 OpenMarket Ltd
# Copyright 2018 New Vector Ltd
# Copyright 2018-9 New Vector Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -27,4 +27,4 @@ try:
except ImportError:
pass
__version__ = "0.33.8"
__version__ = "0.99.2"

23
synapse/_scripts/register_new_matrix_user.py
View file

@ -35,6 +35,7 @@ def request_registration(
server_location,
shared_secret,
admin=False,
user_type=None,
requests=_requests,
_print=print,
exit=sys.exit,
@ -45,7 +46,7 @@ def request_registration(
# Get the nonce
r = requests.get(url, verify=False)
if r.status_code is not 200:
if r.status_code != 200:
_print("ERROR! Received %d %s" % (r.status_code, r.reason))
if 400 <= r.status_code < 500:
try:
@ -65,6 +66,9 @@ def request_registration(
mac.update(password.encode('utf8'))
mac.update(b"\x00")
mac.update(b"admin" if admin else b"notadmin")
if user_type:
mac.update(b"\x00")
mac.update(user_type.encode('utf8'))
mac = mac.hexdigest()
@ -74,12 +78,13 @@ def request_registration(
"password": password,
"mac": mac,
"admin": admin,
"user_type": user_type,
}
_print("Sending registration request...")
r = requests.post(url, json=data, verify=False)
if r.status_code is not 200:
if r.status_code != 200:
_print("ERROR! Received %d %s" % (r.status_code, r.reason))
if 400 <= r.status_code < 500:
try:
@ -91,7 +96,7 @@ def request_registration(
_print("Success!")
def register_new_user(user, password, server_location, shared_secret, admin):
def register_new_user(user, password, server_location, shared_secret, admin, user_type):
if not user:
try:
default_user = getpass.getuser()
@ -129,7 +134,8 @@ def register_new_user(user, password, server_location, shared_secret, admin):
else:
admin = False
request_registration(user, password, server_location, shared_secret, bool(admin))
request_registration(user, password, server_location, shared_secret,
bool(admin), user_type)
def main():
@ -154,6 +160,12 @@ def main():
default=None,
help="New password for user. Will prompt if omitted.",
)
parser.add_argument(
"-t",
"--user_type",
default=None,
help="User type as specified in synapse.api.constants.UserTypes",
)
admin_group = parser.add_mutually_exclusive_group()
admin_group.add_argument(
"-a",
@ -208,7 +220,8 @@ def main():
if args.admin or args.no_admin:
admin = args.admin
register_new_user(args.user, args.password, args.server_url, secret, admin)
register_new_user(args.user, args.password, args.server_url, secret,
admin, args.user_type)
if __name__ == "__main__":

132
synapse/api/auth.py
View file

@ -65,7 +65,7 @@ class Auth(object):
register_cache("cache", "token_cache", self.token_cache)
@defer.inlineCallbacks
def check_from_context(self, event, context, do_sig_check=True):
def check_from_context(self, room_version, event, context, do_sig_check=True):
prev_state_ids = yield context.get_prev_state_ids(self.store)
auth_events_ids = yield self.compute_auth_events(
event, prev_state_ids, for_verification=True,
@ -74,12 +74,16 @@ class Auth(object):
auth_events = {
(e.type, e.state_key): e for e in itervalues(auth_events)
}
self.check(event, auth_events=auth_events, do_sig_check=do_sig_check)
self.check(
room_version, event,
auth_events=auth_events, do_sig_check=do_sig_check,
)
def check(self, event, auth_events, do_sig_check=True):
def check(self, room_version, event, auth_events, do_sig_check=True):
""" Checks if this event is correctly authed.
Args:
room_version (str): version of the room
event: the event being checked.
auth_events (dict: event-key -> event): the existing room state.
@ -88,7 +92,9 @@ class Auth(object):
True if the auth checks pass.
"""
with Measure(self.clock, "auth.check"):
event_auth.check(event, auth_events, do_sig_check=do_sig_check)
event_auth.check(
room_version, event, auth_events, do_sig_check=do_sig_check
)
@defer.inlineCallbacks
def check_joined_room(self, room_id, user_id, current_state=None):
@ -188,16 +194,32 @@ class Auth(object):
"""
# Can optionally look elsewhere in the request (e.g. headers)
try:
ip_addr = self.hs.get_ip_from_request(request)
user_agent = request.requestHeaders.getRawHeaders(
b"User-Agent",
default=[b""]
)[0].decode('ascii', 'surrogateescape')
access_token = self.get_access_token_from_request(
request, self.TOKEN_NOT_FOUND_HTTP_STATUS
)
user_id, app_service = yield self._get_appservice_user_id(request)
if user_id:
request.authenticated_entity = user_id
defer.returnValue(
synapse.types.create_requester(user_id, app_service=app_service)
if ip_addr and self.hs.config.track_appservice_user_ips:
yield self.store.insert_client_ip(
user_id=user_id,
access_token=access_token,
ip=ip_addr,
user_agent=user_agent,
device_id="dummy-device", # stubbed
)
access_token = self.get_access_token_from_request(
request, self.TOKEN_NOT_FOUND_HTTP_STATUS
defer.returnValue(
synapse.types.create_requester(user_id, app_service=app_service)
)
user_info = yield self.get_user_by_access_token(access_token, rights)
@ -209,11 +231,6 @@ class Auth(object):
# stubbed out.
device_id = user_info.get("device_id")
ip_addr = self.hs.get_ip_from_request(request)
user_agent = request.requestHeaders.getRawHeaders(
b"User-Agent",
default=[b""]
)[0].decode('ascii', 'surrogateescape')
if user and access_token and ip_addr:
yield self.store.insert_client_ip(
user_id=user.to_string(),
@ -291,20 +308,28 @@ class Auth(object):
Raises:
AuthError if no user by that token exists or the token is invalid.
"""
try:
user_id, guest = self._parse_and_validate_macaroon(token, rights)
except _InvalidMacaroonException:
# doesn't look like a macaroon: treat it as an opaque token which
# must be in the database.
# TODO: it would be nice to get rid of this, but apparently some
# people use access tokens which aren't macaroons
if rights == "access":
# first look in the database
r = yield self._look_up_user_by_access_token(token)
if r:
defer.returnValue(r)
# otherwise it needs to be a valid macaroon
try:
user_id, guest = self._parse_and_validate_macaroon(token, rights)
user = UserID.from_string(user_id)
if guest:
if rights == "access":
if not guest:
# non-guest access tokens must be in the database
logger.warning("Unrecognised access token - not in store.")
raise AuthError(
self.TOKEN_NOT_FOUND_HTTP_STATUS,
"Unrecognised access token.",
errcode=Codes.UNKNOWN_TOKEN,
)
# Guest access tokens are not stored in the database (there can
# only be one access token per guest, anyway).
#
@ -345,31 +370,15 @@ class Auth(object):
"device_id": None,
}
else:
# This codepath exists for several reasons:
# * so that we can actually return a token ID, which is used
# in some parts of the schema (where we probably ought to
# use device IDs instead)
# * the only way we currently have to invalidate an
# access_token is by removing it from the database, so we
# have to check here that it is still in the db
# * some attributes (notably device_id) aren't stored in the
# macaroon. They probably should be.
# TODO: build the dictionary from the macaroon once the
# above are fixed
ret = yield self._look_up_user_by_access_token(token)
if ret["user"] != user:
logger.error(
"Macaroon user (%s) != DB user (%s)",
user,
ret["user"]
)
raise AuthError(
self.TOKEN_NOT_FOUND_HTTP_STATUS,
"User mismatch in macaroon",
errcode=Codes.UNKNOWN_TOKEN
)
raise RuntimeError("Unknown rights setting %s", rights)
defer.returnValue(ret)
except (pymacaroons.exceptions.MacaroonException, TypeError, ValueError):
except (
_InvalidMacaroonException,
pymacaroons.exceptions.MacaroonException,
TypeError,
ValueError,
) as e:
logger.warning("Invalid macaroon in auth: %s %s", type(e), e)
raise AuthError(
self.TOKEN_NOT_FOUND_HTTP_STATUS, "Invalid macaroon passed.",
errcode=Codes.UNKNOWN_TOKEN
@ -499,11 +508,8 @@ class Auth(object):
def _look_up_user_by_access_token(self, token):
ret = yield self.store.get_user_by_access_token(token)
if not ret:
logger.warn("Unrecognised access token - not in store.")
raise AuthError(
self.TOKEN_NOT_FOUND_HTTP_STATUS, "Unrecognised access token.",
errcode=Codes.UNKNOWN_TOKEN
)
defer.returnValue(None)
# we use ret.get() below because *lots* of unit tests stub out
# get_user_by_access_token in a way where it only returns a couple of
# the fields.
@ -531,17 +537,6 @@ class Auth(object):
"""
return self.store.is_server_admin(user)
@defer.inlineCallbacks
def add_auth_events(self, builder, context):
prev_state_ids = yield context.get_prev_state_ids(self.store)
auth_ids = yield self.compute_auth_events(builder, prev_state_ids)
auth_events_entries = yield self.store.add_event_hashes(
auth_ids
)
builder.auth_events = auth_events_entries
@defer.inlineCallbacks
def compute_auth_events(self, event, current_state_ids, for_verification=False):
if event.type == EventTypes.Create:
@ -558,7 +553,7 @@ class Auth(object):
key = (EventTypes.JoinRules, "", )
join_rule_event_id = current_state_ids.get(key)
key = (EventTypes.Member, event.user_id, )
key = (EventTypes.Member, event.sender, )
member_event_id = current_state_ids.get(key)
key = (EventTypes.Create, "", )
@ -608,7 +603,7 @@ class Auth(object):
defer.returnValue(auth_ids)
def check_redaction(self, event, auth_events):
def check_redaction(self, room_version, event, auth_events):
"""Check whether the event sender is allowed to redact the target event.
Returns:
@ -621,7 +616,7 @@ class Auth(object):
AuthError if the event sender is definitely not allowed to redact
the target event.
"""
return event_auth.check_redaction(event, auth_events)
return event_auth.check_redaction(room_version, event, auth_events)
@defer.inlineCallbacks
def check_can_change_room_list(self, room_id, user):
@ -778,9 +773,10 @@ class Auth(object):
threepid should never be set at the same time.
"""
# Never fail an auth check for the server notices users
# Never fail an auth check for the server notices users or support user
# This can be a problem where event creation is prohibited due to blocking
if user_id == self.hs.config.server_notices_mxid:
is_support = yield self.store.is_support_user(user_id)
if user_id == self.hs.config.server_notices_mxid or is_support:
return
if self.hs.config.hs_disabled:
@ -805,7 +801,9 @@ class Auth(object):
elif threepid:
# If the user does not exist yet, but is signing up with a
# reserved threepid then pass auth check
if is_threepid_reserved(self.hs.config, threepid):
if is_threepid_reserved(
self.hs.config.mau_limits_reserved_threepids, threepid
):
return
# Else if there is no room in the MAU bucket, bail
current_mau = yield self.store.get_monthly_active_count()

44
synapse/api/constants.py
View file

@ -51,6 +51,7 @@ class LoginType(object):
EMAIL_IDENTITY = u"m.login.email.identity"
MSISDN = u"m.login.msisdn"
RECAPTCHA = u"m.login.recaptcha"
TERMS = u"m.login.terms"
DUMMY = u"m.login.dummy"
# Only for C/S API v1
@ -61,15 +62,18 @@ class LoginType(object):
class EventTypes(object):
Member = "m.room.member"
Create = "m.room.create"
Tombstone = "m.room.tombstone"
JoinRules = "m.room.join_rules"
PowerLevels = "m.room.power_levels"
Aliases = "m.room.aliases"
Redaction = "m.room.redaction"
ThirdPartyInvite = "m.room.third_party_invite"
Encryption = "m.room.encryption"
RoomHistoryVisibility = "m.room.history_visibility"
CanonicalAlias = "m.room.canonical_alias"
RoomAvatar = "m.room.avatar"
RoomEncryption = "m.room.encryption"
GuestAccess = "m.room.guest_access"
Encryption = "m.room.encryption"
@ -101,7 +105,14 @@ class ThirdPartyEntityKind(object):
class RoomVersions(object):
V1 = "1"
VDH_TEST = "vdh-test-version"
V2 = "2"
V3 = "3"
STATE_V2_TEST = "state-v2-test"
class RoomDisposition(object):
STABLE = "stable"
UNSTABLE = "unstable"
# the version we will give rooms which are created on this server
@ -109,7 +120,36 @@ DEFAULT_ROOM_VERSION = RoomVersions.V1
# vdh-test-version is a placeholder to get room versioning support working and tested
# until we have a working v2.
KNOWN_ROOM_VERSIONS = {RoomVersions.V1, RoomVersions.VDH_TEST}
KNOWN_ROOM_VERSIONS = {
RoomVersions.V1,
RoomVersions.V2,
RoomVersions.V3,
RoomVersions.STATE_V2_TEST,
RoomVersions.V3,
}
class EventFormatVersions(object):
"""This is an internal enum for tracking the version of the event format,
independently from the room version.
"""
V1 = 1
V2 = 2
KNOWN_EVENT_FORMAT_VERSIONS = {
EventFormatVersions.V1,
EventFormatVersions.V2,
}
ServerNoticeMsgType = "m.server_notice"
ServerNoticeLimitReached = "m.server_notice.usage_limit_reached"
class UserTypes(object):
"""Allows for user type specific behaviour. With the benefit of hindsight
'admin' and 'guest' users should also be UserTypes. Normal users are type None
"""
SUPPORT = "support"
ALL_USER_TYPES = (SUPPORT,)

18
synapse/api/errors.py
View file

@ -348,6 +348,24 @@ class IncompatibleRoomVersionError(SynapseError):
)
class RequestSendFailed(RuntimeError):
"""Sending a HTTP request over federation failed due to not being able to
talk to the remote server for some reason.
This exception is used to differentiate "expected" errors that arise due to
networking (e.g. DNS failures, connection timeouts etc), versus unexpected
errors (like programming errors).
"""
def __init__(self, inner_exception, can_retry):
super(RequestSendFailed, self).__init__(
"Failed to send request: %s: %s" % (
type(inner_exception).__name__, inner_exception,
)
)
self.inner_exception = inner_exception
self.can_retry = can_retry
def cs_error(msg, code=Codes.UNKNOWN, **kwargs):
""" Utility method for constructing an error response for client-server
interactions.

25
synapse/api/filtering.py
View file

@ -12,6 +12,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from six import text_type
import jsonschema
from canonicaljson import json
from jsonschema import FormatChecker
@ -353,7 +355,7 @@ class Filter(object):
sender = event.user_id
room_id = None
ev_type = "m.presence"
is_url = False
contains_url = False
else:
sender = event.get("sender", None)
if not sender:
@ -368,13 +370,16 @@ class Filter(object):
room_id = event.get("room_id", None)
ev_type = event.get("type", None)
is_url = "url" in event.get("content", {})
content = event.get("content", {})
# check if there is a string url field in the content for filtering purposes
contains_url = isinstance(content.get("url"), text_type)
return self.check_fields(
room_id,
sender,
ev_type,
is_url,
contains_url,
)
def check_fields(self, room_id, sender, event_type, contains_url):
@ -439,6 +444,20 @@ class Filter(object):
def include_redundant_members(self):
return self.filter_json.get("include_redundant_members", False)
def with_room_ids(self, room_ids):
"""Returns a new filter with the given room IDs appended.
Args:
room_ids (iterable[unicode]): The room_ids to add
Returns:
filter: A new filter including the given rooms and the old
filter's rooms.
"""
newFilter = Filter(self.filter_json)
newFilter.rooms += room_ids
return newFilter
def _matches_wildcard(actual_value, filter_value):
if filter_value.endswith("*"):

5
synapse/api/urls.py
View file

@ -24,11 +24,12 @@ from synapse.config import ConfigError
CLIENT_PREFIX = "/_matrix/client/api/v1"
CLIENT_V2_ALPHA_PREFIX = "/_matrix/client/v2_alpha"
FEDERATION_PREFIX = "/_matrix/federation/v1"
FEDERATION_PREFIX = "/_matrix/federation"
FEDERATION_V1_PREFIX = FEDERATION_PREFIX + "/v1"
FEDERATION_V2_PREFIX = FEDERATION_PREFIX + "/v2"
STATIC_PREFIX = "/_matrix/static"
WEB_CLIENT_PREFIX = "/_matrix/client"
CONTENT_REPO_PREFIX = "/_matrix/content"
SERVER_KEY_PREFIX = "/_matrix/key/v1"
SERVER_KEY_V2_PREFIX = "/_matrix/key/v2"
MEDIA_PREFIX = "/_matrix/media/r0"
LEGACY_MEDIA_PREFIX = "/_matrix/media/v1"

34
synapse/app/__init__.py
View file

@ -12,22 +12,38 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
import sys
from synapse import python_dependencies # noqa: E402
sys.dont_write_bytecode = True
logger = logging.getLogger(__name__)
try:
python_dependencies.check_requirements()
except python_dependencies.MissingRequirementError as e:
message = "\n".join([
"Missing Requirement: %s" % (str(e),),
"To install run:",
" pip install --upgrade --force \"%s\"" % (e.dependency,),
"",
])
sys.stderr.writelines(message)
except python_dependencies.DependencyException as e:
sys.stderr.writelines(e.message)
sys.exit(1)
def check_bind_error(e, address, bind_addresses):
"""
This method checks an exception occurred while binding on 0.0.0.0.
If :: is specified in the bind addresses a warning is shown.
The exception is still raised otherwise.
Binding on both 0.0.0.0 and :: causes an exception on Linux and macOS
because :: binds on both IPv4 and IPv6 (as per RFC 3493).
When binding on 0.0.0.0 after :: this can safely be ignored.
Args:
e (Exception): Exception that was caught.
address (str): Address on which binding was attempted.
bind_addresses (list): Addresses on which the service listens.
"""
if address == '0.0.0.0' and '::' in bind_addresses:
logger.warn('Failed to listen on 0.0.0.0, continuing because listening on [::]')
else:
raise e

148
synapse/app/_base.py
View file

@ -15,18 +15,38 @@
import gc
import logging
import signal
import sys
import traceback
import psutil
from daemonize import Daemonize
from twisted.internet import error, reactor
from twisted.protocols.tls import TLSMemoryBIOFactory
import synapse
from synapse.app import check_bind_error
from synapse.crypto import context_factory
from synapse.util import PreserveLoggingContext
from synapse.util.rlimit import change_resource_limit
from synapse.util.versionstring import get_version_string
logger = logging.getLogger(__name__)
_sighup_callbacks = []
def register_sighup(func):
"""
Register a function to be called when a SIGHUP occurs.
Args:
func (function): Function to be called when sent a SIGHUP signal.
Will be called with a single argument, the homeserver.
"""
_sighup_callbacks.append(func)
def start_worker_reactor(appname, config):
""" Run the reactor in the main process
@ -135,35 +155,47 @@ def listen_metrics(bind_addresses, port):
from prometheus_client import start_http_server
for host in bind_addresses:
reactor.callInThread(start_http_server, int(port),
addr=host, registry=RegistryProxy)
logger.info("Metrics now reporting on %s:%d", host, port)
logger.info("Starting metrics listener on %s:%d", host, port)
start_http_server(port, addr=host, registry=RegistryProxy)
def listen_tcp(bind_addresses, port, factory, reactor=reactor, backlog=50):
"""
Create a TCP socket for a port and several addresses
Returns:
list[twisted.internet.tcp.Port]: listening for TCP connections
"""
r = []
for address in bind_addresses:
try:
r.append(
reactor.listenTCP(
port,
factory,
backlog,
address
)
)
except error.CannotListenError as e:
check_bind_error(e, address, bind_addresses)
return r
def listen_ssl(
bind_addresses, port, factory, context_factory, reactor=reactor, backlog=50
):
"""
Create an SSL socket for a port and several addresses
Create an TLS-over-TCP socket for a port and several addresses
Returns:
list of twisted.internet.tcp.Port listening for TLS connections
"""
r = []
for address in bind_addresses:
try:
r.append(
reactor.listenSSL(
port,
factory,
@ -171,26 +203,106 @@ def listen_ssl(
backlog,
address
)
)
except error.CannotListenError as e:
check_bind_error(e, address, bind_addresses)
return r
def check_bind_error(e, address, bind_addresses):
def refresh_certificate(hs):
"""
Refresh the TLS certificates that Synapse is using by re-reading them from
disk and updating the TLS context factories to use them.
"""
This method checks an exception occurred while binding on 0.0.0.0.
If :: is specified in the bind addresses a warning is shown.
The exception is still raised otherwise.
Binding on both 0.0.0.0 and :: causes an exception on Linux and macOS
because :: binds on both IPv4 and IPv6 (as per RFC 3493).
When binding on 0.0.0.0 after :: this can safely be ignored.
if not hs.config.has_tls_listener():
# attempt to reload the certs for the good of the tls_fingerprints
hs.config.read_certificate_from_disk(require_cert_and_key=False)
return
hs.config.read_certificate_from_disk(require_cert_and_key=True)
hs.tls_server_context_factory = context_factory.ServerContextFactory(hs.config)
if hs._listening_services:
logger.info("Updating context factories...")
for i in hs._listening_services:
# When you listenSSL, it doesn't make an SSL port but a TCP one with
# a TLS wrapping factory around the factory you actually want to get
# requests. This factory attribute is public but missing from
# Twisted's documentation.
if isinstance(i.factory, TLSMemoryBIOFactory):
addr = i.getHost()
logger.info(
"Replacing TLS context factory on [%s]:%i", addr.host, addr.port,
)
# We want to replace TLS factories with a new one, with the new
# TLS configuration. We do this by reaching in and pulling out
# the wrappedFactory, and then re-wrapping it.
i.factory = TLSMemoryBIOFactory(
hs.tls_server_context_factory,
False,
i.factory.wrappedFactory
)
logger.info("Context factories updated.")
def start(hs, listeners=None):
"""
Start a Synapse server or worker.
Args:
e (Exception): Exception that was caught.
address (str): Address on which binding was attempted.
bind_addresses (list): Addresses on which the service listens.
hs (synapse.server.HomeServer)
listeners (list[dict]): Listener configuration ('listeners' in homeserver.yaml)
"""
if address == '0.0.0.0' and '::' in bind_addresses:
logger.warn('Failed to listen on 0.0.0.0, continuing because listening on [::]')
else:
raise e
try:
# Set up the SIGHUP machinery.
if hasattr(signal, "SIGHUP"):
def handle_sighup(*args, **kwargs):
for i in _sighup_callbacks:
i(hs)
signal.signal(signal.SIGHUP, handle_sighup)
register_sighup(refresh_certificate)
# Load the certificate from disk.
refresh_certificate(hs)
# It is now safe to start your Synapse.
hs.start_listening(listeners)
hs.get_datastore().start_profiling()
setup_sentry(hs)
except Exception:
traceback.print_exc(file=sys.stderr)
reactor = hs.get_reactor()
if reactor.running:
reactor.stop()
sys.exit(1)
def setup_sentry(hs):
"""Enable sentry integration, if enabled in configuration
Args:
hs (synapse.server.HomeServer)
"""
if not hs.config.sentry_enabled:
return
import sentry_sdk
sentry_sdk.init(
dsn=hs.config.sentry_dsn,
release=get_version_string(synapse),
)
# We set some default tags that give some context to this instance
with sentry_sdk.configure_scope() as scope:
scope.set_tag("matrix_server_name", hs.config.server_name)
app = hs.config.worker_app if hs.config.worker_app else "synapse.app.homeserver"
name = hs.config.worker_name if hs.config.worker_name else "master"
scope.set_tag("worker_app", app)
scope.set_tag("worker_name", name)

7
synapse/app/appservice.py
View file

@ -168,12 +168,7 @@ def start(config_options):
)
ps.setup()
ps.start_listening(config.worker_listeners)
def start():
ps.get_datastore().start_profiling()
reactor.callWhenRunning(start)
reactor.callWhenRunning(_base.start, ps, config.worker_listeners)
_base.start_worker_reactor("synapse-appservice", config)

17
synapse/app/client_reader.py
View file

@ -25,7 +25,6 @@ from synapse.app import _base
from synapse.config._base import ConfigError
from synapse.config.homeserver import HomeServerConfig
from synapse.config.logger import setup_logging
from synapse.crypto import context_factory
from synapse.http.server import JsonResource
from synapse.http.site import SynapseSite
from synapse.metrics import RegistryProxy
@ -41,6 +40,7 @@ from synapse.replication.slave.storage.registration import SlavedRegistrationSto
from synapse.replication.slave.storage.room import RoomStore
from synapse.replication.slave.storage.transactions import SlavedTransactionStore
from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.rest.client.v1.login import LoginRestServlet
from synapse.rest.client.v1.room import (
JoinedRoomMemberListRestServlet,
PublicRoomListRestServlet,
@ -48,6 +48,7 @@ from synapse.rest.client.v1.room import (
RoomMemberListRestServlet,
RoomStateRestServlet,
)
from synapse.rest.client.v2_alpha.register import RegisterRestServlet
from synapse.server import HomeServer
from synapse.storage.engines import create_engine
from synapse.util.httpresourcetree import create_resource_tree
@ -93,6 +94,8 @@ class ClientReaderServer(HomeServer):
JoinedRoomMemberListRestServlet(self).register(resource)
RoomStateRestServlet(self).register(resource)
RoomEventContextServlet(self).register(resource)
RegisterRestServlet(self).register(resource)
LoginRestServlet(self).register(resource)
resources.update({
"/_matrix/client/r0": resource,
@ -164,26 +167,16 @@ def start(config_options):
database_engine = create_engine(config.database_config)
tls_server_context_factory = context_factory.ServerContextFactory(config)
tls_client_options_factory = context_factory.ClientTLSOptionsFactory(config)
ss = ClientReaderServer(
config.server_name,
db_config=config.database_config,
tls_server_context_factory=tls_server_context_factory,
tls_client_options_factory=tls_client_options_factory,
config=config,
version_string="Synapse/" + get_version_string(synapse),
database_engine=database_engine,
)
ss.setup()
ss.start_listening(config.worker_listeners)
def start():
ss.get_datastore().start_profiling()
reactor.callWhenRunning(start)
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
_base.start_worker_reactor("synapse-client-reader", config)

Some files were not shown because too many files have changed in this diff Show more