Improve advice regarding poor performance

Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
This commit is contained in:
Benjamin Saunders 2018-12-08 02:33:42 -08:00
parent c831748f4d
commit 047486a384
3 changed files with 29 additions and 7 deletions

View file

@ -72,3 +72,6 @@ Jason Robinson <jasonr at matrix.org>
Joseph Weston <joseph at weston.cloud> Joseph Weston <joseph at weston.cloud>
+ Add admin API for querying HS version + Add admin API for querying HS version
Benjamin Saunders <ben.e.saunders at gmail dot com>
* Documentation improvements

View file

@ -340,8 +340,11 @@ log lines and looking for any 'Processed request' lines which take more than
a few seconds to execute. Please let us know at #synapse:matrix.org if a few seconds to execute. Please let us know at #synapse:matrix.org if
you see this failure mode so we can help debug it, however. you see this failure mode so we can help debug it, however.
Help!! Synapse eats all my RAM! Help!! Synapse is slow and eats all my RAM/CPU!
------------------------------- ===============================================
First, ensure you are running the latest version of Synapse, using Python 3
with a PostgreSQL database.
Synapse's architecture is quite RAM hungry currently - we deliberately Synapse's architecture is quite RAM hungry currently - we deliberately
cache a lot of recent room data and metadata in RAM in order to speed up cache a lot of recent room data and metadata in RAM in order to speed up
@ -352,14 +355,29 @@ variable. The default is 0.5, which can be decreased to reduce RAM usage
in memory constrained enviroments, or increased if performance starts to in memory constrained enviroments, or increased if performance starts to
degrade. degrade.
However, degraded performance due to a low cache factor, common on
machines with slow disks, often leads to explosions in memory use due
backlogged requests. In this case, reducing the cache factor will make
things worse. Instead, try increasing it drastically. 2.0 is a good
starting value.
Using `libjemalloc <http://jemalloc.net/>`_ can also yield a significant Using `libjemalloc <http://jemalloc.net/>`_ can also yield a significant
improvement in overall amount, and especially in terms of giving back RAM improvement in overall memory use, and especially in terms of giving back
to the OS. To use it, the library must simply be put in the LD_PRELOAD RAM to the OS. To use it, the library must simply be put in the
environment variable when launching Synapse. On Debian, this can be done LD_PRELOAD environment variable when launching Synapse. On Debian, this
by installing the ``libjemalloc1`` package and adding this line to can be done by installing the ``libjemalloc1`` package and adding this
``/etc/default/matrix-synapse``:: line to ``/etc/default/matrix-synapse``::
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
This can make a significant difference on Python 2.7 - it's unclear how This can make a significant difference on Python 2.7 - it's unclear how
much of an improvement it provides on Python 3.x. much of an improvement it provides on Python 3.x.
If you're encountering high CPU use by the Synapse process itself, you
may be affected by a bug with presence tracking that leads to a
massive excess of outgoing federation requests (see `discussion
<https://github.com/matrix-org/synapse/issues/3971>`_). If metrics
indicate that your server is also issuing far more outgoing federation
requests than can be accounted for by your users' activity, this is a
likely cause. The misbehavior can be worked around by setting
``use_presence: false`` in the Synapse config file.

1
changelog.d/4276.misc Normal file
View file

@ -0,0 +1 @@
Improve README section on performance troubleshooting.