<buttonid="sidebar-toggle"class="icon-button"type="button"title="Toggle Table of Contents"aria-label="Toggle Table of Contents"aria-controls="sidebar">
<ahref="https://github.com/matrix-org/synapse/edit/develop/docs/development/dependencies.md"title="Suggest an edit"aria-label="Suggest an edit">
<iid="git-edit-button"class="fa fa-edit"></i>
</a>
</div>
</div>
<divid="search-wrapper"class="hidden">
<formid="searchbar-outer"class="searchbar-outer">
<inputtype="search"id="searchbar"name="searchbar"placeholder="Search this book ..."aria-controls="searchresults-outer"aria-describedby="searchresults-header">
<p>The locked dependencies get included in our "self-contained" releases: namely,
our docker images and our debian packages. We also use the locked dependencies
in development and our continuous integration.</p>
<p>Separately, our "broad" dependencies—the version ranges specified in
<code>pyproject.toml</code>—are included as metadata in our "sdists" and "wheels"<ahref="https://pypi.org/project/matrix-synapse">uploaded
to PyPI</a>. Installing from PyPI or from
the Synapse source tree directly will <em>not</em> use the locked dependencies; instead,
they'll pull in the latest version of each package available at install time.</p>
<li>Teach direnv about poetry. The <ahref="https://github.com/direnv/direnv/wiki/Python#poetry">shell config here</a>
needs to be added to <code>~/.config/direnv/direnvrc</code> (or more generally <code>$XDG_CONFIG_HOME/direnv/direnvrc</code>).</li>
<li>Mark the synapse checkout as a poetry project: <code>echo layout poetry > .envrc</code>.</li>
<li>Convince yourself that you trust this <code>.envrc</code> configuration and project.
Then formally confirm this to <code>direnv</code> by running <code>direnv allow</code>.</li>
</ol>
<p>Then whenever you navigate to the synapse checkout, you should be able to run
e.g. <code>mypy</code> instead of <code>poetry run mypy</code>; <code>python</code> instead of
<code>poetry run python</code>; and your shell commands will automatically run in the
context of poetry's venv, without having to run <code>poetry shell</code> beforehand.</p>
<h1id="how-do-i"><aclass="header"href="#how-do-i">How do I...</a></h1>
<h2id="reset-my-venv-to-the-locked-environment"><aclass="header"href="#reset-my-venv-to-the-locked-environment">...reset my venv to the locked environment?</a></h2>
<pre><codeclass="language-shell">poetry install --extras all --remove-untracked
</code></pre>
<h2id="run-a-command-in-the-poetry-virtualenv"><aclass="header"href="#run-a-command-in-the-poetry-virtualenv">...run a command in the <code>poetry</code> virtualenv?</a></h2>
<p>Use <code>poetry run cmd args</code> when you need the python virtualenv context.
To avoid typing <code>poetry run</code> all the time, you can run <code>poetry shell</code>
to start a new shell in the poetry virtualenv context. Within <code>poetry shell</code>,
<code>python</code>, <code>pip</code>, <code>mypy</code>, <code>trial</code>, etc. are all run inside the project virtualenv
and isolated from the rest o the system.</p>
<p>Roughly speaking, the translation from a traditional virtualenv is:</p>
<li><code>deactivate</code> -> close the terminal (Ctrl-D, <code>exit</code>, etc.)</li>
</ul>
<p>See also the direnv recommendation above, which makes <code>poetry run</code> and
<code>poetry shell</code> unnecessary.</p>
<h2id="inspect-the-poetry-virtualenv"><aclass="header"href="#inspect-the-poetry-virtualenv">...inspect the <code>poetry</code> virtualenv?</a></h2>
<p>Some suggestions:</p>
<pre><codeclass="language-shell"># Current env only
poetry env info
# All envs: this allows you to have e.g. a poetry managed venv for Python 3.7,
# and another for Python 3.10.
poetry env list --full-path
poetry run pip list
</code></pre>
<p>Note that <code>poetry show</code> describes the abstract <em>lock file</em> rather than your
on-disk environment. With that said, <code>poetry show --tree</code> can sometimes be
useful.</p>
<h2id="add-a-new-dependency"><aclass="header"href="#add-a-new-dependency">...add a new dependency?</a></h2>
<p>Either:</p>
<ul>
<li>manually update <code>pyproject.toml</code>; then <code>poetry lock --no-update</code>; or else</li>
<li><code>poetry add packagename</code>. See <code>poetry add --help</code>; note the <code>--dev</code>,
<code>--extras</code> and <code>--optional</code> flags in particular.
<ul>
<li><strong>NB</strong>: this specifies the new package with a version given by a "caret bound". This won't get forced to its lowest version in the old deps CI job: see <ahref="https://github.com/matrix-org/synapse/blob/4e1374373857f2f7a911a31c50476342d9070681/.ci/scripts/test_old_deps.sh#L35-L39">this TODO</a>.</li>
</ul>
</li>
</ul>
<p>Include the updated <code>pyproject.toml</code> and <code>poetry.lock</code> files in your commit.</p>
<h2id="remove-a-dependency"><aclass="header"href="#remove-a-dependency">...remove a dependency?</a></h2>
<p>This is not done often and is untested, but</p>
<p>ought to do the trick. Alternatively, manually update <code>pyproject.toml</code> and
<code>poetry lock --no-update</code>. Include the updated <code>pyproject.toml</code> and poetry.lock`
files in your commit.</p>
<h2id="update-the-version-range-for-an-existing-dependency"><aclass="header"href="#update-the-version-range-for-an-existing-dependency">...update the version range for an existing dependency?</a></h2>
<p>Best done by manually editing <code>pyproject.toml</code>, then <code>poetry lock --no-update</code>.
Include the updated <code>pyproject.toml</code> and <code>poetry.lock</code> in your commit.</p>
<h2id="update-a-dependency-in-the-locked-environment"><aclass="header"href="#update-a-dependency-in-the-locked-environment">...update a dependency in the locked environment?</a></h2>
<h2id="check-the-version-of-poetry-with-poetry---version"><aclass="header"href="#check-the-version-of-poetry-with-poetry---version">Check the version of poetry with <code>poetry --version</code>.</a></h2>
<p>At the time of writing, the 1.2 series is beta only. We have seen some examples
where the lockfiles generated by 1.2 prereleasese aren't interpreted correctly
by poetry 1.1.x. For now, use poetry 1.1.14, which includes a critical
<ahref="https://github.com/python-poetry/poetry/pull/5973">change</a> needed to remain
<ahref="https://github.com/pypi/warehouse/pull/11775">compatible with PyPI</a>.</p>
<p>It can also be useful to check the version of <code>poetry-core</code> in use. If you've
installed <code>poetry</code> with <code>pipx</code>, try <code>pipx runpip poetry list | grep poetry-core</code>.</p>
<p>Poetry caches a bunch of information about packages that isn't readily available
from PyPI. (This is what makes poetry seem slow when doing the first
<code>poetry install</code>.) Try <code>poetry cache list</code> and <code>poetry cache clear --all <name of cache></code> to see if that fixes things.</p>
<h2id="try---verbose-or---dry-run-arguments"><aclass="header"href="#try---verbose-or---dry-run-arguments">Try <code>--verbose</code> or <code>--dry-run</code> arguments.</a></h2>
<p>Sometimes useful to see what poetry's internal logic is.</p>
