Eric Eastwood
8461faf384
Add historical case to background update
2024-08-15 21:56:12 -05:00
Eric Eastwood
6c2fc1d20f
Move background updates to StateBackgroundUpdateStore
...
So we can access `_get_state_groups_from_groups_txn(...)`
2024-08-15 20:51:43 -05:00
Eric Eastwood
cbeff57402
Use helper
2024-08-15 20:31:57 -05:00
Eric Eastwood
4b42e44ef9
Work on background update for sliding_sync_membership_snapshots
2024-08-15 00:21:35 -05:00
Eric Eastwood
d113e743ae
Fix lints
2024-08-14 19:30:52 -05:00
Eric Eastwood
23e0d34a2d
Add more tests
2024-08-14 19:30:22 -05:00
Eric Eastwood
1c931cb3e7
Add background update for sliding_sync_joined_rooms
2024-08-14 19:19:15 -05:00
Eric Eastwood
9f551f0e97
Fix lints
2024-08-14 11:32:33 -05:00
Eric Eastwood
c8508f113a
Clean up tables when a room is purged/deleted
2024-08-14 11:27:57 -05:00
Eric Eastwood
f49003c35c
No invites needed
2024-08-13 18:55:59 -05:00
Eric Eastwood
8b0e1692f9
More realistic remote room forgotten test
2024-08-13 18:51:11 -05:00
Eric Eastwood
5df94f47b5
Fix running into StopIteration
...
More context about how/why `StopIteration` was being ignored silently
which made this problem harder to debug. See
https://github.com/element-hq/synapse/pull/17512#discussion_r1715954505
2024-08-13 17:15:09 -05:00
Eric Eastwood
3566abd9bc
Fix boolean schema for Postgres
2024-08-13 15:14:48 -05:00
Eric Eastwood
96a4614f92
Update fixme comment
2024-08-13 14:50:11 -05:00
Eric Eastwood
dc447a673f
Clarify when/why we upsert
2024-08-13 14:47:17 -05:00
Eric Eastwood
32ae162278
Fix rejecting invite when no_longer_in_room (and other non-join transitions)
2024-08-13 14:35:24 -05:00
Eric Eastwood
a90f3d4ae2
Merge branch 'develop' into madlittlemods/sliding-sync-pre-populate-room-meta-data
2024-08-13 12:28:36 -05:00
Eric Eastwood
a308d99f30
Sliding Sync: Exclude partially stated rooms if we must await full state ( #17538 )
...
Build release artifacts / Build sdist (push) Waiting to run
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / lint (push) Blocked by required conditions
Tests / changes (push) Waiting to run
Tests / check-sampleconfig (push) Blocked by required conditions
Tests / check-schema-delta (push) Blocked by required conditions
Tests / check-lockfile (push) Waiting to run
Tests / Typechecking (push) Blocked by required conditions
Tests / lint-crlf (push) Waiting to run
Tests / lint-newsfile (push) Waiting to run
Tests / lint-pydantic (push) Blocked by required conditions
Tests / lint-clippy (push) Blocked by required conditions
Tests / lint-clippy-nightly (push) Blocked by required conditions
Tests / lint-rustfmt (push) Blocked by required conditions
Tests / lint-readme (push) Blocked by required conditions
Tests / linting-done (push) Blocked by required conditions
Tests / calculate-test-jobs (push) Blocked by required conditions
Tests / trial (push) Blocked by required conditions
Tests / trial-pypy (all, pypy-3.8) (push) Blocked by required conditions
Tests / sytest (push) Blocked by required conditions
Tests / export-data (push) Blocked by required conditions
Tests / trial-olddeps (push) Blocked by required conditions
Tests / portdb (11, 3.8) (push) Blocked by required conditions
Tests / portdb (15, 3.11) (push) Blocked by required conditions
Tests / complement (monolith, Postgres) (push) Blocked by required conditions
Tests / complement (monolith, SQLite) (push) Blocked by required conditions
Tests / complement (workers, Postgres) (push) Blocked by required conditions
Tests / cargo-test (push) Blocked by required conditions
Tests / cargo-bench (push) Blocked by required conditions
Tests / tests-done (push) Blocked by required conditions
Previously, we just had very basic partial room exclusion based on
whether we were lazy-loading room members. Now with this PR, we added
`must_await_full_state(...)` with rules to check if we have a we're only
requesting `required_state` which is completely satisfied even with
partial state.
Partially-stated rooms should have all state events except for remote
membership events so if we require a remote membership event anywhere,
then we need to return `True`.
2024-08-13 12:27:42 -05:00
Eric Eastwood
eb3a185cfc
Fix federating backfill test
2024-08-13 12:24:53 -05:00
Erik Johnston
a9fc1fd112
Use a larger, dedicated threadpool for media sending ( #17564 )
2024-08-13 17:59:47 +01:00
Andrew Morgan
6a11bdf01d
Add a utility function for generating fake event IDs ( #17557 )
2024-08-13 16:55:05 +00:00
Patrick Cloke
8fea190a1f
Add missing docstrings related to profile methods. ( #17559 )
2024-08-13 17:04:35 +01:00
Erik Johnston
81c19c4cd2
Merge branch 'master' into develop
2024-08-13 15:58:11 +01:00
Erik Johnston
aaa3c36420
Remove logging in multipart ( #17563 )
...
This is really spurious and causes a lot of spam. I don't think there is
a use for it even at DEBUG level.
2024-08-13 15:56:18 +01:00
Erik Johnston
3e7eb45eb1
Fixup media logcontexts ( #17561 )
...
Regression from #17558
2024-08-13 15:00:57 +01:00
Erik Johnston
bab37dfc6f
1.113.0
Tests / changes (push) Has been cancelled
Tests / check-lockfile (push) Has been cancelled
Tests / lint-crlf (push) Has been cancelled
Tests / cargo-bench (push) Has been cancelled
Deploy the documentation / GitHub Pages (push) Has been cancelled
Build release artifacts / Build .deb packages (push) Has been cancelled
Build release artifacts / Attach assets to release (push) Has been cancelled
Tests / check-sampleconfig (push) Has been cancelled
Tests / check-schema-delta (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / Typechecking (push) Has been cancelled
Tests / lint-pydantic (push) Has been cancelled
Tests / lint-clippy (push) Has been cancelled
Tests / lint-clippy-nightly (push) Has been cancelled
Tests / lint-rustfmt (push) Has been cancelled
Tests / lint-readme (push) Has been cancelled
Tests / linting-done (push) Has been cancelled
Tests / calculate-test-jobs (push) Has been cancelled
Tests / trial (push) Has been cancelled
Tests / trial-olddeps (push) Has been cancelled
Tests / trial-pypy (all, pypy-3.8) (push) Has been cancelled
Tests / sytest (push) Has been cancelled
Tests / export-data (push) Has been cancelled
Tests / portdb (11, 3.8) (push) Has been cancelled
Tests / portdb (15, 3.11) (push) Has been cancelled
Tests / complement (monolith, Postgres) (push) Has been cancelled
Tests / complement (monolith, SQLite) (push) Has been cancelled
Tests / complement (workers, Postgres) (push) Has been cancelled
Tests / cargo-test (push) Has been cancelled
Tests / tests-done (push) Has been cancelled
2024-08-13 14:37:01 +01:00
Erik Johnston
9f9ec92526
Speed up responding to media requests ( #17558 )
...
Build release artifacts / Build sdist (push) Waiting to run
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / lint-crlf (push) Waiting to run
Tests / Typechecking (push) Blocked by required conditions
Tests / check-schema-delta (push) Blocked by required conditions
Tests / lint-newsfile (push) Waiting to run
Tests / lint-pydantic (push) Blocked by required conditions
Tests / lint-clippy (push) Blocked by required conditions
Tests / changes (push) Waiting to run
Tests / check-sampleconfig (push) Blocked by required conditions
Tests / check-lockfile (push) Waiting to run
Tests / lint (push) Blocked by required conditions
Tests / lint-clippy-nightly (push) Blocked by required conditions
Tests / lint-rustfmt (push) Blocked by required conditions
Tests / lint-readme (push) Blocked by required conditions
Tests / linting-done (push) Blocked by required conditions
Tests / calculate-test-jobs (push) Blocked by required conditions
Tests / trial (push) Blocked by required conditions
Tests / trial-olddeps (push) Blocked by required conditions
Tests / trial-pypy (all, pypy-3.8) (push) Blocked by required conditions
Tests / sytest (push) Blocked by required conditions
Tests / export-data (push) Blocked by required conditions
Tests / portdb (11, 3.8) (push) Blocked by required conditions
Tests / portdb (15, 3.11) (push) Blocked by required conditions
Tests / complement (monolith, Postgres) (push) Blocked by required conditions
Tests / complement (monolith, SQLite) (push) Blocked by required conditions
Tests / complement (workers, Postgres) (push) Blocked by required conditions
Tests / cargo-test (push) Blocked by required conditions
Tests / cargo-bench (push) Blocked by required conditions
Tests / tests-done (push) Blocked by required conditions
We do this by reading from a threadpool, rather than blocking the main
thread.
This is broadly what we do in the [S3 storage
provider](https://github.com/matrix-org/synapse-s3-storage-provider/blob/main/s3_storage_provider.py#L234 )
2024-08-13 14:06:17 +01:00
dependabot[bot]
ff7b27013e
Bump serde from 1.0.204 to 1.0.206 ( #17556 )
2024-08-13 08:59:35 +01:00
Eric Eastwood
517946d940
Fix lints
2024-08-12 20:31:25 -05:00
Eric Eastwood
f600eacd0d
Adjust test description
2024-08-12 20:30:48 -05:00
Eric Eastwood
3423eb72d5
Add test to make sure snapshot evolves with membership
2024-08-12 20:29:58 -05:00
Eric Eastwood
5589ae48ca
Add test for remote invite rejected/retracted
2024-08-12 20:14:14 -05:00
Eric Eastwood
83a5858083
Add tests for remote invites
2024-08-12 19:57:28 -05:00
Eric Eastwood
3e1f24ea11
User ID is not unique because user is joined to many rooms
2024-08-12 19:46:23 -05:00
Eric Eastwood
ab074f5335
Fix events from rooms we're not joined to affecting the joined room stream ordering
2024-08-12 19:40:53 -05:00
Eric Eastwood
53232e6df5
Fill in for remote invites (out of band, outlier membership)
2024-08-12 18:14:02 -05:00
Eric Eastwood
f069659343
Fix lints
2024-08-12 15:49:40 -05:00
Eric Eastwood
552f8f496d
Update descriptions
2024-08-12 15:43:06 -05:00
Eric Eastwood
0af3b4822c
Refactor to sliding_sync_membership_snapshots
2024-08-12 15:10:44 -05:00
Eric Eastwood
ed47a7eff5
Fix bumping when events are persisted out of order
2024-08-12 11:27:17 -05:00
Andrew Morgan
e1f5f0fbb8
Bump setuptools from 67.6.0 to 72.1.0 ( #17542 )
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / lint-clippy-nightly (push) Blocked by required conditions
Tests / lint-rustfmt (push) Blocked by required conditions
Tests / lint-readme (push) Blocked by required conditions
Tests / changes (push) Waiting to run
Tests / check-sampleconfig (push) Blocked by required conditions
Tests / check-schema-delta (push) Blocked by required conditions
Tests / check-lockfile (push) Waiting to run
Tests / lint (push) Blocked by required conditions
Tests / Typechecking (push) Blocked by required conditions
Tests / lint-crlf (push) Waiting to run
Tests / portdb (11, 3.8) (push) Blocked by required conditions
Tests / portdb (15, 3.11) (push) Blocked by required conditions
Tests / complement (monolith, Postgres) (push) Blocked by required conditions
Tests / lint-newsfile (push) Waiting to run
Tests / lint-pydantic (push) Blocked by required conditions
Tests / lint-clippy (push) Blocked by required conditions
Tests / linting-done (push) Blocked by required conditions
Tests / calculate-test-jobs (push) Blocked by required conditions
Tests / trial (push) Blocked by required conditions
Tests / trial-olddeps (push) Blocked by required conditions
Tests / trial-pypy (all, pypy-3.8) (push) Blocked by required conditions
Tests / sytest (push) Blocked by required conditions
Tests / export-data (push) Blocked by required conditions
Tests / complement (monolith, SQLite) (push) Blocked by required conditions
Tests / complement (workers, Postgres) (push) Blocked by required conditions
Tests / cargo-test (push) Blocked by required conditions
Tests / cargo-bench (push) Blocked by required conditions
Tests / tests-done (push) Blocked by required conditions
/ Check locked dependencies have sdists (push) Has been cancelled
2024-08-12 14:58:01 +01:00
dependabot[bot]
8c9f2743bc
Bump serde_json from 1.0.122 to 1.0.124 ( #17555 )
2024-08-12 14:33:12 +01:00
dependabot[bot]
b076941a36
Bump sentry-sdk from 2.10.0 to 2.12.0 ( #17553 )
2024-08-12 14:32:18 +01:00
dependabot[bot]
8bbe65f3c0
Bump types-pyyaml from 6.0.12.20240311 to 6.0.12.20240808 ( #17552 )
2024-08-12 14:32:05 +01:00
dependabot[bot]
b7faf01f26
Bump phonenumbers from 8.13.42 to 8.13.43 ( #17551 )
2024-08-12 14:31:56 +01:00
dependabot[bot]
4f7f6ee9a0
Bump lxml from 5.2.2 to 5.3.0 ( #17550 )
2024-08-12 14:31:45 +01:00
dependabot[bot]
a640b318df
Bump sigstore/cosign-installer from 3.5.0 to 3.6.0 ( #17549 )
2024-08-12 14:31:34 +01:00
dependabot[bot]
34b7586446
Bump types-requests from 2.31.0.20240406 to 2.32.0.20240712 ( #17524 )
...
Build release artifacts / Build sdist (push) Waiting to run
Build release artifacts / Attach assets to release (push) Blocked by required conditions
Tests / changes (push) Has been cancelled
Tests / check-sampleconfig (push) Has been cancelled
Tests / check-schema-delta (push) Has been cancelled
Tests / check-lockfile (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / Typechecking (push) Has been cancelled
Tests / lint-crlf (push) Has been cancelled
Tests / lint-newsfile (push) Has been cancelled
Tests / lint-pydantic (push) Has been cancelled
Tests / lint-clippy (push) Has been cancelled
Tests / lint-clippy-nightly (push) Has been cancelled
Tests / lint-rustfmt (push) Has been cancelled
Tests / lint-readme (push) Has been cancelled
Tests / linting-done (push) Has been cancelled
Tests / calculate-test-jobs (push) Has been cancelled
Tests / trial (push) Has been cancelled
Tests / trial-olddeps (push) Has been cancelled
Tests / trial-pypy (all, pypy-3.8) (push) Has been cancelled
Tests / sytest (push) Has been cancelled
Tests / export-data (push) Has been cancelled
Tests / portdb (11, 3.8) (push) Has been cancelled
Tests / portdb (15, 3.11) (push) Has been cancelled
Tests / complement (monolith, Postgres) (push) Has been cancelled
Tests / complement (monolith, SQLite) (push) Has been cancelled
Tests / complement (workers, Postgres) (push) Has been cancelled
Tests / cargo-test (push) Has been cancelled
Tests / cargo-bench (push) Has been cancelled
Tests / tests-done (push) Has been cancelled
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-12 10:20:38 +01:00
Erik Johnston
70b0e38603
Fix performance of device lists in /key/changes and sliding sync ( #17537 )
...
Tests / changes (push) Has been cancelled
Tests / lint-crlf (push) Has been cancelled
Tests / lint-newsfile (push) Has been cancelled
Tests / check-schema-delta (push) Has been cancelled
Deploy the documentation / GitHub Pages (push) Has been cancelled
Build release artifacts / Build .deb packages (push) Has been cancelled
Build release artifacts / Attach assets to release (push) Has been cancelled
Tests / check-sampleconfig (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / complement (monolith, Postgres) (push) Has been cancelled
Tests / Typechecking (push) Has been cancelled
Tests / lint-pydantic (push) Has been cancelled
Tests / lint-clippy (push) Has been cancelled
Tests / lint-clippy-nightly (push) Has been cancelled
Tests / lint-rustfmt (push) Has been cancelled
Tests / lint-readme (push) Has been cancelled
Tests / linting-done (push) Has been cancelled
Tests / calculate-test-jobs (push) Has been cancelled
Tests / trial (push) Has been cancelled
Tests / trial-olddeps (push) Has been cancelled
Tests / trial-pypy (all, pypy-3.8) (push) Has been cancelled
Tests / sytest (push) Has been cancelled
Tests / export-data (push) Has been cancelled
Tests / portdb (11, 3.8) (push) Has been cancelled
Tests / portdb (15, 3.11) (push) Has been cancelled
Tests / complement (monolith, SQLite) (push) Has been cancelled
Tests / cargo-bench (push) Has been cancelled
Tests / tests-done (push) Has been cancelled
Tests / complement (workers, Postgres) (push) Has been cancelled
Tests / cargo-test (push) Has been cancelled
We do this by reusing the code from sync v2.
Reviewable commit-by-commit. The function `get_user_ids_changed` has
been rewritten entirely, so I would recommend not looking at the diff.
2024-08-09 11:59:44 +01:00
Eric Eastwood
3367422fd3
Need to fix upsert
2024-08-08 18:23:50 -05:00
