Eric Eastwood
693c06b2f1
Move away from backfill language
2024-08-22 16:48:02 -05:00
Eric Eastwood
4d87fa61c6
"backfill" -> "bg_update"
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1726837698
2024-08-22 16:44:02 -05:00
Eric Eastwood
d61aada8ba
Simplify _update_sliding_sync_tables_with_new_persisted_events_txn()
...
See
https://github.com/element-hq/synapse/pull/17512#discussion_r1719997640
https://github.com/element-hq/synapse/pull/17512#discussion_r1726828894
https://github.com/element-hq/synapse/pull/17512#discussion_r1726836440
2024-08-22 16:35:59 -05:00
Eric Eastwood
6723824c4a
Prefer simple_delete_many_txn
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1726819380
2024-08-22 15:47:44 -05:00
Eric Eastwood
980ee9aad6
Prefer simple_update_txn
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1726808112
2024-08-22 15:40:08 -05:00
Eric Eastwood
fc73b6ffc9
Rename insert_key/insert_value
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1726805894
2024-08-22 11:44:57 -05:00
Eric Eastwood
9b8d2017af
Check events_and_context for state events
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1726798803
2024-08-22 11:34:40 -05:00
Eric Eastwood
339500d067
Fix sub-query selecting multiple rows
2024-08-21 22:56:25 -05:00
Eric Eastwood
31300f4ce5
More docstring
2024-08-21 22:15:19 -05:00
Eric Eastwood
b45b1896aa
Fill out docstring
2024-08-21 22:09:57 -05:00
Eric Eastwood
ee2ef0b4d9
Add forgotten column
2024-08-21 21:54:22 -05:00
Eric Eastwood
0a938b137a
Add missing boolean column to portdb script
2024-08-21 19:59:35 -05:00
Eric Eastwood
97248362d0
Log which room is strange
2024-08-21 19:26:14 -05:00
Eric Eastwood
02711552cf
Better handle none case
2024-08-21 19:11:08 -05:00
Eric Eastwood
8ddf5c7235
Add tombstone to tests
2024-08-21 19:05:59 -05:00
Eric Eastwood
513ec8e906
Update tests
2024-08-21 18:51:04 -05:00
Eric Eastwood
cda2311520
Add tombstone_successor_room_id column
2024-08-21 18:21:44 -05:00
Eric Eastwood
c612572d12
Move away from stream_id
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1725806543
2024-08-21 17:14:34 -05:00
Eric Eastwood
5b1db39bb7
Add sender column so we can tell leaves from kicks
2024-08-21 16:42:12 -05:00
Eric Eastwood
e7a3328228
Pre-populate membership and membership_event_stream_ordering
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1725311745
2024-08-21 16:20:00 -05:00
Eric Eastwood
f6d7ffd9c5
Move _calculate_sliding_sync_table_changes(...) after we assign stream_ordering to events
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1725728637
2024-08-21 16:10:14 -05:00
Eric Eastwood
772c501bb6
Use available stream_id
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1725310035
2024-08-21 15:04:51 -05:00
Eric Eastwood
cda92af4a6
No need to update event_stream_ordering/bump_stamp ON CONFLICT
2024-08-21 14:46:50 -05:00
Eric Eastwood
d3f90e4bd8
Get full events for _sliding_sync_joined_rooms_backfill
2024-08-21 14:39:54 -05:00
Eric Eastwood
a5e06c6a8d
Move back to the main store
2024-08-21 11:14:15 -05:00
Eric Eastwood
0233e20aa3
Use full event version after solving the circular import issues
2024-08-21 10:44:49 -05:00
Eric Eastwood
357132db1d
Go back to simpler fetching senders
2024-08-20 21:54:03 -05:00
Eric Eastwood
726a8e9698
Attempt getting real events in backgroun update (needs work)
2024-08-20 21:48:44 -05:00
Eric Eastwood
cc200ee9f5
Merge branch 'develop' into madlittlemods/sliding-sync-pre-populate-room-meta-data
2024-08-20 17:25:48 -05:00
Eric Eastwood
3eb77c3a2a
Add sanity checks and fix wrong variable usage
2024-08-20 17:24:43 -05:00
Eric Eastwood
45c89ec625
Move pre-processing completely outside transaction
2024-08-20 15:41:53 -05:00
Erik Johnston
92b38c1afd
Sliding sync: Split up handler into its own module ( #17595 )
...
That file was getting long.
The changes are non functional, and simply split things up into:
- the main class
- the connection store
- the extensions
- the types
2024-08-20 18:30:23 +00:00
Eric Eastwood
ac5b05c86b
Use TypedDict
2024-08-20 13:29:56 -05:00
Eric Eastwood
2964c567d3
Use dicts
2024-08-20 13:21:19 -05:00
Eric Eastwood
95d39db772
Closer types
2024-08-20 11:55:24 -05:00
Eric Eastwood
6cc6bdbedf
Start of moving logic outside of the transaction (pre-process)
2024-08-20 11:10:34 -05:00
Quentin Gliech
a8e313836d
changelog: move SSSS some changes in the features section
2024-08-20 15:18:13 +02:00
Quentin Gliech
7c9684b5dc
1.114.0rc1
2024-08-20 14:57:22 +02:00
Erik Johnston
f1e8d2d15a
Sliding Sync: Speed up getting receipts for initial rooms ( #17592 )
...
Let's only pull out the events we care about. Note that the index isn't
necessary here, as postgres is happy to scan the set of rooms for the
events.
2024-08-20 12:57:34 +01:00
Erik Johnston
10428046e4
Add metrics for sliding sync processing time ( #17593 )
...
This should let us see how quickly we actually process things in
practice.
2024-08-20 11:36:49 +00:00
Erik Johnston
6eb98a4f1c
Sliding Sync: Handle timeline limit changes (take 2) ( #17579 )
...
This supersedes #17503 , given the per-connection state is being heavily
rewritten it felt easier to recreate the PR on top of that work.
This correctly handles the case of timeline limits going up and down.
This does not handle changes in `required_state`, but that can be done
as a separate PR.
Based on #17575 .
---------
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-08-20 10:31:25 +01:00
Erik Johnston
950ba844f7
Sliding Sync: Batch up fetching receipts ( #17589 )
...
This is to make initial sliding sync a bit faster
2024-08-20 10:13:26 +01:00
Eric Eastwood
574a04a40f
Test state reset on membership
2024-08-19 23:30:25 -05:00
Eric Eastwood
8ee2e114dd
Add test to handle state reset in the meta data
2024-08-19 23:22:24 -05:00
Eric Eastwood
98fb56e5fe
Prefer _update_sliding_sync_tables_with_new_persisted_events_txn(...) to do the right thing
...
See https://github.com/element-hq/synapse/pull/17512#discussion_r1719992152
2024-08-19 22:30:50 -05:00
Eric Eastwood
df0c57d383
Merge branch 'develop' into madlittlemods/sliding-sync-pre-populate-room-meta-data
2024-08-19 16:34:41 -05:00
Erik Johnston
8b8d74d12f
Sliding sync: Correctly track which read receipts we have or have not sent down. ( #17575 )
...
Add connection tracking to the receipts extension.
Based on #17574
---------
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-08-19 21:16:07 +01:00
Erik Johnston
261e746281
Sliding sync: Add classes for per-connection state ( #17574 )
...
This is some prep work ahead of correctly tracking receipts, where we
will also want to track the room status in terms of last receipt we had
sent down.
Essentially, we add two classes `PerConnectionState` and a mutable
version, and then operate on those.
---------
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-08-19 20:09:41 +01:00
Erik Johnston
993644ded0
Fix zero length media handling ( #17570 )
...
Results in:
```
AssertionError: null
File "synapse/http/server.py", line 332, in _async_render_wrapper
callback_return = await self._async_render(request)
File "synapse/http/server.py", line 544, in _async_render
callback_return = await raw_callback_return
File "synapse/federation/transport/server/_base.py", line 369, in new_func
response = await func(
File "synapse/federation/transport/server/federation.py", line 826, in on_GET
await self.media_repo.get_local_media(
File "synapse/media/media_repository.py", line 473, in get_local_media
await respond_with_multipart_responder(
File "synapse/media/_base.py", line 353, in respond_with_multipart_responder
assert content_length is not None
```
2024-08-19 15:06:44 +01:00
Erik Johnston
a5d25bb623
Test github token before running release script ( #17562 )
...
This stops people from getting half way through a step and it failing
due to the github token having expired (this happens to me every damn
time).
2024-08-19 14:15:36 +01:00
