synapse/tests/storage
Eric Eastwood df8b91ed2b
Limit and filter the number of backfill points to get from the database (#13879)
There is no need to grab thousands of backfill points when we only need 5 to make the `/backfill` request with. We need to grab a few extra in case the first few aren't visible in the history.

Previously, we grabbed thousands of backfill points from the database, then sorted and filtered them in the app. Fetching the 4.6k backfill points for `#matrix:matrix.org` from the database takes ~50ms - ~570ms so it's not like this saves a lot of time 🤷. But it might save us more time now that `get_backfill_points_in_room`/`get_insertion_event_backward_extremities_in_room` are more complicated after https://github.com/matrix-org/synapse/pull/13635 

This PR moves the filtering and limiting to the SQL query so we just have less data to work with in the first place.

Part of https://github.com/matrix-org/synapse/issues/13356
2022-09-28 15:26:16 -05:00
..
databases Fix have_seen_event cache not being invalidated (#13863) 2022-09-27 15:55:43 -05:00
util Wait for lazy join to complete when getting current state (#12872) 2022-06-01 16:02:53 +01:00
__init__.py Reference Matrix Home Server 2014-08-12 15:10:52 +01:00
test__base.py Optimise _update_client_ips_batch_txn to batch together database operations. (#12252) 2022-04-08 15:29:13 +01:00
test_account_data.py Use the ignored_users table to test event visibility & sync. (#12225) 2022-03-15 14:06:05 -04:00
test_appservice.py Remove code which updates application_services_state.last_txn (#12680) 2022-05-17 11:07:18 +01:00
test_background_update.py Add type hints to tests files. (#12256) 2022-03-21 09:43:16 -04:00
test_base.py Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
test_cleanup_extrems.py Default to private room visibility rather than public when a client does not specify one, according to spec. (#12350) 2022-04-01 15:55:09 +01:00
test_client_ips.py Remove HomeServer.get_datastore() (#12031) 2022-02-23 11:04:02 +00:00
test_database.py Handle cancellation in DatabasePool.runInteraction() (#12199) 2022-03-16 15:07:41 +00:00
test_devices.py Additional constants for EDU types. (#12884) 2022-05-27 07:14:36 -04:00
test_directory.py Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
test_e2e_room_keys.py Remove HomeServer.get_datastore() (#12031) 2022-02-23 11:04:02 +00:00
test_end_to_end_keys.py Remove HomeServer.get_datastore() (#12031) 2022-02-23 11:04:02 +00:00
test_event_chain.py Persist CreateRoom events to DB in a batch (#13800) 2022-09-28 10:11:48 +00:00
test_event_federation.py Limit and filter the number of backfill points to get from the database (#13879) 2022-09-28 15:26:16 -05:00
test_event_metrics.py Fix (final) Bugbear violations (#9838) 2021-04-20 11:50:49 +01:00
test_event_push_actions.py Improve tests for get_unread_push_actions_for_user_in_range_*. (#13893) 2022-09-26 18:28:12 +00:00
test_events.py Faster room joins: avoid blocking when pulling events with missing prevs (#13355) 2022-07-26 12:39:23 +01:00
test_id_generators.py Remove redundant get_success calls in test code (#12346) 2022-04-01 16:10:31 +01:00
test_keys.py Remove HomeServer.get_datastore() (#12031) 2022-02-23 11:04:02 +00:00
test_main.py Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
test_monthly_active_users.py Add storage and module API methods to get monthly active users and their appservices (#12838) 2022-05-27 10:25:57 +00:00
test_profile.py Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
test_purge.py Async get event cache prep (#13242) 2022-07-15 09:30:46 +00:00
test_receipts.py Accept & store thread IDs for receipts (implement MSC3771). (#13782) 2022-09-23 14:33:28 +00:00
test_redaction.py Rename storage classes (#12913) 2022-05-31 12:17:50 +00:00
test_registration.py Add timestamp to user's consent (#13741) 2022-09-08 15:41:48 +00:00
test_rollback_worker.py Remove HomeServer.get_datastore() (#12031) 2022-02-23 11:04:02 +00:00
test_room.py Remove obsolete RoomEventsStoreTestCase (#13200) 2022-07-07 13:47:26 +01:00
test_room_search.py Rename storage classes (#12913) 2022-05-31 12:17:50 +00:00
test_roommember.py Fix that user cannot /forget rooms after the last member has left (#13546) 2022-08-30 09:58:38 +00:00
test_state.py Make DictionaryCache have better expiry properties (#13292) 2022-07-21 17:13:44 +01:00
test_stream.py Generate historic pagination token for /messages when no ?from token provided (#12370) 2022-04-06 11:40:28 +01:00
test_transactions.py Remove HomeServer.get_datastore() (#12031) 2022-02-23 11:04:02 +00:00
test_txn_limit.py Require direct references to configuration variables. (#10985) 2021-10-06 10:47:41 -04:00
test_unsafe_locale.py Refuse to start if DB has an unsafe locale (#12262) 2022-03-23 10:23:05 -07:00
test_user_directory.py Remove remaining bits of groups code. (#12936) 2022-06-01 09:41:25 -04:00