feat(base,sdk): Support for dirty cross-process lock in the EventCache
#5856
+2,249
−922
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 tasks
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
4 times, most recently
from
d6bf9c0 to
b13c1fa
Compare
CodSpeed Performance ReportMerging #5856 will not alter performanceComparing Summary
|
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
7 times, most recently
from
08d2859 to
2305356
Compare
This patch renames the `CrossProcessLockKind` type to `CrossProcessLockState`.
This patch adds a `#[must_use]` attribute on `CrossProcessLockGuard` and `CrossProcessLockState` to avoid a misuse.
This patch adds the `CrossProcessLockState::map` method along with its companion `MappedCrossProcessLockState` type. The idea is to facilitate the creation of custom `CrossProcessLockState`-like type in various usage of the cross-process lock.
This patch updates `EventCacheStoreLock::lock()` to return an `EventCacheStoreLockState` instead of an `EventCacheStoreLockGuard`, so that the caller has to handle dirty locks.
This patch replicates the `is_dirty` and `clear_dirty` methods from `CrossProcessLock` to `CrossProcessLockGuard`. It allows to get an access to this API from a guard when one doesn't have the cross-process lock at hand.
…store. This patch changes `EventCacheStoreLockState` to own a clone of the inner store. It helps to remove the `'a` lifetime, and so it “disconnects” from the lifetime of the store.
This patch implements `Clone` for `CrossProcessLockGuard`.
This patch implements `Clone` for `EventCacheStoreLockGuard`.
This patch extracts fields from `RoomEventCacheState` and move them into `RoomEventCacheStateLock`. This lock provides 2 methods: `read` and `write`, respectively to acquire a read-only lock, and a write-only lock, represented by the `RoomEventCacheStateLockReadGuard` and the `RoomEventCacheStateLockWriteGuard` types. All “public” methods on `RoomEventCacheState` now are facade to the read and write guards. This refactoring makes the code to compile with the last change in `EventCacheStore::lock`, which now returns a `EventCacheStoreLockState`. The next step is to re-load `RoomEventCacheStateLock` when the lock is dirty! But before doing that, we need this new mechanism to centralise the management of the store lock.
… dirty. This patch updates the `RoomEventCacheStateLock::read` and `write` methods to reset the state when the cross-process lock is dirty.
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
from
2305356 to
8ee41f4
Compare
This patch replaces `sleep` by `yield_now`, which has the same effect in this case, and makes the tests run faster.
This patch implements `RoomEventCacheStateLockWriteGuard::downgrade` to simplify the code a little bit.
This patch fixes a problem found in a test (not commited yet) where it was impossible to do multiple calls to `read` if the first guard was still alive. See the comments to learn more.
This patch adds the new `test_reset_when_dirty` test, which ensures the state is correctly reset when the cross-process lock over the store becomes dirty.
Hywan
commented
Nov 19, 2025
andybalaam
approved these changes
Nov 20, 2025
Member
andybalaam
left a comment
andybalaam
left a comment
There was a problem hiding this comment.
This looks good, but I definitely didn't understand it all...
I appreciate you breaking it up - that made it much easier to work through, thanks!
| }) | ||
| } | ||
|
|
||
| /// Lock this [`RoomEventCacheStateLock`] with per-thread shared access. |
Member
There was a problem hiding this comment.
Does this really have per-thread shared access given it calls let state_guard = self.locked_state.write().await; inside?
Member
Author
There was a problem hiding this comment.
The first write lock is transformed to a read lock as soon as possible. The result for the caller is indeed a per-thread shared access.
| /// The per-thread/state lock is downgraded atomically, without allowing | ||
| /// any writers to take exclusive access of the lock in the meantime. | ||
| /// | ||
| /// It returns an RAII guard which will drop the write access to the |
Member
There was a problem hiding this comment.
Since it's been downgraded, won't the RAII guard drop the read access to state, not the write access?
Member
Author
There was a problem hiding this comment.
Hmm no, it drops the write lock and obtains a read lock, so the RAII guard drops the write access.
This patch renames the `sender` field of `RoomEventCacheInner` to `update_sender` to clarify what the sender is about.
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
from
9687ab8 to
61aa04f
Compare
This patch updates the reloading of `RoomEventCacheStateLock` when the cross-process lock over the store is dirty to broadcast `RoomEventCacheUpdate` and `RoomEventCacheGenericUpdate`. That way the `Timeline` and other components can react to this reload.
This patch ensures that the `EventCacheStoreLockGuard::clear_dirty` method is correctly called.
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
from
a16c4f3 to
a6eae60
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
EventCachestores and organises all events. ARoomEventCacherepresents a subset of theEventCache, be all events associated to a particular room. That's the type used by eachRoomto store and organise their events. TheEventCacheStoreis the type representing the store for theEventCache. It is used byRoomEventCacheto represent the stored data.RoomEventCachealso has in-memory data. In-memory data are always in-sync with data in the store. TheEventCacheStoreis behind aCrossProcessLock, a special lock that protects data from being acquired across several processes. So far, so good.We've recently introduced the possibility for a cross-process to be invalidated/to be dirty, see #4874. When a
CrossProcessLockis obtained, it is now possible to know if another process obtained it before, thus invalidating the loaded data from the store we might have in-memory. It is a sign we need to refresh the in-memory data.This is what this set of patches is doing: we are reacting to a dirty cross-process lock and refreshing the in-memory state of
RoomEventCacheaccordingly.This set of patches must be reviewed one-by-one to understand the modifications and their impact. The tl;dr is the following:
RoomEventCacheStatebecomesRoomEventCacheStateLock, representing 2 locks at once: the per-thread lock over the state, and the cross-process lock over the store,RoomEventCacheStateInnerbecomesRoomEventCacheStateLockInner,RoomEventCacheInner::stateholds aRoomEventCacheLockinstead of aRwLock<RoomEventCacheState>,RoomEventCacheLockimplements areadand awritemethods, to respectively obtain a read and a write lock. These operations are no longer infallible. It impacts a couple of callers, but nothing fancy.The impacted code is mostly internal code.
Todo
VectorDiffs are broadcasted to the subcribers. This isn't done byshrink_to_last_chunkbecause it's supposed to run when no more subscribers are listening. We must handle this, I forgot!