- could do this by only loading all fragments into index when filling gaps, backpaginating, ... and on persister load only load the last fragment. This wouldn't even need a FragmentIndex?
implement SortedArray::setManySorted in a performant manner
implement FragmentIdComparator::add in a performant manner
there is some duplication (also in memory) between SortedArray and TilesCollection. Both keep a sorted list based on fragmentId/eventIndex... TilesCollection doesn't use the index in the event handlers at all. we could allow timeline to export a structure that just emits "these entries are a thing (now)" and not have to go through sorting twice. Timeline would have to keep track of the earliest key so it can use it in loadAtTop, but that should be easy. Hmmm. also, Timeline might want to be in charge of unloading parts of the loaded timeline, and for that it would need to know the order of entries. So maybe not ... we'll see.
so a gap is two connected fragments where either the first fragment has a nextToken and/or the second fragment has a previousToken. It can be both, so we can have a gap where you can fill in from the top, from the bottom (like when limited sync) or both.
also, filling gaps and storing /context, how do we find the fragment we could potentially merge with to look for overlapping events?
with /sync this is all fine and dandy, but with /context is there a way where we don't need to look up every event_id in the store to see if it's already there?
we can do a anyOf(event_id) on timelineStore.index("by_index") by sorting the event ids according to IndexedDb.cmp and passing the next value to cursor.continue(nextId).
so we'll need to remove previous/nextEvent on the timeline store and come up with a method to find the first matched event in a list of eventIds.
so we'll need to map all event ids to an event and return the first one that is not null. If we haven't read all events but we know that all the previous ones are null, then we can already return the result.
we can call this findFirstEventIn(roomId, [event ids])
thoughts:
- ranges in timeline store with fragmentId might not make sense anymore as doing queries over multiple fragment ids doesn't make sense anymore ... still makes sense to have them part of SortKey though ...
- we need a test for querytarget::lookup, or make sure it works well ...
- what format does the persister return newEntries after persisting sync or a gap fill?
- a new fragment can be created during a limited sync
- when doing a /context or /messages call, we could have joined with another fragment
- don't think we need to describe a result spanning multiple fragments here
so:
in case of limited sync, we just say there was a limited sync, this is the fragment that was created for it so we can show a gap in the timeline
in case of a gap fill, we need to return what was changed to the fragment (was it joined with another fragment, what's the new token), and which events were actually added.
we return entries! fragmentboundaryentry(start or end) or evententry. so looks much like the gaps we had before, but now they are not stored in the timeline store, but based on fragments.
so, it feels simpler to store fragmentId and eventIndex as fields on the entry instead of an array/arraybuffer in the field sortKey. Currently, the tiles code somewhat relies on having sortKeys but nothing too hard to change.
so, what we could do:
- we create EventKey(fragmentId, eventIndex) that has the nextKey methods.
- we create a class EventEntry that wraps what is stored in the timeline store. This has a reference to the fragmentindex and has an opaque compare method. Tiles delegate to this method. EventEntry could later on also contain methods like MatrixEvent has in the riot js-sdk, e.g. something to safely dig into the event object.
