Compare commits

...
This repository has been archived on 2022-08-19. You can view files and clone it, but cannot push or open issues or pull requests.

233 commits

Author SHA1 Message Date
Ajay Bura
a28ede2f72 Fix undetectable race condition in open timeline 2022-08-14 15:07:15 +05:30
Ajay Bura
dfc84186a8 Merge branch 'thirdroom/dev' of https://github.com/vector-im/hydrogen-web into thirdroom/dev 2022-07-26 10:19:23 +05:30
Ajay Bura
e7b8f553ba v0.0.14 2022-07-26 10:19:17 +05:30
Robert Long
643f43413e
Merge pull request #814 from vector-im/leave-room-hsapi
Add kick, ban & unban api
2022-07-25 21:42:55 -07:00
Ajay Bura
7cfd34823b Add kick, ban & unban api 2022-07-25 16:44:46 +05:30
Robert Long
92a8b4afd0 v0.0.13 2022-07-15 16:28:47 -07:00
Robert Long
a9af3f955c
Merge pull request #796 from vector-im/update-thirdroom-calls
Update thirdroom calls
2022-07-15 16:05:57 -07:00
Robert Long
7c11992626 Fix CallView types 2022-07-15 15:48:53 -07:00
Robert Long
d4eeca346d Merge branch 'bwindels/calls' into update-thirdroom-calls 2022-07-15 15:48:33 -07:00
Bruno Windels
3346f68d25 WIP 2022-07-12 11:59:52 +02:00
Bruno Windels
e9649ec7c2 different streams never have the same id, even for same devices 2022-07-07 15:47:09 +02:00
Bruno Windels
2f08cd8984 clone localMedia in Member when connection, like we do for setMedia 2022-07-07 15:36:49 +02:00
Bruno Windels
f187a51c97 stop replaced track in PeerCall 2022-07-07 15:36:30 +02:00
Robert Long
836bcfb142 v0.0.11 2022-07-06 12:31:24 -07:00
Robert Long
3959968c29
Merge pull request #774 from vector-im/add-more-hsApi
Add more homeserver api
2022-07-06 12:23:34 -07:00
Bruno Windels
206ac6e2dd WIP: prevent stream id from changing when upgrading call 2022-07-05 18:22:36 +02:00
Bruno Windels
5527e2b22c also remove deferred log items from open list when finishing them
otherwise they end up in the logs twice when exporting
2022-07-05 11:02:36 +02:00
Bruno Windels
6aab049052 Merge branch 'master' into bwindels/calls 2022-07-04 16:59:17 +02:00
Ajay Bura
83b233b97e Add type for invite rom 2022-06-29 15:58:51 +05:30
Ajay Bura
eb51e76f9d Add more hs api 2022-06-29 15:58:29 +05:30
Bruno Windels
8a90c48d1e debugging unmuting not working 2022-06-25 05:56:43 +02:00
Bruno Windels
d4ee19c4e4 fix video elements not respecting parent height in callview grid 2022-06-14 12:24:38 +02:00
Bruno Windels
595a15c533 make overlay buttons on call view clickable on chrome 2022-06-14 12:12:10 +02:00
Bruno Windels
ee5bd3b95f Merge branch 'master' into bwindels/calls 2022-06-14 11:09:19 +02:00
Robert Long
bebbbd9db0 v0.0.10 2022-06-13 22:37:13 -07:00
Robert Long
4a2ea5f7f4
Merge pull request #743 from vector-im/make-hsapi-public
Make hs api public
2022-06-13 22:36:00 -07:00
Bruno Windels
41288683fc allow unmuting when we don't yet have a mic/cam track 2022-06-10 17:10:23 +02:00
Bruno Windels
10caba6872 improve calls view 2022-06-09 15:33:59 +02:00
Ajay Bura
dc1f0fecb5 Add profile world type 2022-06-06 13:04:48 +05:30
Bruno Windels
bfdea03bbd start with seq 1, like Element Call does 2022-06-03 15:50:02 +02:00
Bruno Windels
1fab314dd5 return user id for own avatar in call if member hasn't been found 2022-06-03 15:49:45 +02:00
Bruno Windels
ed5fdb8154 don't withhold member event for call just because we don't have profile 2022-06-03 12:43:51 +02:00
Bruno Windels
f8b01ac3cc show profile info for own call member by observing member on room 2022-06-02 17:31:17 +02:00
Bruno Windels
5280467e66 return type is actual subclass options, not the options of ViewModel 2022-06-02 17:30:43 +02:00
Bruno Windels
c8b5c6dd41 expose own user on BaseRoom
so we don't have to pass session around everywhere we need this
2022-06-02 17:30:17 +02:00
Bruno Windels
8ba1d085f6 fix refactor mistake in logging 2022-06-02 15:58:50 +02:00
Bruno Windels
f452c3ff4c enable 96px avatars 2022-06-02 15:58:38 +02:00
Bruno Windels
d66d810fe2 pass updates to avatar view 2022-06-02 15:58:26 +02:00
Bruno Windels
0c20beb1c0 always pass mediaRepo to call vm 2022-06-02 15:58:03 +02:00
Bruno Windels
90b6a5ccb6 update call member info with room member info 2022-06-02 15:56:23 +02:00
Bruno Windels
a52740ed1b give room state handler access to member sync to get sender profile info 2022-06-02 15:55:08 +02:00
Bruno Windels
a530944f7d add logging to seq queueing 2022-06-02 11:11:32 +02:00
Bruno Windels
513c059459 buffer messages as long as seq numbers in between haven't been received 2022-06-02 10:59:14 +02:00
Bruno Windels
a139571e20 move setting seq on outbound messages to member, is specific to_device 2022-06-02 10:59:03 +02:00
Bruno Windels
a014740e72 don't throw when we can't encrypt, just fall back to sending unencrypted 2022-06-01 15:55:43 +02:00
Bruno Windels
83eef2be9d log lack of persisted storage in ... persisted logs! 2022-06-01 15:30:41 +02:00
Bruno Windels
3edfbd2cf6 await hangup here, so log doesn't terminate early 2022-06-01 15:30:25 +02:00
Bruno Windels
9efe294a79 fetch and verify keys on olm call signalling message 2022-06-01 15:29:24 +02:00
Bruno Windels
50ae51e893 encrypt call signalling message only for given device 2022-06-01 15:28:49 +02:00
Bruno Windels
6f0ebeacb7 fetch single device key in DeviceTracker 2022-06-01 15:27:00 +02:00
Bruno Windels
9384fdc885 Merge branch 'bwindels/fix-tracker-changed-key-check' into bwindels/calls 2022-05-31 13:46:10 +02:00
Ajay Bura
46230e59ad Add setProfileAvatarURl and displayName 2022-05-26 11:59:50 +05:30
Ajay Bura
96f58327ee Export LoginFailure from client 2022-05-26 08:57:38 +05:30
Ajay Bura
2465180300 Make sessionInfo public 2022-05-25 14:48:20 +05:30
Ajay Bura
936ac2e932 Make hsApi public 2022-05-25 08:52:31 +05:30
Robert Long
7a4a4f56d8 v0.0.9 2022-05-19 13:20:56 -07:00
Robert Long
28c699939e
Merge pull request #737 from vector-im/add-room-type-in-summary
Add room type in summary
2022-05-19 13:19:44 -07:00
Ajay Bura
203a832c47 Export attachment upload 2022-05-19 21:32:38 +05:30
Bruno Windels
b2d787b96c fix wrong extension in import 2022-05-17 15:55:15 +02:00
Ajay Bura
01cb66ccbf Add isDirectMessage in base room 2022-05-17 12:27:20 +05:30
Ajay Bura
959c03f12f Merge branch 'thirdroom/dev' into add-room-type-in-summary 2022-05-17 09:50:25 +05:30
Robert Long
165b76a486 v0.0.8 2022-05-16 11:29:44 -07:00
Robert Long
cc54dbc1d5
Merge pull request #736 from vector-im/fix-create-room-type
Fix room type with federation allowed
2022-05-16 11:28:10 -07:00
Ajay Bura
d37910c5ac Add getStateEvent in room 2022-05-16 13:53:14 +05:30
Ajay Bura
004e3cb924 Add room type in room summary 2022-05-16 13:40:34 +05:30
Ajay Bura
675ec273fe Fix room type with federation allowded 2022-05-13 11:48:47 +05:30
Robert Long
b9eb45a602 v0.0.7 2022-05-12 21:31:17 -07:00
Robert Long
8f3adf7dc6 Export RoomType 2022-05-12 21:31:07 -07:00
Robert Long
f2633647af fix common.ts import 2022-05-12 21:27:13 -07:00
Robert Long
340c3aa068 Add initial_state and room type to room creation options 2022-05-12 21:25:00 -07:00
Robert Long
190a405e33 Merge branch 'bwindels/calls' into thirdroom/dev 2022-05-12 21:11:11 -07:00
Bruno Windels
6225574df6 write test for ObservedStateKeyValue 2022-05-12 17:52:17 +02:00
Bruno Windels
a50ea7e77b add support for observing room state for single room + initial state 2022-05-12 17:27:03 +02:00
Bruno Windels
db05338596 extract function to iterate over room response state events 2022-05-12 17:26:29 +02:00
Bruno Windels
d727dfd843 add session.observeRoomState to observe state changes in all rooms
and use it for calls
this won't be called for state already received and stored in storage,
that you need to still do yourself
2022-05-12 11:58:28 +02:00
Bruno Windels
ec1568cf1c fix lint error 2022-05-12 11:53:29 +02:00
Robert Long
d6ddf1a469 Merge branch 'bwindels/calls' into thirdroom/dev 2022-05-11 17:33:15 -07:00
Bruno Windels
ae0973b916 Merge branch 'master' into bwindels/calls 2022-05-11 15:13:27 +02:00
Bruno Windels
a923e7e5e1 don't pass errors as log levels 2022-05-11 13:15:03 +02:00
Bruno Windels
2bd8f0fbf3
Merge pull request #734 from vector-im/fix-handleCallMemberEvent
Fix removing members in handleCallMemberEvent
2022-05-11 10:09:03 +02:00
Bruno Windels
5ee4e39bc7 add return type 2022-05-11 10:06:15 +02:00
Robert Long
2abdaf1fc9 v0.0.6 2022-05-10 17:08:11 -07:00
Robert Long
c6d1cba81c Merge branch 'fix-handleCallMemberEvent' into thirdroom/dev 2022-05-10 17:01:11 -07:00
Robert Long
21065791a8 Fix removing members in handleCallMemberEvent 2022-05-10 16:58:03 -07:00
Bruno Windels
e2621015e1 don't include log viewer in production build 2022-05-10 20:08:58 +02:00
Bruno Windels
f6ea7803f2 move logviewer to own package 2022-05-10 18:03:15 +02:00
Bruno Windels
1d900b5184 finish open window and poll code for logviewer 2022-05-10 12:14:09 +02:00
Bruno Windels
c823bb125f fix lint error 2022-05-10 11:20:25 +02:00
Bruno Windels
d85f93fb16 allow opening the logs straight in the log viewer from settings 2022-05-10 11:02:39 +02:00
Bruno Windels
2a729f8969 support loading logs through postMessage in logviewer 2022-05-10 11:02:15 +02:00
Bruno Windels
a2a17dbf7a fix unit test 2022-05-09 14:50:52 +02:00
Bruno Windels
cd8fac2872 update TODO 2022-05-09 14:31:19 +02:00
Bruno Windels
8140e4f2c3 fix typescript errors 2022-05-09 14:23:57 +02:00
Bruno Windels
814cee214c rename asJSON to toJSON 2022-05-06 17:23:07 +02:00
Bruno Windels
d69b1dc3e2 expose log items for exposing debugging info in sdk users 2022-05-06 17:06:56 +02:00
Bruno Windels
fc08fc3744 always log device removal in same way and prevent call id overwritten 2022-05-06 16:59:26 +02:00
Bruno Windels
0fdc6b1c3a log both to idb storage and console, include open items in export
refactor logging api so a logger has multiple reporters, IDBLogPersister
and/or ConsoleReporter.

By default, we add the idb persister for production and both for dev
You can also inject your own logger when creating the platform now.
2022-05-06 15:54:45 +02:00
Robert Long
bc41232105 v0.0.5 2022-05-05 16:54:37 -07:00
Robert Long
f1e152b8aa Merge branch 'bwindels/calls' into thirdroom/dev 2022-05-05 11:02:23 -07:00
Bruno Windels
1a08616df1 logging improvements 2022-05-04 18:44:11 +02:00
Bruno Windels
1a0b11ff7e also log payload when receiving to_device call message, help debug with thirdroom 2022-04-29 14:59:19 +01:00
Bruno Windels
c1c08e9eb0 more logging of callId and sessionIds 2022-04-29 14:58:44 +01:00
Bruno Windels
9938071e1c more sessionId logging 2022-04-29 14:34:03 +01:00
Bruno Windels
bb92d2e30d log session id when adding a member entry 2022-04-29 14:19:10 +01:00
Bruno Windels
8e2e92cd2c this timer should not fire after disposing 2022-04-29 10:11:12 +01:00
Bruno Windels
e1974711f3 dont close this when disconnecting as long as we haven't restructured the log items in general, we can always connect again fr now and assume to reuse the same log item 2022-04-28 16:56:32 +01:00
Bruno Windels
d346f4a3fb add & remove rather than update when session id changed 2022-04-28 16:52:42 +01:00
Bruno Windels
3d83fda69f some more cleanup 2022-04-28 16:52:17 +01:00
Bruno Windels
2d9b69751f some logging cleanup 2022-04-28 16:52:00 +01:00
Bruno Windels
0be75d9c59 update the TODO 2022-04-28 12:45:15 +01:00
Bruno Windels
a91bcf5d22 ensure there is no race between reconnecting & updating the session id
it's probably fine as connect has to wait to receive the
negotiationneeded event before it can send out an invite, but just to
be sure update the session id beforehand
2022-04-28 12:44:13 +01:00
Robert Long
db4c42a5b7 v0.0.4 2022-04-27 11:13:28 -07:00
Robert Long
bf0638b2f3 Merge branch 'bwindels/calls' into thirdroom/dev 2022-04-27 11:08:25 -07:00
Bruno Windels
aa709ee6e9 make text white for now 2022-04-27 19:41:42 +02:00
Bruno Windels
b03b296391 comments, todo housekeeping 2022-04-27 19:41:25 +02:00
Bruno Windels
4f2999f8d8 reconnect when detecting session id change, so we send invite if needed 2022-04-27 19:41:02 +02:00
Bruno Windels
bffce7fafe more logging 2022-04-27 19:40:49 +02:00
Bruno Windels
6394138c4a fix isMuted logic in view model 2022-04-27 19:40:13 +02:00
Bruno Windels
be04eeded0 always reevaluate remote media when receiving a new remote track
not just when we don't know the stream already
this caused the video track to not appear when the other party sends the
invite.
Also added more logging
2022-04-27 17:33:27 +02:00
Bruno Windels
230ccd95ab reset retryCount when disconnecting 2022-04-27 17:33:12 +02:00
Bruno Windels
6b22078140 prevent localMedia being disposed when disconnecting on session change
this would cause us to not send any media anymore and a black screen
on the other side that just refreshed
2022-04-27 11:34:05 +01:00
Bruno Windels
beeb191588 reset member when seeing a new session id
also buffer to_device messages for members we don't have a member event
for already.
2022-04-26 21:11:41 +02:00
Robert Long
ae29adb4e4 Re-enable ts defs 2022-04-26 11:54:03 -07:00
Robert Long
f6a0986b3c Merge branch 'bwindels/calls' into thirdroom/dev 2022-04-26 11:53:50 -07:00
Bruno Windels
da654a8c59 some cleanup 2022-04-26 17:56:46 +02:00
Bruno Windels
eea3830146 emit change when muting so our own video feed gets hidden in the view 2022-04-26 16:18:49 +02:00
Bruno Windels
9ab75e8ed4 fix c/p error mixing up audio and video muting 2022-04-26 15:48:03 +02:00
Bruno Windels
b46ec8bac4 Merge branch 'bwindels/calls-wip' into bwindels/calls 2022-04-26 14:29:04 +02:00
Bruno Windels
f61064c462 nicer UI for calls, show avatar when muted, muted status 2022-04-26 14:27:28 +02:00
Bruno Windels
433dc957ee utility: turn observable value into observable map with one K,V pair 2022-04-26 14:26:56 +02:00
Bruno Windels
c7f7d24273 utility: observable value that emits when event is fired 2022-04-26 14:26:33 +02:00
Bruno Windels
330f234b5a prefer undefined over null 2022-04-26 14:21:19 +02:00
Bruno Windels
3198ca6a92 expose remote mute settings 2022-04-26 14:20:44 +02:00
Bruno Windels
3767f6a420 put theme back to default 2022-04-26 14:19:13 +02:00
Bruno Windels
6e1174e03d Merge branch 'master' into bwindels/calls-wip 2022-04-25 16:44:44 +02:00
Bruno Windels
14dbe340c7 Merge branch 'master' into bwindels/calls-wip 2022-04-25 14:17:21 +02:00
Bruno Windels
a52423856d template view: remove type duplication 2022-04-25 14:05:31 +02:00
Bruno Windels
22df062bbb fix observable typescript errors 2022-04-25 14:05:02 +02:00
Bruno Windels
8b16782270 Merge branch 'master' into bwindels/calls-wip 2022-04-25 12:43:01 +02:00
Bruno Windels
39ecc6cc6d WIP typing errors 2022-04-25 11:27:33 +02:00
Bruno Windels
cdb2a79b62 add muting again, separate from changing media 2022-04-22 14:48:14 +01:00
Bruno Windels
ac60d1b61d remove thick abstraction layer
instead just copy the DOM typing and make it part of the platform layer
2022-04-21 17:40:45 +02:00
Bruno Windels
baa884e9d0 Merge branch 'bwindels/calls-wip' into bwindels/calls-thinner-abstraction 2022-04-21 10:20:03 +02:00
Bruno Windels
10a6269147 always send new metadata after calling setMedia 2022-04-21 10:15:57 +02:00
Bruno Windels
55c6dcf613 don't re-clone streams when not needed 2022-04-21 10:11:24 +02:00
Bruno Windels
99769eb84e implement basic renegotiation 2022-04-21 10:10:49 +02:00
Bruno Windels
82ffb557e5 update TODO 2022-04-21 10:09:31 +02:00
Robert Long
452582e1bf Update package 2022-04-20 16:38:37 -07:00
Robert Long
9f4743e1ce Add eventTimestamp and deviceId for members for host election 2022-04-20 16:35:11 -07:00
Robert Long
239d075084 Update base-manifest.json 2022-04-20 10:36:30 -07:00
Robert Long
fdd067038d Merge branch 'bwindels/calls' into thirdroom/dev 2022-04-20 08:22:58 -07:00
Bruno Windels
d6b239e58f ensure we always set the correct session id when joining 2022-04-20 16:42:20 +02:00
Bruno Windels
4a8af83c8f WIP 2022-04-20 10:57:42 +02:00
Bruno Windels
c42292f1b0 more WIP 2022-04-20 10:57:07 +02:00
Robert Long
47ec614c56 Stop building the failing typescript definitions 2022-04-18 21:59:23 -07:00
Robert Long
3ed9ae7098 Export CallIntent 2022-04-18 21:59:00 -07:00
Robert Long
0f340282e7 Add callType to createCall function parameters 2022-04-18 21:58:49 -07:00
Robert Long
85b77e277f Add power_level_content_override to create room 2022-04-18 21:57:50 -07:00
Bruno Windels
382fba88bd WIP for muting 2022-04-14 23:19:44 +02:00
Bruno Windels
468a0a9698 Merge branch 'master' into bwindels/calls 2022-04-14 13:48:34 +02:00
Bruno Windels
ea1c3a2b86 Merge remote-tracking branch 'origin/bwindels/calls' into bwindels/calls 2022-04-14 13:47:23 +02:00
Bruno Windels
021b8cdcdc send hangup when leaving the call
but not when somebody else leaves the call through a member event
2022-04-14 13:45:21 +02:00
Bruno Windels
ff856d843c ensure all member streams are cloned
so we can stop them without affecting the main one
also, only stop them when disconnecting from the member, rather then
when the peer call ends, as we might want to retry connecting to
the peer with the same stream.
2022-04-14 13:44:11 +02:00
Robert Long
55097e4154 Add intent to CallHandler 2022-04-13 13:08:47 -07:00
Robert Long
2d00d10161 Export LocalMedia 2022-04-13 13:08:33 -07:00
Bruno Windels
bc118b5c0b WIP 2022-04-13 18:34:01 +02:00
Bruno Windels
2d4301fe5a WIP: expose streams, senders and receivers 2022-04-12 21:20:24 +02:00
Bruno Windels
36dc463d23 update TODO 2022-04-12 21:20:15 +02:00
Bruno Windels
0e9307608b update TODO 2022-04-12 14:02:57 +02:00
Bruno Windels
2635adb232 hardcode turn server for now 2022-04-12 14:02:38 +02:00
Bruno Windels
797cb23cc7 implement receiving hangup, and retry on connection failure 2022-04-12 14:02:13 +02:00
Bruno Windels
fd5b2aa7bb only create datachannel on side that sends invite 2022-04-11 16:29:46 +02:00
Bruno Windels
d734a61447 Merge branch 'master' into bwindels/calls 2022-04-11 16:14:34 +02:00
Bruno Windels
a710f394eb fix lint warning 2022-04-11 15:57:23 +02:00
Bruno Windels
517e796e90 remove obsolete import 2022-04-11 15:56:31 +02:00
Bruno Windels
5cacdcfee0 Add leave button to call view 2022-04-11 15:55:02 +02:00
Bruno Windels
c99fc2ad70 use deviceId getter in Member 2022-04-11 15:54:41 +02:00
Bruno Windels
e0efbaeb4e show start time in console logger 2022-04-11 15:54:31 +02:00
Bruno Windels
387bad73b0 remove debug alert 2022-04-11 15:54:20 +02:00
Bruno Windels
9be64730b6 don't automatically join a call we create 2022-04-11 15:54:06 +02:00
Bruno Windels
b84c90891c add very early datachannel support 2022-04-11 15:53:34 +02:00
Bruno Windels
c02e1de001 log when renegotiation would be triggered 2022-04-11 14:55:14 +02:00
Bruno Windels
8e82aad86b fix logic error that made tracks disappear on the second track event 2022-04-11 14:55:08 +02:00
Bruno Windels
8153060831 only send to target device, not all user devices 2022-04-11 13:39:40 +02:00
Bruno Windels
302d4bc02d use session id from member event, and also send it for other party 2022-04-11 13:39:18 +02:00
Bruno Windels
1b0abebe8f remove unused constants 2022-04-11 12:37:05 +02:00
Bruno Windels
156f5b78bf use session_id from member event to set dest_session_id
so our invite event isn't ignored by EC
2022-04-11 12:36:02 +02:00
Bruno Windels
8a06663023 load all call members for now at startup
later on we can be smarter and load then once you interact with the call
2022-04-07 16:55:41 +02:00
Bruno Windels
ad140d5af1 only show video feed when connected 2022-04-07 16:55:26 +02:00
Bruno Windels
a78ae52a54 to test with EC, also load prompt calls at startup 2022-04-07 16:55:10 +02:00
Bruno Windels
b133f58f7a don't throw here for now, although it is probably a sign of why the tracks disappear 2022-04-07 16:54:47 +02:00
Bruno Windels
bade40acc6 log track length 2022-04-07 16:54:36 +02:00
Bruno Windels
1dc46127c3 no need to throw here 2022-04-07 16:54:24 +02:00
Bruno Windels
79411437cf fix who initiates call, needs to be lower, not higher 2022-04-07 16:53:57 +02:00
Bruno Windels
6472800387 impl session id so EC does not ignore our messages 2022-04-07 16:53:37 +02:00
Bruno Windels
fe6e7b09b5 don't encrypt to_device messages for now 2022-04-07 16:50:16 +02:00
Bruno Windels
ad1cceac86 fix error thrown during request when response code is not used 2022-04-07 10:33:12 +02:00
Bruno Windels
2852834ce3 persist calls so they can be quickly loaded after a restart
also use event prefixes compatible with Element Call/MSC
2022-04-07 10:32:43 +02:00
Bruno Windels
1ad5db73a9 some logviewer improvement to help debug call signalling 2022-04-06 18:11:06 +02:00
Bruno Windels
42b470b06b helper to print open items with console logger 2022-03-30 15:19:07 +02:00
Bruno Windels
d7360e7741 fix multiple device support 2022-03-30 15:18:46 +02:00
Bruno Windels
c54ffd4fc3 support multiple devices in call per user 2022-03-29 17:13:33 +02:00
Bruno Windels
ba45178e04 implement terminate and hangup (currently unused) 2022-03-29 12:01:47 +02:00
Bruno Windels
11a9177592 log state changes in PeerCall 2022-03-29 12:01:47 +02:00
Bruno Windels
4bf171def9 small fixes 2022-03-29 12:01:47 +02:00
Bruno Windels
eaf92b382b add structured logging to call code 2022-03-29 12:01:47 +02:00
Bruno Windels
a0a07355d4 more improvements, make hangup work 2022-03-29 12:01:47 +02:00
Bruno Windels
0a37fd561e just enough view code to join a call 2022-03-29 12:01:47 +02:00
Bruno Windels
9efd191f4e some more fixes 2022-03-29 12:01:46 +02:00
Bruno Windels
cad2aa760d some fixes 2022-03-29 12:01:46 +02:00
Bruno Windels
4be82cd472 WIP on UI 2022-03-29 12:01:46 +02:00
Bruno Windels
e760b8e556 basic view model setup 2022-03-29 12:01:46 +02:00
Bruno Windels
e482e3aeef expose mediaDevices and webRTC from platform 2022-03-29 12:01:46 +02:00
Bruno Windels
6daae797e5 fix some ts/lint errors 2022-03-29 12:01:46 +02:00
Bruno Windels
07bc0a2376 move observable values each in their own file 2022-03-29 12:01:46 +02:00
Bruno Windels
1bccbbfa08 fix typescript errors 2022-03-29 12:01:46 +02:00
Bruno Windels
f674492685 remove local media promises (handle them outside of call code) + glare 2022-03-29 12:01:46 +02:00
Bruno Windels
3c160c8a37 handle remote ice candidates 2022-03-29 12:01:46 +02:00
Bruno Windels
b213a45c5c WIP: work on group call state transitions 2022-03-29 12:01:46 +02:00
Bruno Windels
b2ac4bc291 WIP13 2022-03-29 12:01:46 +02:00
Bruno Windels
6da4a4209c WIP: work on group calling code 2022-03-29 12:01:46 +02:00
Bruno Windels
4bedd4737b WIP11 2022-03-29 12:01:46 +02:00
Bruno Windels
60da85d641 WIP10 2022-03-29 12:01:46 +02:00
Bruno Windels
6fe90e60db WIP9 2022-03-29 12:01:46 +02:00
Bruno Windels
ecf7eab3ee WIP8 - implement PeerCall.handleAnswer and other things 2022-03-29 12:01:46 +02:00
Bruno Windels
25b0148073 WIP8 2022-03-29 12:01:46 +02:00
Bruno Windels
98b77fc761 WIP7 2022-03-29 12:01:46 +02:00
Bruno Windels
179c7e74b5 WIP6 2022-03-29 12:01:46 +02:00
Bruno Windels
98e1dcf799 WIP5 2022-03-29 12:01:46 +02:00
Bruno Windels
e5f44aecfb WIP4 2022-03-29 12:01:46 +02:00
Bruno Windels
468841ecea WIP3 2022-03-29 12:01:46 +02:00
Bruno Windels
b12bc52c4a WIP2 2022-03-29 12:01:46 +02:00
Bruno Windels
46ebd55092 WIP 2022-03-29 12:01:46 +02:00
117 changed files with 6053 additions and 1281 deletions

View file

@ -17,6 +17,7 @@ module.exports = {
"globals": { "globals": {
"DEFINE_VERSION": "readonly", "DEFINE_VERSION": "readonly",
"DEFINE_GLOBAL_HASH": "readonly", "DEFINE_GLOBAL_HASH": "readonly",
"DEFINE_PROJECT_DIR": "readonly",
// only available in sw.js // only available in sw.js
"DEFINE_UNHASHED_PRECACHED_ASSETS": "readonly", "DEFINE_UNHASHED_PRECACHED_ASSETS": "readonly",
"DEFINE_HASHED_PRECACHED_ASSETS": "readonly", "DEFINE_HASHED_PRECACHED_ASSETS": "readonly",

View file

@ -31,6 +31,7 @@
}, },
"homepage": "https://github.com/vector-im/hydrogen-web/#readme", "homepage": "https://github.com/vector-im/hydrogen-web/#readme",
"devDependencies": { "devDependencies": {
"@matrixdotorg/structured-logviewer": "^0.0.1",
"@typescript-eslint/eslint-plugin": "^4.29.2", "@typescript-eslint/eslint-plugin": "^4.29.2",
"@typescript-eslint/parser": "^4.29.2", "@typescript-eslint/parser": "^4.29.2",
"acorn": "^8.6.0", "acorn": "^8.6.0",
@ -51,7 +52,7 @@
"postcss-value-parser": "^4.2.0", "postcss-value-parser": "^4.2.0",
"regenerator-runtime": "^0.13.7", "regenerator-runtime": "^0.13.7",
"text-encoding": "^0.7.0", "text-encoding": "^0.7.0",
"typescript": "^4.3.5", "typescript": "^4.4",
"vite": "^2.9.8", "vite": "^2.9.8",
"xxhashjs": "^0.2.2" "xxhashjs": "^0.2.2"
}, },

View file

@ -1,51 +0,0 @@
/*
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
export function openFile(mimeType = null) {
const input = document.createElement("input");
input.setAttribute("type", "file");
input.className = "hidden";
if (mimeType) {
input.setAttribute("accept", mimeType);
}
const promise = new Promise((resolve, reject) => {
const checkFile = () => {
input.removeEventListener("change", checkFile, true);
const file = input.files[0];
document.body.removeChild(input);
if (file) {
resolve(file);
} else {
reject(new Error("no file picked"));
}
}
input.addEventListener("change", checkFile, true);
});
// IE11 needs the input to be attached to the document
document.body.appendChild(input);
input.click();
return promise;
}
export function readFileAsText(file) {
const reader = new FileReader();
const promise = new Promise((resolve, reject) => {
reader.addEventListener("load", evt => resolve(evt.target.result));
reader.addEventListener("error", evt => reject(evt.target.error));
});
reader.readAsText(file);
return promise;
}

View file

@ -1,110 +0,0 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// DOM helper functions
export function isChildren(children) {
// children should be an not-object (that's the attributes), or a domnode, or an array
return typeof children !== "object" || !!children.nodeType || Array.isArray(children);
}
export function classNames(obj, value) {
return Object.entries(obj).reduce((cn, [name, enabled]) => {
if (typeof enabled === "function") {
enabled = enabled(value);
}
if (enabled) {
return cn + (cn.length ? " " : "") + name;
} else {
return cn;
}
}, "");
}
export function setAttribute(el, name, value) {
if (name === "className") {
name = "class";
}
if (value === false) {
el.removeAttribute(name);
} else {
if (value === true) {
value = name;
}
el.setAttribute(name, value);
}
}
export function el(elementName, attributes, children) {
return elNS(HTML_NS, elementName, attributes, children);
}
export function elNS(ns, elementName, attributes, children) {
if (attributes && isChildren(attributes)) {
children = attributes;
attributes = null;
}
const e = document.createElementNS(ns, elementName);
if (attributes) {
for (let [name, value] of Object.entries(attributes)) {
if (name === "className" && typeof value === "object" && value !== null) {
value = classNames(value);
}
setAttribute(e, name, value);
}
}
if (children) {
if (!Array.isArray(children)) {
children = [children];
}
for (let c of children) {
if (!c.nodeType) {
c = text(c);
}
e.appendChild(c);
}
}
return e;
}
export function text(str) {
return document.createTextNode(str);
}
export const HTML_NS = "http://www.w3.org/1999/xhtml";
export const SVG_NS = "http://www.w3.org/2000/svg";
export const TAG_NAMES = {
[HTML_NS]: [
"br", "a", "ol", "ul", "li", "div", "h1", "h2", "h3", "h4", "h5", "h6",
"p", "strong", "em", "span", "img", "section", "main", "article", "aside",
"pre", "button", "time", "input", "textarea", "label", "form", "progress", "output"],
[SVG_NS]: ["svg", "circle"]
};
export const tag = {};
for (const [ns, tags] of Object.entries(TAG_NAMES)) {
for (const tagName of tags) {
tag[tagName] = function(attributes, children) {
return elNS(ns, tagName, attributes, children);
}
}
}

View file

@ -1,209 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<style type="text/css">
html, body {
height: 100%;
}
body {
font-family: sans-serif;
font-size: 1rem;
margin: 0;
display: grid;
grid-template-areas: "nav nav" "items details";
grid-template-columns: 1fr 400px;
grid-template-rows: auto 1fr;
min-height: 0;
}
main {
grid-area: items;
min-width: 0;
min-height: 0;
overflow-y: auto;
padding: 8px;
}
main section h2 {
margin: 2px 14px;
font-size: 1rem;
}
aside {
grid-area: details;
padding: 8px;
}
aside h3 {
word-wrap: anywhere;
}
aside p {
margin: 2px 0;
}
aside .values li span {
word-wrap: ;
word-wrap: anywhere;
padding: 4px;
}
aside .values {
list-style: none;
padding: 0;
border: 1px solid lightgray;
}
aside .values span.key {
width: 30%;
display: block;
}
aside .values span.value {
width: 70%;
display: block;
white-space: pre-wrap;
}
aside .values li {
display: flex;
}
aside .values li:not(:first-child) {
border-top: 1px solid lightgray;
}
nav {
grid-area: nav;
}
.timeline li:not(.expanded) > ol {
display: none;
}
.timeline li > div {
display: flex;
}
.timeline .toggleExpanded {
border: none;
background: none;
width: 24px;
height: 24px;
margin-right: 4px;
cursor: pointer;
}
.timeline .toggleExpanded:before {
content: "▶";
}
.timeline li.expanded > div > .toggleExpanded:before {
content: "▼";
}
.timeline ol {
list-style: none;
padding: 0 0 0 20px;
margin: 0;
}
.timeline .item {
--hue: 100deg;
--brightness: 80%;
background-color: hsl(var(--hue), 60%, var(--brightness));
border: 1px solid hsl(var(--hue), 60%, calc(var(--brightness) - 40%));
border-radius: 4px;
padding: 2px;
display: flex;
margin: 1px;
flex: 1;
min-width: 0;
color: inherit;
text-decoration: none;
}
.timeline .item:not(.has-children) {
margin-left: calc(24px + 4px + 1px);
}
.timeline .item .caption {
white-space: nowrap;
text-overflow: ellipsis;
overflow: hidden;
flex: 1;
}
.timeline .item.level-3 {
--brightness: 90%;
}
.timeline .item.level-2 {
--brightness: 95%;
}
.timeline .item.level-5 {
--brightness: 80%;
}
.timeline .item.level-6, .timeline .item.level-7 {
--hue: 0deg !important;
}
.timeline .item.level-7 {
--brightness: 50%;
color: white;
}
.timeline .item.type-network {
--hue: 30deg;
}
.timeline .item.type-navigation {
--hue: 200deg;
}
.timeline .item.selected {
background-color: Highlight;
border-color: Highlight;
color: HighlightText;
}
.timeline .item.highlighted {
background-color: fuchsia;
color: white;
}
.hidden {
display: none;
}
#highlight {
width: 300px;
}
nav form {
display: inline;
}
</style>
</head>
<body>
<nav>
<button id="openFile">Open log file</button>
<button id="collapseAll">Collapse all</button>
<button id="hideCollapsed">Hide collapsed root items</button>
<button id="hideHighlightedSiblings" title="Hide collapsed siblings of highlighted">Hide non-highlighted</button>
<button id="showAll">Show all</button>
<form id="highlightForm">
<input type="text" id="highlight" name="highlight" placeholder="Highlight a search term" autocomplete="on">
<output id="highlightMatches"></output>
</form>
</nav>
<main></main>
<aside></aside>
<script type="module" src="main.js"></script>
</body>
</html>

View file

@ -1,398 +0,0 @@
/*
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {tag as t} from "./html.js";
import {openFile, readFileAsText} from "./file.js";
const main = document.querySelector("main");
let selectedItemNode;
let rootItem;
let itemByRef;
const logLevels = [undefined, "All", "Debug", "Detail", "Info", "Warn", "Error", "Fatal", "Off"];
main.addEventListener("click", event => {
if (event.target.classList.contains("toggleExpanded")) {
const li = event.target.parentElement.parentElement;
li.classList.toggle("expanded");
} else {
// allow clicking any links other than .item in the timeline, like refs
if (event.target.tagName === "A" && !event.target.classList.contains("item")) {
return;
}
const itemNode = event.target.closest(".item");
if (itemNode) {
// we don't want scroll to jump when clicking
// so prevent default behaviour, and select and push to history manually
event.preventDefault();
selectNode(itemNode);
history.pushState(null, null, `#${itemNode.id}`);
}
}
});
window.addEventListener("hashchange", () => {
const id = window.location.hash.substr(1);
const itemNode = document.getElementById(id);
if (itemNode && itemNode.closest("main")) {
selectNode(itemNode);
itemNode.scrollIntoView({behavior: "smooth", block: "nearest"});
}
});
function selectNode(itemNode) {
if (selectedItemNode) {
selectedItemNode.classList.remove("selected");
}
selectedItemNode = itemNode;
selectedItemNode.classList.add("selected");
let item = rootItem;
let parent;
const indices = selectedItemNode.id.split("/").map(i => parseInt(i, 10));
for(const i of indices) {
parent = item;
item = itemChildren(item)[i];
}
showItemDetails(item, parent, selectedItemNode);
}
function stringifyItemValue(value) {
if (typeof value === "object" && value !== null) {
return JSON.stringify(value, undefined, 2);
} else {
return value + "";
}
}
function showItemDetails(item, parent, itemNode) {
const parentOffset = itemStart(parent) ? `${itemStart(item) - itemStart(parent)}ms` : "none";
const expandButton = t.button("Expand recursively");
expandButton.addEventListener("click", () => expandResursively(itemNode.parentElement.parentElement));
const start = itemStart(item);
const aside = t.aside([
t.h3(itemCaption(item)),
t.p([t.strong("Log level: "), logLevels[itemLevel(item)]]),
t.p([t.strong("Error: "), itemError(item) ? `${itemError(item).name} ${itemError(item).stack}` : "none"]),
t.p([t.strong("Parent offset: "), parentOffset]),
t.p([t.strong("Start: "), new Date(start).toString(), ` (${start})`]),
t.p([t.strong("Duration: "), `${itemDuration(item)}ms`]),
t.p([t.strong("Child count: "), itemChildren(item) ? `${itemChildren(item).length}` : "none"]),
t.p([t.strong("Forced finish: "), (itemForcedFinish(item) || false) + ""]),
t.p(t.strong("Values:")),
t.ul({class: "values"}, Object.entries(itemValues(item)).map(([key, value]) => {
let valueNode;
if (key === "ref") {
const refItem = itemByRef.get(value);
if (refItem) {
valueNode = t.a({href: `#${refItem.id}`}, itemCaption(refItem));
} else {
valueNode = `unknown ref ${value}`;
}
} else {
valueNode = stringifyItemValue(value);
}
return t.li([
t.span({className: "key"}, normalizeValueKey(key)),
t.span({className: "value"}, valueNode)
]);
})),
t.p(expandButton)
]);
document.querySelector("aside").replaceWith(aside);
}
function expandResursively(li) {
li.classList.add("expanded");
const ol = li.querySelector("ol");
if (ol) {
const len = ol.children.length;
for (let i = 0; i < len; i += 1) {
expandResursively(ol.children[i]);
}
}
}
document.getElementById("openFile").addEventListener("click", loadFile);
function getRootItemHeader(prevItem, item) {
if (prevItem) {
const diff = itemStart(item) - itemEnd(prevItem);
if (diff >= 0) {
return `+ ${formatTime(diff)}`;
} else {
const overlap = -diff;
if (overlap >= itemDuration(item)) {
return `ran entirely in parallel with`;
} else {
return `ran ${formatTime(-diff)} in parallel with`;
}
}
} else {
return new Date(itemStart(item)).toString();
}
}
async function loadFile() {
const file = await openFile();
const json = await readFileAsText(file);
const logs = JSON.parse(json);
logs.items.sort((a, b) => itemStart(a) - itemStart(b));
rootItem = {c: logs.items};
itemByRef = new Map();
preprocessRecursively(rootItem, null, itemByRef, []);
const fragment = logs.items.reduce((fragment, item, i, items) => {
const prevItem = i === 0 ? null : items[i - 1];
fragment.appendChild(t.section([
t.h2(getRootItemHeader(prevItem, item)),
t.div({className: "timeline"}, t.ol(itemToNode(item, [i])))
]));
return fragment;
}, document.createDocumentFragment());
main.replaceChildren(fragment);
}
// TODO: make this use processRecursively
function preprocessRecursively(item, parentElement, refsMap, path) {
item.s = (parentElement?.s || 0) + item.s;
if (itemRefSource(item)) {
refsMap.set(itemRefSource(item), item);
}
if (itemChildren(item)) {
for (let i = 0; i < itemChildren(item).length; i += 1) {
// do it in advance for a child as we don't want to do it for the rootItem
const child = itemChildren(item)[i];
const childPath = path.concat(i);
child.id = childPath.join("/");
preprocessRecursively(child, item, refsMap, childPath);
}
}
}
const MS_IN_SEC = 1000;
const MS_IN_MIN = MS_IN_SEC * 60;
const MS_IN_HOUR = MS_IN_MIN * 60;
const MS_IN_DAY = MS_IN_HOUR * 24;
function formatTime(ms) {
let str = "";
if (ms > MS_IN_DAY) {
const days = Math.floor(ms / MS_IN_DAY);
ms -= days * MS_IN_DAY;
str += `${days}d`;
}
if (ms > MS_IN_HOUR) {
const hours = Math.floor(ms / MS_IN_HOUR);
ms -= hours * MS_IN_HOUR;
str += `${hours}h`;
}
if (ms > MS_IN_MIN) {
const mins = Math.floor(ms / MS_IN_MIN);
ms -= mins * MS_IN_MIN;
str += `${mins}m`;
}
if (ms > MS_IN_SEC) {
const secs = ms / MS_IN_SEC;
str += `${secs.toFixed(2)}s`;
} else if (ms > 0 || !str.length) {
str += `${ms}ms`;
}
return str;
}
function itemChildren(item) { return item.c; }
function itemStart(item) { return item.s; }
function itemEnd(item) { return item.s + item.d; }
function itemDuration(item) { return item.d; }
function itemValues(item) { return item.v; }
function itemLevel(item) { return item.l; }
function itemLabel(item) { return item.v?.l; }
function itemType(item) { return item.v?.t; }
function itemError(item) { return item.e; }
function itemForcedFinish(item) { return item.f; }
function itemRef(item) { return item.v?.ref; }
function itemRefSource(item) { return item.v?.refId; }
function itemShortErrorMessage(item) {
if (itemError(item)) {
const e = itemError(item);
return e.name || e.stack.substr(0, e.stack.indexOf("\n"));
}
}
function itemCaption(item) {
if (itemType(item) === "network") {
return `${itemValues(item)?.method} ${itemValues(item)?.url}`;
} else if (itemLabel(item) && itemValues(item)?.id) {
return `${itemLabel(item)} ${itemValues(item).id}`;
} else if (itemLabel(item) && itemValues(item)?.status) {
return `${itemLabel(item)} (${itemValues(item).status})`;
} else if (itemLabel(item) && itemError(item)) {
return `${itemLabel(item)} (${itemShortErrorMessage(item)})`;
} else if (itemRef(item)) {
const refItem = itemByRef.get(itemRef(item));
if (refItem) {
return `ref "${itemCaption(refItem)}"`
} else {
return `unknown ref ${itemRef(item)}`
}
} else {
return itemLabel(item) || itemType(item);
}
}
function normalizeValueKey(key) {
switch (key) {
case "t": return "type";
case "l": return "label";
default: return key;
}
}
// returns the node and the total range (recursively) occupied by the node
function itemToNode(item) {
const hasChildren = !!itemChildren(item)?.length;
const className = {
item: true,
"has-children": hasChildren,
error: itemError(item),
[`type-${itemType(item)}`]: !!itemType(item),
[`level-${itemLevel(item)}`]: true,
};
const id = item.id;
let captionNode;
if (itemRef(item)) {
const refItem = itemByRef.get(itemRef(item));
if (refItem) {
captionNode = ["ref ", t.a({href: `#${refItem.id}`}, itemCaption(refItem))];
}
}
if (!captionNode) {
captionNode = itemCaption(item);
}
const li = t.li([
t.div([
hasChildren ? t.button({className: "toggleExpanded"}) : "",
t.a({className, id, href: `#${id}`}, [
t.span({class: "caption"}, captionNode),
t.span({class: "duration"}, `(${formatTime(itemDuration(item))})`),
])
])
]);
if (itemChildren(item) && itemChildren(item).length) {
li.appendChild(t.ol(itemChildren(item).map(item => {
return itemToNode(item);
})));
}
return li;
}
const highlightForm = document.getElementById("highlightForm");
highlightForm.addEventListener("submit", evt => {
evt.preventDefault();
const matchesOutput = document.getElementById("highlightMatches");
const query = document.getElementById("highlight").value;
if (query) {
matchesOutput.innerText = "Searching…";
let matches = 0;
processRecursively(rootItem, item => {
let domNode = document.getElementById(item.id);
if (itemMatchesFilter(item, query)) {
matches += 1;
domNode.classList.add("highlighted");
domNode = domNode.parentElement;
while (domNode.nodeName !== "SECTION") {
if (domNode.nodeName === "LI") {
domNode.classList.add("expanded");
}
domNode = domNode.parentElement;
}
} else {
domNode.classList.remove("highlighted");
}
});
matchesOutput.innerText = `${matches} matches`;
} else {
for (const node of document.querySelectorAll(".highlighted")) {
node.classList.remove("highlighted");
}
matchesOutput.innerText = "";
}
});
function itemMatchesFilter(item, query) {
if (itemError(item)) {
if (valueMatchesQuery(itemError(item), query)) {
return true;
}
}
return valueMatchesQuery(itemValues(item), query);
}
function valueMatchesQuery(value, query) {
if (typeof value === "string") {
return value.includes(query);
} else if (typeof value === "object" && value !== null) {
for (const key in value) {
if (value.hasOwnProperty(key) && valueMatchesQuery(value[key], query)) {
return true;
}
}
} else if (typeof value === "number") {
return value.toString().includes(query);
}
return false;
}
function processRecursively(item, callback, parentItem) {
if (item.id) {
callback(item, parentItem);
}
if (itemChildren(item)) {
for (let i = 0; i < itemChildren(item).length; i += 1) {
// do it in advance for a child as we don't want to do it for the rootItem
const child = itemChildren(item)[i];
processRecursively(child, callback, item);
}
}
}
document.getElementById("collapseAll").addEventListener("click", () => {
for (const node of document.querySelectorAll(".expanded")) {
node.classList.remove("expanded");
}
});
document.getElementById("hideCollapsed").addEventListener("click", () => {
for (const node of document.querySelectorAll("section > div.timeline > ol > li:not(.expanded)")) {
node.closest("section").classList.add("hidden");
}
});
document.getElementById("hideHighlightedSiblings").addEventListener("click", () => {
for (const node of document.querySelectorAll(".highlighted")) {
const list = node.closest("ol");
const siblings = Array.from(list.querySelectorAll("li > div > a:not(.highlighted)")).map(n => n.closest("li"));
for (const sibling of siblings) {
if (!sibling.classList.contains("expanded")) {
sibling.classList.add("hidden");
}
}
}
});
document.getElementById("showAll").addEventListener("click", () => {
for (const node of document.querySelectorAll(".hidden")) {
node.classList.remove("hidden");
}
});

View file

@ -24,7 +24,13 @@ const idToPrepend = "icon-url";
function findAndReplaceUrl(decl, urlVariables, counter) { function findAndReplaceUrl(decl, urlVariables, counter) {
const value = decl.value; const value = decl.value;
const parsed = valueParser(value); let parsed;
try {
parsed = valueParser(value);
} catch (err) {
console.log(`Error trying to parse ${decl}`);
throw err;
}
parsed.walk(node => { parsed.walk(node => {
if (node.type !== "function" || node.value !== "url") { if (node.type !== "function" || node.value !== "url") {
return; return;

View file

@ -1,7 +1,7 @@
{ {
"name": "hydrogen-view-sdk", "name": "@thirdroom/hydrogen-view-sdk",
"description": "Embeddable matrix client library, including view components", "description": "Embeddable matrix client library, including view components",
"version": "0.0.13", "version": "0.0.14",
"main": "./lib-build/hydrogen.cjs.js", "main": "./lib-build/hydrogen.cjs.js",
"exports": { "exports": {
".": { ".": {

View file

@ -0,0 +1,23 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
export interface AvatarSource {
get avatarLetter(): string;
get avatarColorNumber(): number;
avatarUrl(size: number): string | undefined;
get avatarTitle(): string;
}

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {SortedArray} from "../observable/index.js"; import {SortedArray} from "../observable/index";
import {ViewModel} from "./ViewModel"; import {ViewModel} from "./ViewModel";
import {avatarInitials, getIdentifierColorNumber} from "./avatar"; import {avatarInitials, getIdentifierColorNumber} from "./avatar";

View file

@ -47,7 +47,7 @@ export class ViewModel<O extends Options = Options> extends EventEmitter<{change
this._options = options; this._options = options;
} }
childOptions<T extends Object>(explicitOptions: T): T & Options { childOptions<T extends Object>(explicitOptions: T): T & O {
return Object.assign({}, this._options, explicitOptions); return Object.assign({}, this._options, explicitOptions);
} }
@ -115,7 +115,7 @@ export class ViewModel<O extends Options = Options> extends EventEmitter<{change
return result; return result;
} }
emitChange(changedProps: any): void { emitChange(changedProps?: any): void {
if (this._options.emitChange) { if (this._options.emitChange) {
this._options.emitChange(changedProps); this._options.emitChange(changedProps);
} else { } else {

View file

@ -51,10 +51,10 @@ export function getIdentifierColorNumber(id: string): number {
return (hashCode(id) % 8) + 1; return (hashCode(id) % 8) + 1;
} }
export function getAvatarHttpUrl(avatarUrl: string, cssSize: number, platform: Platform, mediaRepository: MediaRepository): string | null { export function getAvatarHttpUrl(avatarUrl: string | undefined, cssSize: number, platform: Platform, mediaRepository: MediaRepository): string | undefined {
if (avatarUrl) { if (avatarUrl) {
const imageSize = cssSize * platform.devicePixelRatio; const imageSize = cssSize * platform.devicePixelRatio;
return mediaRepository.mxcUrlThumbnail(avatarUrl, imageSize, imageSize, "crop"); return mediaRepository.mxcUrlThumbnail(avatarUrl, imageSize, imageSize, "crop");
} }
return null; return undefined;
} }

View file

@ -14,7 +14,8 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseObservableValue, ObservableValue} from "../../observable/ObservableValue"; import {ObservableValue} from "../../observable/value/ObservableValue";
import {BaseObservableValue} from "../../observable/value/BaseObservableValue";
export class Navigation { export class Navigation {
constructor(allowsChild) { constructor(allowsChild) {

View file

@ -16,7 +16,7 @@ limitations under the License.
import {ViewModel} from "../ViewModel"; import {ViewModel} from "../ViewModel";
import {imageToInfo} from "./common.js"; import {imageToInfo} from "./common.js";
import {RoomType} from "../../matrix/room/common"; import {RoomVisibility} from "../../matrix/room/common";
export class CreateRoomViewModel extends ViewModel { export class CreateRoomViewModel extends ViewModel {
constructor(options) { constructor(options) {
@ -89,7 +89,7 @@ export class CreateRoomViewModel extends ViewModel {
} }
} }
const roomBeingCreated = this._session.createRoom({ const roomBeingCreated = this._session.createRoom({
type: this.isPublic ? RoomType.Public : RoomType.Private, type: this.isPublic ? RoomVisibility.Public : RoomVisibility.Private,
name: this._name ?? undefined, name: this._name ?? undefined,
topic: this._topic ?? undefined, topic: this._topic ?? undefined,
isEncrypted: !this.isPublic && this._isEncrypted, isEncrypted: !this.isPublic && this._isEncrypted,

View file

@ -186,7 +186,7 @@ export class RoomGridViewModel extends ViewModel {
} }
import {createNavigation} from "../navigation/index.js"; import {createNavigation} from "../navigation/index.js";
import {ObservableValue} from "../../observable/ObservableValue"; import {ObservableValue} from "../../observable/value/ObservableValue";
export function tests() { export function tests() {
class RoomVMMock { class RoomVMMock {

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {ObservableValue} from "../../observable/ObservableValue"; import {ObservableValue} from "../../observable/value/ObservableValue";
import {RoomStatus} from "../../matrix/room/common"; import {RoomStatus} from "../../matrix/room/common";
/** /**

View file

@ -99,6 +99,9 @@ export class SessionViewModel extends ViewModel {
start() { start() {
this._sessionStatusViewModel.start(); this._sessionStatusViewModel.start();
this._client.session.callHandler.loadCalls("m.ring");
// TODO: only do this when opening the room
this._client.session.callHandler.loadCalls("m.prompt");
} }
get activeMiddleViewModel() { get activeMiddleViewModel() {
@ -174,7 +177,7 @@ export class SessionViewModel extends ViewModel {
_createRoomViewModelInstance(roomId) { _createRoomViewModelInstance(roomId) {
const room = this._client.session.rooms.get(roomId); const room = this._client.session.rooms.get(roomId);
if (room) { if (room) {
const roomVM = new RoomViewModel(this.childOptions({room})); const roomVM = new RoomViewModel(this.childOptions({room, session: this._client.session}));
roomVM.load(); roomVM.load();
return roomVM; return roomVM;
} }
@ -191,7 +194,7 @@ export class SessionViewModel extends ViewModel {
async _createArchivedRoomViewModel(roomId) { async _createArchivedRoomViewModel(roomId) {
const room = await this._client.session.loadArchivedRoom(roomId); const room = await this._client.session.loadArchivedRoom(roomId);
if (room) { if (room) {
const roomVM = new RoomViewModel(this.childOptions({room})); const roomVM = new RoomViewModel(this.childOptions({room, session: this._client.session}));
roomVM.load(); roomVM.load();
return roomVM; return roomVM;
} }

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {ViewModel} from "../../ViewModel"; import {ViewModel} from "../../ViewModel";
import {RoomType} from "../../../matrix/room/common"; import {RoomVisibility} from "../../../matrix/room/common";
import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar"; import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar";
export class MemberDetailsViewModel extends ViewModel { export class MemberDetailsViewModel extends ViewModel {
@ -87,7 +87,7 @@ export class MemberDetailsViewModel extends ViewModel {
let roomId = room?.id; let roomId = room?.id;
if (!roomId) { if (!roomId) {
const roomBeingCreated = await this._session.createRoom({ const roomBeingCreated = await this._session.createRoom({
type: RoomType.DirectMessage, type: RoomVisibility.DirectMessage,
invites: [this.userId] invites: [this.userId]
}); });
roomId = roomBeingCreated.id; roomId = roomBeingCreated.id;

View file

@ -0,0 +1,245 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AvatarSource} from "../../AvatarSource";
import {ViewModel, Options as BaseOptions} from "../../ViewModel";
import {getStreamVideoTrack, getStreamAudioTrack} from "../../../matrix/calls/common";
import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar";
import {EventObservableValue} from "../../../observable/value/EventObservableValue";
import {ObservableValueMap} from "../../../observable/map/ObservableValueMap";
import type {Room} from "../../../matrix/room/Room";
import type {GroupCall} from "../../../matrix/calls/group/GroupCall";
import type {Member} from "../../../matrix/calls/group/Member";
import type {RoomMember} from "../../../matrix/room/members/RoomMember";
import type {BaseObservableList} from "../../../observable/list/BaseObservableList";
import type {BaseObservableValue} from "../../../observable/value/BaseObservableValue";
import type {Stream} from "../../../platform/types/MediaDevices";
import type {MediaRepository} from "../../../matrix/net/MediaRepository";
type Options = BaseOptions & {
call: GroupCall,
room: Room,
};
export class CallViewModel extends ViewModel<Options> {
public readonly memberViewModels: BaseObservableList<IStreamViewModel>;
constructor(options: Options) {
super(options);
const ownMemberViewModelMap = new ObservableValueMap("self", new EventObservableValue(this.call, "change"))
.mapValues((call, emitChange) => new OwnMemberViewModel(this.childOptions({call, emitChange})), () => {});
this.memberViewModels = this.call.members
.filterValues(member => member.isConnected)
.mapValues(member => new CallMemberViewModel(this.childOptions({member, mediaRepository: this.getOption("room").mediaRepository})))
.join(ownMemberViewModelMap)
.sortValues((a, b) => a.compare(b));
this.track(this.memberViewModels.subscribe({
onRemove: () => {
this.emitChange(); // update memberCount
},
onAdd: () => {
this.emitChange(); // update memberCount
},
onUpdate: () => {},
onReset: () => {},
onMove: () => {}
}))
}
get isCameraMuted(): boolean {
return this.call.muteSettings?.camera ?? true;
}
get isMicrophoneMuted(): boolean {
return this.call.muteSettings?.microphone ?? true;
}
get memberCount(): number {
return this.memberViewModels.length;
}
get name(): string {
return this.call.name;
}
get id(): string {
return this.call.id;
}
private get call(): GroupCall {
return this.getOption("call");
}
async hangup() {
if (this.call.hasJoined) {
await this.call.leave();
}
}
async toggleCamera() {
const {localMedia, muteSettings} = this.call;
if (muteSettings && localMedia) {
// unmute but no track?
if (muteSettings.camera && !getStreamVideoTrack(localMedia.userMedia)) {
const stream = await this.platform.mediaDevices.getMediaTracks(!muteSettings.microphone, true);
await this.call.setMedia(localMedia.withUserMedia(stream));
} else {
await this.call.setMuted(muteSettings.toggleCamera());
}
this.emitChange();
}
}
async toggleMicrophone() {
const {localMedia, muteSettings} = this.call;
if (muteSettings && localMedia) {
// unmute but no track?
if (muteSettings.microphone && !getStreamAudioTrack(localMedia.userMedia)) {
const stream = await this.platform.mediaDevices.getMediaTracks(true, !muteSettings.camera);
console.log("got tracks", Array.from(stream.getTracks()).map((t: MediaStreamTrack) => { return {kind: t.kind, id: t.id};}))
await this.call.setMedia(localMedia.withUserMedia(stream));
} else {
await this.call.setMuted(muteSettings.toggleMicrophone());
}
this.emitChange();
}
}
}
class OwnMemberViewModel extends ViewModel<Options> implements IStreamViewModel {
private memberObservable: undefined | BaseObservableValue<RoomMember>;
constructor(options: Options) {
super(options);
this.init();
}
async init() {
const room = this.getOption("room");
this.memberObservable = await room.observeMember(room.user.id);
this.track(this.memberObservable!.subscribe(() => {
this.emitChange(undefined);
}));
}
get stream(): Stream | undefined {
return this.call.localMedia?.userMedia;
}
private get call(): GroupCall {
return this.getOption("call");
}
get isCameraMuted(): boolean {
return this.call.muteSettings?.camera ?? true;
}
get isMicrophoneMuted(): boolean {
return this.call.muteSettings?.microphone ?? true;
}
get avatarLetter(): string {
const member = this.memberObservable?.get();
if (member) {
return avatarInitials(member.name);
} else {
return this.getOption("room").user.id;
}
}
get avatarColorNumber(): number {
return getIdentifierColorNumber(this.getOption("room").user.id);
}
avatarUrl(size: number): string | undefined {
const member = this.memberObservable?.get();
if (member) {
return getAvatarHttpUrl(member.avatarUrl, size, this.platform, this.getOption("room").mediaRepository);
}
}
get avatarTitle(): string {
const member = this.memberObservable?.get();
if (member) {
return member.name;
} else {
return this.getOption("room").user.id;
}
}
compare(other: OwnMemberViewModel | CallMemberViewModel): number {
return -1;
}
}
type MemberOptions = BaseOptions & {
member: Member,
mediaRepository: MediaRepository
};
export class CallMemberViewModel extends ViewModel<MemberOptions> implements IStreamViewModel {
get stream(): Stream | undefined {
return this.member.remoteMedia?.userMedia;
}
private get member(): Member {
return this.getOption("member");
}
get isCameraMuted(): boolean {
return this.member.remoteMuteSettings?.camera ?? true;
}
get isMicrophoneMuted(): boolean {
return this.member.remoteMuteSettings?.microphone ?? true;
}
get avatarLetter(): string {
return avatarInitials(this.member.member.name);
}
get avatarColorNumber(): number {
return getIdentifierColorNumber(this.member.userId);
}
avatarUrl(size: number): string | undefined {
const {avatarUrl} = this.member.member;
const mediaRepository = this.getOption("mediaRepository");
return getAvatarHttpUrl(avatarUrl, size, this.platform, mediaRepository);
}
get avatarTitle(): string {
return this.member.member.name;
}
compare(other: OwnMemberViewModel | CallMemberViewModel): number {
if (other instanceof OwnMemberViewModel) {
return -other.compare(this);
}
const myUserId = this.member.member.userId;
const otherUserId = other.member.member.userId;
if(myUserId === otherUserId) {
return 0;
}
return myUserId < otherUserId ? -1 : 1;
}
}
export interface IStreamViewModel extends AvatarSource, ViewModel {
get stream(): Stream | undefined;
get isCameraMuted(): boolean;
get isMicrophoneMuted(): boolean;
}

View file

@ -17,9 +17,12 @@ limitations under the License.
import {TimelineViewModel} from "./timeline/TimelineViewModel.js"; import {TimelineViewModel} from "./timeline/TimelineViewModel.js";
import {ComposerViewModel} from "./ComposerViewModel.js" import {ComposerViewModel} from "./ComposerViewModel.js"
import {CallViewModel} from "./CallViewModel"
import {PickMapObservableValue} from "../../../observable/value/PickMapObservableValue";
import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar"; import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar";
import {ViewModel} from "../../ViewModel"; import {ViewModel} from "../../ViewModel";
import {imageToInfo} from "../common.js"; import {imageToInfo} from "../common.js";
import {LocalMedia} from "../../../matrix/calls/LocalMedia";
// TODO: remove fallback so default isn't included in bundle for SDK users that have their custom tileClassForEntry // TODO: remove fallback so default isn't included in bundle for SDK users that have their custom tileClassForEntry
// this is a breaking SDK change though to make this option mandatory // this is a breaking SDK change though to make this option mandatory
import {tileClassForEntry as defaultTileClassForEntry} from "./timeline/tiles/index"; import {tileClassForEntry as defaultTileClassForEntry} from "./timeline/tiles/index";
@ -43,6 +46,31 @@ export class RoomViewModel extends ViewModel {
} }
this._clearUnreadTimout = null; this._clearUnreadTimout = null;
this._closeUrl = this.urlCreator.urlUntilSegment("session"); this._closeUrl = this.urlCreator.urlUntilSegment("session");
this._setupCallViewModel();
}
_setupCallViewModel() {
// pick call for this room with lowest key
const calls = this.getOption("session").callHandler.calls;
this._callObservable = new PickMapObservableValue(calls.filterValues(c => {
return c.roomId === this._room.id && c.hasJoined;
}));
this._callViewModel = undefined;
this.track(this._callObservable.subscribe(call => {
if (call && this._callViewModel && call.id === this._callViewModel.id) {
return;
}
this._callViewModel = this.disposeTracked(this._callViewModel);
if (call) {
this._callViewModel = this.track(new CallViewModel(this.childOptions({call, room: this._room})));
}
this.emitChange("callViewModel");
}));
const call = this._callObservable.get();
// TODO: cleanup this duplication to create CallViewModel
if (call) {
this._callViewModel = this.track(new CallViewModel(this.childOptions({call, room: this._room})));
}
} }
async load() { async load() {
@ -50,6 +78,7 @@ export class RoomViewModel extends ViewModel {
try { try {
const timeline = await this._room.openTimeline(); const timeline = await this._room.openTimeline();
this._tileOptions = this.childOptions({ this._tileOptions = this.childOptions({
session: this.getOption("session"),
roomVM: this, roomVM: this,
timeline, timeline,
tileClassForEntry: this._tileClassForEntry, tileClassForEntry: this._tileClassForEntry,
@ -317,6 +346,10 @@ export class RoomViewModel extends ViewModel {
return this._composerVM; return this._composerVM;
} }
get callViewModel() {
return this._callViewModel;
}
openDetailsPanel() { openDetailsPanel() {
let path = this.navigation.path.until("room"); let path = this.navigation.path.until("room");
path = path.with(this.navigation.segment("right-panel", true)); path = path.with(this.navigation.segment("right-panel", true));
@ -329,6 +362,19 @@ export class RoomViewModel extends ViewModel {
this._composerVM.setReplyingTo(entry); this._composerVM.setReplyingTo(entry);
} }
} }
async startCall() {
try {
const session = this.getOption("session");
const stream = await this.platform.mediaDevices.getMediaTracks(false, true);
const localMedia = new LocalMedia().withUserMedia(stream);
// this will set the callViewModel above as a call will be added to callHandler.calls
const call = await session.callHandler.createCall(this._room.id, "m.video", "A call " + Math.round(this.platform.random() * 100));
await call.join(localMedia);
} catch (err) {
console.error(err.stack);
}
}
} }
function videoToInfo(video) { function videoToInfo(video) {

View file

@ -189,7 +189,7 @@ import {HomeServer as MockHomeServer} from "../../../../mocks/HomeServer.js";
// other imports // other imports
import {BaseMessageTile} from "./tiles/BaseMessageTile.js"; import {BaseMessageTile} from "./tiles/BaseMessageTile.js";
import {MappedList} from "../../../../observable/list/MappedList"; import {MappedList} from "../../../../observable/list/MappedList";
import {ObservableValue} from "../../../../observable/ObservableValue"; import {ObservableValue} from "../../../../observable/value/ObservableValue";
import {PowerLevels} from "../../../../matrix/room/PowerLevels.js"; import {PowerLevels} from "../../../../matrix/room/PowerLevels.js";
export function tests() { export function tests() {

View file

@ -49,14 +49,6 @@ export class BaseMessageTile extends SimpleTile {
return `https://matrix.to/#/${encodeURIComponent(this.sender)}`; return `https://matrix.to/#/${encodeURIComponent(this.sender)}`;
} }
get displayName() {
return this._entry.displayName || this.sender;
}
get sender() {
return this._entry.sender;
}
get memberPanelLink() { get memberPanelLink() {
return `${this.urlCreator.urlUntilSegment("room")}/member/${this.sender}`; return `${this.urlCreator.urlUntilSegment("room")}/member/${this.sender}`;
} }

View file

@ -0,0 +1,94 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {SimpleTile} from "./SimpleTile.js";
import {LocalMedia} from "../../../../../matrix/calls/LocalMedia";
// TODO: timeline entries for state events with the same state key and type
// should also update previous entries in the timeline, so we can update the name of the call, whether it is terminated, etc ...
// alternatively, we could just subscribe to the GroupCall and spontanously emit an update when it updates
export class CallTile extends SimpleTile {
constructor(entry, options) {
super(entry, options);
const calls = this.getOption("session").callHandler.calls;
this._call = calls.get(this._entry.stateKey);
this._callSubscription = undefined;
if (this._call) {
this._callSubscription = this._call.disposableOn("change", () => {
// unsubscribe when terminated
if (this._call.isTerminated) {
this._callSubscription = this._callSubscription();
this._call = undefined;
}
this.emitChange();
});
}
}
get confId() {
return this._entry.stateKey;
}
get shape() {
return "call";
}
get name() {
return this._entry.content["m.name"];
}
get canJoin() {
return this._call && !this._call.hasJoined;
}
get canLeave() {
return this._call && this._call.hasJoined;
}
get label() {
if (this._call) {
if (this._call.hasJoined) {
return `Ongoing call (${this.name}, ${this.confId})`;
} else {
return `${this.displayName} started a call (${this.name}, ${this.confId})`;
}
} else {
return `Call finished, started by ${this.displayName} (${this.name}, ${this.confId})`;
}
}
async join() {
if (this.canJoin) {
const stream = await this.platform.mediaDevices.getMediaTracks(false, true);
const localMedia = new LocalMedia().withUserMedia(stream);
await this._call.join(localMedia);
}
}
async leave() {
if (this.canLeave) {
this._call.leave();
}
}
dispose() {
if (this._callSubscription) {
this._callSubscription = this._callSubscription();
}
}
}

View file

@ -159,4 +159,12 @@ export class SimpleTile extends ViewModel {
get _ownMember() { get _ownMember() {
return this._options.timeline.me; return this._options.timeline.me;
} }
get displayName() {
return this._entry.displayName || this.sender;
}
get sender() {
return this._entry.sender;
}
} }

View file

@ -26,9 +26,11 @@ import {RoomMemberTile} from "./RoomMemberTile.js";
import {EncryptedEventTile} from "./EncryptedEventTile.js"; import {EncryptedEventTile} from "./EncryptedEventTile.js";
import {EncryptionEnabledTile} from "./EncryptionEnabledTile.js"; import {EncryptionEnabledTile} from "./EncryptionEnabledTile.js";
import {MissingAttachmentTile} from "./MissingAttachmentTile.js"; import {MissingAttachmentTile} from "./MissingAttachmentTile.js";
import {CallTile} from "./CallTile.js";
import type {SimpleTile} from "./SimpleTile.js"; import type {SimpleTile} from "./SimpleTile.js";
import type {Room} from "../../../../../matrix/room/Room"; import type {Room} from "../../../../../matrix/room/Room";
import type {Session} from "../../../../../matrix/Session";
import type {Timeline} from "../../../../../matrix/room/timeline/Timeline"; import type {Timeline} from "../../../../../matrix/room/timeline/Timeline";
import type {FragmentBoundaryEntry} from "../../../../../matrix/room/timeline/entries/FragmentBoundaryEntry"; import type {FragmentBoundaryEntry} from "../../../../../matrix/room/timeline/entries/FragmentBoundaryEntry";
import type {EventEntry} from "../../../../../matrix/room/timeline/entries/EventEntry"; import type {EventEntry} from "../../../../../matrix/room/timeline/entries/EventEntry";
@ -38,6 +40,7 @@ import type {Options as ViewModelOptions} from "../../../../ViewModel";
export type TimelineEntry = FragmentBoundaryEntry | EventEntry | PendingEventEntry; export type TimelineEntry = FragmentBoundaryEntry | EventEntry | PendingEventEntry;
export type TileClassForEntryFn = (entry: TimelineEntry) => TileConstructor | undefined; export type TileClassForEntryFn = (entry: TimelineEntry) => TileConstructor | undefined;
export type Options = ViewModelOptions & { export type Options = ViewModelOptions & {
session: Session,
room: Room, room: Room,
timeline: Timeline timeline: Timeline
tileClassForEntry: TileClassForEntryFn; tileClassForEntry: TileClassForEntryFn;
@ -86,6 +89,14 @@ export function tileClassForEntry(entry: TimelineEntry): TileConstructor | undef
return EncryptedEventTile; return EncryptedEventTile;
case "m.room.encryption": case "m.room.encryption":
return EncryptionEnabledTile; return EncryptionEnabledTile;
case "org.matrix.msc3401.call": {
// if prevContent is present, it's an update to a call event, which we don't render
// as the original event is updated through the call object which receive state event updates
if (entry.stateKey && !entry.prevContent) {
return CallTile;
}
return undefined;
}
default: default:
// unknown type not rendered // unknown type not rendered
return undefined; return undefined;

View file

@ -17,6 +17,7 @@ limitations under the License.
import {ViewModel} from "../../ViewModel"; import {ViewModel} from "../../ViewModel";
import {KeyType} from "../../../matrix/ssss/index"; import {KeyType} from "../../../matrix/ssss/index";
import {createEnum} from "../../../utils/enum"; import {createEnum} from "../../../utils/enum";
import {FlatMapObservableValue} from "../../../observable/value/FlatMapObservableValue";
export const Status = createEnum("Enabled", "SetupKey", "SetupPhrase", "Pending", "NewVersionAvailable"); export const Status = createEnum("Enabled", "SetupKey", "SetupPhrase", "Pending", "NewVersionAvailable");
export const BackupWriteStatus = createEnum("Writing", "Stopped", "Done", "Pending"); export const BackupWriteStatus = createEnum("Writing", "Stopped", "Done", "Pending");
@ -29,8 +30,8 @@ export class KeyBackupViewModel extends ViewModel {
this._isBusy = false; this._isBusy = false;
this._dehydratedDeviceId = undefined; this._dehydratedDeviceId = undefined;
this._status = undefined; this._status = undefined;
this._backupOperation = this._session.keyBackup.flatMap(keyBackup => keyBackup.operationInProgress); this._backupOperation = new FlatMapObservableValue(this._session.keyBackup, keyBackup => keyBackup.operationInProgress);
this._progress = this._backupOperation.flatMap(op => op.progress); this._progress = new FlatMapObservableValue(this._backupOperation, op => op.progress);
this.track(this._backupOperation.subscribe(() => { this.track(this._backupOperation.subscribe(() => {
// see if needsNewKey might be set // see if needsNewKey might be set
this._reevaluateStatus(); this._reevaluateStatus();

View file

@ -150,8 +150,14 @@ export class SettingsViewModel extends ViewModel {
} }
async exportLogs() { async exportLogs() {
const logExport = await this.logger.export(); const logs = await this.exportLogsBlob();
this.platform.saveFileAs(logExport.asBlob(), `hydrogen-logs-${this.platform.clock.now()}.json`); this.platform.saveFileAs(logs, `hydrogen-logs-${this.platform.clock.now()}.json`);
}
async exportLogsBlob() {
const persister = this.logger.reporters.find(r => typeof r.export === "function");
const logExport = await persister.export();
return logExport.asBlob();
} }
get canSendLogsToServer() { get canSendLogsToServer() {

View file

@ -14,9 +14,22 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
export {Logger} from "./logging/Logger";
export type {ILogItem} from "./logging/types";
export {IDBLogPersister} from "./logging/IDBLogPersister";
export {ConsoleReporter} from "./logging/ConsoleReporter";
export {Platform} from "./platform/web/Platform.js"; export {Platform} from "./platform/web/Platform.js";
export {Client, LoadStatus} from "./matrix/Client.js"; export {Client, LoadStatus, LoginFailure} from "./matrix/Client.js";
export {RoomStatus} from "./matrix/room/common"; export {RoomStatus} from "./matrix/room/common";
export {AttachmentUpload} from "./matrix/room/AttachmentUpload";
export {CallIntent} from "./matrix/calls/callEventTypes";
// export everything needed to observe state events on all rooms using session.observeRoomState
export type {RoomStateHandler} from "./matrix/room/state/types";
export type {MemberChange} from "./matrix/room/members/RoomMember";
export type {Transaction} from "./matrix/storage/idb/Transaction";
export type {Room} from "./matrix/room/Room";
export type {StateEvent} from "./matrix/storage/types";
// export main view & view models // export main view & view models
export {createNavigation, createRouter} from "./domain/navigation/index.js"; export {createNavigation, createRouter} from "./domain/navigation/index.js";
export {RootViewModel} from "./domain/RootViewModel.js"; export {RootViewModel} from "./domain/RootViewModel.js";
@ -68,9 +81,10 @@ export {TemplateView} from "./platform/web/ui/general/TemplateView";
export {ViewModel} from "./domain/ViewModel"; export {ViewModel} from "./domain/ViewModel";
export {LoadingView} from "./platform/web/ui/general/LoadingView.js"; export {LoadingView} from "./platform/web/ui/general/LoadingView.js";
export {AvatarView} from "./platform/web/ui/AvatarView.js"; export {AvatarView} from "./platform/web/ui/AvatarView.js";
export {RoomType} from "./matrix/room/common"; export {RoomVisibility, RoomType} from "./matrix/room/common";
export {EventEmitter} from "./utils/EventEmitter"; export {EventEmitter} from "./utils/EventEmitter";
export {Disposables} from "./utils/Disposables"; export {Disposables} from "./utils/Disposables";
export {LocalMedia} from "./matrix/calls/LocalMedia";
// these should eventually be moved to another library // these should eventually be moved to another library
export { export {
ObservableArray, ObservableArray,
@ -80,8 +94,6 @@ export {
ConcatList, ConcatList,
ObservableMap ObservableMap
} from "./observable/index"; } from "./observable/index";
export { export {BaseObservableValue} from "./observable/value/BaseObservableValue";
BaseObservableValue, export {ObservableValue} from "./observable/value/ObservableValue";
ObservableValue, export {RetainedObservableValue} from "./observable/value/RetainedObservableValue";
RetainedObservableValue
} from "./observable/ObservableValue";

View file

@ -13,17 +13,28 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseLogger} from "./BaseLogger";
import {LogItem} from "./LogItem";
import type {ILogItem, LogItemValues, ILogExport} from "./types";
export class ConsoleLogger extends BaseLogger { import type {ILogger, ILogItem, LogItemValues, ILogReporter} from "./types";
_persistItem(item: LogItem): void { import type {LogItem} from "./LogItem";
printToConsole(item);
export class ConsoleReporter implements ILogReporter {
private logger?: ILogger;
reportItem(item: ILogItem): void {
printToConsole(item as LogItem);
} }
async export(): Promise<ILogExport | undefined> { setLogger(logger: ILogger) {
return undefined; this.logger = logger;
}
printOpenItems(): void {
if (!this.logger) {
return;
}
for (const item of this.logger.getOpenRootItems()) {
this.reportItem(item);
}
} }
} }
@ -39,7 +50,7 @@ function filterValues(values: LogItemValues): LogItemValues | null {
} }
function printToConsole(item: LogItem): void { function printToConsole(item: LogItem): void {
const label = `${itemCaption(item)} (${item.duration}ms)`; const label = `${itemCaption(item)} (@${item.start}ms, duration: ${item.duration}ms)`;
const filteredValues = filterValues(item.values); const filteredValues = filterValues(item.values);
const shouldGroup = item.children || filteredValues; const shouldGroup = item.children || filteredValues;
if (shouldGroup) { if (shouldGroup) {
@ -78,6 +89,8 @@ function itemCaption(item: ILogItem): string {
return `${item.values.l} ${item.values.id}`; return `${item.values.l} ${item.values.id}`;
} else if (item.values.l && typeof item.values.status !== "undefined") { } else if (item.values.l && typeof item.values.status !== "undefined") {
return `${item.values.l} (${item.values.status})`; return `${item.values.l} (${item.values.status})`;
} else if (item.values.l && typeof item.values.type !== "undefined") {
return `${item.values.l} (${item.values.type})`;
} else if (item.values.l && item.error) { } else if (item.values.l && item.error) {
return `${item.values.l} failed`; return `${item.values.l} failed`;
} else if (typeof item.values.ref !== "undefined") { } else if (typeof item.values.ref !== "undefined") {

View file

@ -22,36 +22,69 @@ import {
iterateCursor, iterateCursor,
fetchResults, fetchResults,
} from "../matrix/storage/idb/utils"; } from "../matrix/storage/idb/utils";
import {BaseLogger} from "./BaseLogger";
import type {Interval} from "../platform/web/dom/Clock"; import type {Interval} from "../platform/web/dom/Clock";
import type {Platform} from "../platform/web/Platform.js"; import type {Platform} from "../platform/web/Platform.js";
import type {BlobHandle} from "../platform/web/dom/BlobHandle.js"; import type {BlobHandle} from "../platform/web/dom/BlobHandle.js";
import type {ILogItem, ILogExport, ISerializedItem} from "./types"; import type {ILogItem, ILogger, ILogReporter, ISerializedItem} from "./types";
import type {LogFilter} from "./LogFilter"; import {LogFilter} from "./LogFilter";
type QueuedItem = { type QueuedItem = {
json: string; json: string;
id?: number; id?: number;
} }
export class IDBLogger extends BaseLogger { type Options = {
private readonly _name: string; name: string,
private readonly _limit: number; flushInterval?: number,
limit?: number,
platform: Platform,
serializedTransformer?: (item: ISerializedItem) => ISerializedItem
}
export class IDBLogPersister implements ILogReporter {
private readonly _flushInterval: Interval; private readonly _flushInterval: Interval;
private _queuedItems: QueuedItem[]; private _queuedItems: QueuedItem[];
private readonly options: Options;
private logger?: ILogger;
constructor(options: {name: string, flushInterval?: number, limit?: number, platform: Platform, serializedTransformer?: (item: ISerializedItem) => ISerializedItem}) { constructor(options: Options) {
super(options); this.options = options;
const {name, flushInterval = 60 * 1000, limit = 3000} = options;
this._name = name;
this._limit = limit;
this._queuedItems = this._loadQueuedItems(); this._queuedItems = this._loadQueuedItems();
// TODO: also listen for unload just in case sync keeps on running after pagehide is fired? // TODO: also listen for unload just in case sync keeps on running after pagehide is fired?
window.addEventListener("pagehide", this, false); window.addEventListener("pagehide", this, false);
this._flushInterval = this._platform.clock.createInterval(() => this._tryFlush(), flushInterval); this._flushInterval = this.options.platform.clock.createInterval(
() => this._tryFlush(),
this.options.flushInterval ?? 60 * 1000
);
}
setLogger(logger: ILogger): void {
this.logger = logger;
}
reportItem(logItem: ILogItem, filter: LogFilter, forced: boolean): void {
const queuedItem = this.prepareItemForQueue(logItem, filter, forced);
if (queuedItem) {
this._queuedItems.push(queuedItem);
}
}
async export(): Promise<IDBLogExport> {
const db = await this._openDB();
try {
const txn = db.transaction(["logs"], "readonly");
const logs = txn.objectStore("logs");
const storedItems: QueuedItem[] = await fetchResults(logs.openCursor(), () => false);
const openItems = this.getSerializedOpenItems();
const allItems = storedItems.concat(this._queuedItems).concat(openItems);
return new IDBLogExport(allItems, this, this.options.platform);
} finally {
try {
db.close();
} catch (e) {}
}
} }
// TODO: move dispose to ILogger, listen to pagehide elsewhere and call dispose from there, which calls _finishAllAndFlush
dispose(): void { dispose(): void {
window.removeEventListener("pagehide", this, false); window.removeEventListener("pagehide", this, false);
this._flushInterval.dispose(); this._flushInterval.dispose();
@ -63,7 +96,7 @@ export class IDBLogger extends BaseLogger {
} }
} }
async _tryFlush(): Promise<void> { private async _tryFlush(): Promise<void> {
const db = await this._openDB(); const db = await this._openDB();
try { try {
const txn = db.transaction(["logs"], "readwrite"); const txn = db.transaction(["logs"], "readwrite");
@ -73,9 +106,10 @@ export class IDBLogger extends BaseLogger {
logs.add(i); logs.add(i);
} }
const itemCount = await reqAsPromise(logs.count()); const itemCount = await reqAsPromise(logs.count());
if (itemCount > this._limit) { const limit = this.options.limit ?? 3000;
if (itemCount > limit) {
// delete an extra 10% so we don't need to delete every time we flush // delete an extra 10% so we don't need to delete every time we flush
let deleteAmount = (itemCount - this._limit) + Math.round(0.1 * this._limit); let deleteAmount = (itemCount - limit) + Math.round(0.1 * limit);
await iterateCursor(logs.openCursor(), (_, __, cursor) => { await iterateCursor(logs.openCursor(), (_, __, cursor) => {
cursor.delete(); cursor.delete();
deleteAmount -= 1; deleteAmount -= 1;
@ -93,14 +127,16 @@ export class IDBLogger extends BaseLogger {
} }
} }
_finishAllAndFlush(): void { private _finishAllAndFlush(): void {
this._finishOpenItems(); if (this.logger) {
this.log({l: "pagehide, closing logs", t: "navigation"}); this.logger.log({l: "pagehide, closing logs", t: "navigation"});
this.logger.forceFinish();
}
this._persistQueuedItems(this._queuedItems); this._persistQueuedItems(this._queuedItems);
} }
_loadQueuedItems(): QueuedItem[] { private _loadQueuedItems(): QueuedItem[] {
const key = `${this._name}_queuedItems`; const key = `${this.options.name}_queuedItems`;
try { try {
const json = window.localStorage.getItem(key); const json = window.localStorage.getItem(key);
if (json) { if (json) {
@ -113,44 +149,32 @@ export class IDBLogger extends BaseLogger {
return []; return [];
} }
_openDB(): Promise<IDBDatabase> { private _openDB(): Promise<IDBDatabase> {
return openDatabase(this._name, db => db.createObjectStore("logs", {keyPath: "id", autoIncrement: true}), 1); return openDatabase(this.options.name, db => db.createObjectStore("logs", {keyPath: "id", autoIncrement: true}), 1);
} }
_persistItem(logItem: ILogItem, filter: LogFilter, forced: boolean): void { private prepareItemForQueue(logItem: ILogItem, filter: LogFilter, forced: boolean): QueuedItem | undefined {
const serializedItem = logItem.serialize(filter, undefined, forced); let serializedItem = logItem.serialize(filter, undefined, forced);
if (serializedItem) { if (serializedItem) {
const transformedSerializedItem = this._serializedTransformer(serializedItem); if (this.options.serializedTransformer) {
this._queuedItems.push({ serializedItem = this.options.serializedTransformer(serializedItem);
json: JSON.stringify(transformedSerializedItem) }
}); return {
json: JSON.stringify(serializedItem)
};
} }
} }
_persistQueuedItems(items: QueuedItem[]): void { private _persistQueuedItems(items: QueuedItem[]): void {
try { try {
window.localStorage.setItem(`${this._name}_queuedItems`, JSON.stringify(items)); window.localStorage.setItem(`${this.options.name}_queuedItems`, JSON.stringify(items));
} catch (e) { } catch (e) {
console.error("Could not persist queued log items in localStorage, they will likely be lost", e); console.error("Could not persist queued log items in localStorage, they will likely be lost", e);
} }
} }
async export(): Promise<ILogExport> { /** @internal called by ILogExport.removeFromStore */
const db = await this._openDB(); async removeItems(items: QueuedItem[]): Promise<void> {
try {
const txn = db.transaction(["logs"], "readonly");
const logs = txn.objectStore("logs");
const storedItems: QueuedItem[] = await fetchResults(logs.openCursor(), () => false);
const allItems = storedItems.concat(this._queuedItems);
return new IDBLogExport(allItems, this, this._platform);
} finally {
try {
db.close();
} catch (e) {}
}
}
async _removeItems(items: QueuedItem[]): Promise<void> {
const db = await this._openDB(); const db = await this._openDB();
try { try {
const txn = db.transaction(["logs"], "readwrite"); const txn = db.transaction(["logs"], "readwrite");
@ -173,14 +197,29 @@ export class IDBLogger extends BaseLogger {
} catch (e) {} } catch (e) {}
} }
} }
private getSerializedOpenItems(): QueuedItem[] {
const openItems: QueuedItem[] = [];
if (!this.logger) {
return openItems;
}
const filter = new LogFilter();
for(const item of this.logger!.getOpenRootItems()) {
const openItem = this.prepareItemForQueue(item, filter, false);
if (openItem) {
openItems.push(openItem);
}
}
return openItems;
}
} }
class IDBLogExport implements ILogExport { export class IDBLogExport {
private readonly _items: QueuedItem[]; private readonly _items: QueuedItem[];
private readonly _logger: IDBLogger; private readonly _logger: IDBLogPersister;
private readonly _platform: Platform; private readonly _platform: Platform;
constructor(items: QueuedItem[], logger: IDBLogger, platform: Platform) { constructor(items: QueuedItem[], logger: IDBLogPersister, platform: Platform) {
this._items = items; this._items = items;
this._logger = logger; this._logger = logger;
this._platform = platform; this._platform = platform;
@ -194,18 +233,23 @@ class IDBLogExport implements ILogExport {
* @return {Promise} * @return {Promise}
*/ */
removeFromStore(): Promise<void> { removeFromStore(): Promise<void> {
return this._logger._removeItems(this._items); return this._logger.removeItems(this._items);
} }
asBlob(): BlobHandle { asBlob(): BlobHandle {
const json = this.toJSON();
const buffer: Uint8Array = this._platform.encoding.utf8.encode(json);
const blob: BlobHandle = this._platform.createBlob(buffer, "application/json");
return blob;
}
toJSON(): string {
const log = { const log = {
formatVersion: 1, formatVersion: 1,
appVersion: this._platform.updateService?.version, appVersion: this._platform.updateService?.version,
items: this._items.map(i => JSON.parse(i.json)) items: this._items.map(i => JSON.parse(i.json))
}; };
const json = JSON.stringify(log); const json = JSON.stringify(log);
const buffer: Uint8Array = this._platform.encoding.utf8.encode(json); return json;
const blob: BlobHandle = this._platform.createBlob(buffer, "application/json");
return blob;
} }
} }

View file

@ -16,7 +16,7 @@ limitations under the License.
*/ */
import {LogLevel, LogFilter} from "./LogFilter"; import {LogLevel, LogFilter} from "./LogFilter";
import type {BaseLogger} from "./BaseLogger"; import type {Logger} from "./Logger";
import type {ISerializedItem, ILogItem, LogItemValues, LabelOrValues, FilterCreator, LogCallback} from "./types"; import type {ISerializedItem, ILogItem, LogItemValues, LabelOrValues, FilterCreator, LogCallback} from "./types";
export class LogItem implements ILogItem { export class LogItem implements ILogItem {
@ -25,11 +25,11 @@ export class LogItem implements ILogItem {
public error?: Error; public error?: Error;
public end?: number; public end?: number;
private _values: LogItemValues; private _values: LogItemValues;
private _logger: BaseLogger; protected _logger: Logger;
private _filterCreator?: FilterCreator; private _filterCreator?: FilterCreator;
private _children?: Array<LogItem>; private _children?: Array<LogItem>;
constructor(labelOrValues: LabelOrValues, logLevel: LogLevel, logger: BaseLogger, filterCreator?: FilterCreator) { constructor(labelOrValues: LabelOrValues, logLevel: LogLevel, logger: Logger, filterCreator?: FilterCreator) {
this._logger = logger; this._logger = logger;
this.start = logger._now(); this.start = logger._now();
// (l)abel // (l)abel
@ -38,7 +38,7 @@ export class LogItem implements ILogItem {
this._filterCreator = filterCreator; this._filterCreator = filterCreator;
} }
/** start a new root log item and run it detached mode, see BaseLogger.runDetached */ /** start a new root log item and run it detached mode, see Logger.runDetached */
runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem { runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem {
return this._logger.runDetached(labelOrValues, callback, logLevel, filterCreator); return this._logger.runDetached(labelOrValues, callback, logLevel, filterCreator);
} }
@ -221,6 +221,11 @@ export class LogItem implements ILogItem {
} }
} }
/** @internal */
forceFinish(): void {
this.finish();
}
// expose log level without needing import everywhere // expose log level without needing import everywhere
get level(): typeof LogLevel { get level(): typeof LogLevel {
return LogLevel; return LogLevel;
@ -235,7 +240,7 @@ export class LogItem implements ILogItem {
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): LogItem { child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): LogItem {
if (this.end) { if (this.end) {
console.trace("log item is finished, additional logs will likely not be recorded"); console.trace(`log item ${this.values.l} finished, additional log ${JSON.stringify(labelOrValues)} will likely not be recorded`);
} }
if (!logLevel) { if (!logLevel) {
logLevel = this.logLevel || LogLevel.Info; logLevel = this.logLevel || LogLevel.Info;
@ -248,7 +253,7 @@ export class LogItem implements ILogItem {
return item; return item;
} }
get logger(): BaseLogger { get logger(): Logger {
return this._logger; return this._logger;
} }

View file

@ -17,17 +17,17 @@ limitations under the License.
import {LogItem} from "./LogItem"; import {LogItem} from "./LogItem";
import {LogLevel, LogFilter} from "./LogFilter"; import {LogLevel, LogFilter} from "./LogFilter";
import type {ILogger, ILogExport, FilterCreator, LabelOrValues, LogCallback, ILogItem, ISerializedItem} from "./types"; import type {ILogger, ILogReporter, FilterCreator, LabelOrValues, LogCallback, ILogItem, ISerializedItem} from "./types";
import type {Platform} from "../platform/web/Platform.js"; import type {Platform} from "../platform/web/Platform.js";
export abstract class BaseLogger implements ILogger { export class Logger implements ILogger {
protected _openItems: Set<LogItem> = new Set(); protected _openItems: Set<LogItem> = new Set();
protected _platform: Platform; protected _platform: Platform;
protected _serializedTransformer: (item: ISerializedItem) => ISerializedItem; protected _serializedTransformer: (item: ISerializedItem) => ISerializedItem;
public readonly reporters: ILogReporter[] = [];
constructor({platform, serializedTransformer = (item: ISerializedItem) => item}) { constructor({platform}) {
this._platform = platform; this._platform = platform;
this._serializedTransformer = serializedTransformer;
} }
log(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info): void { log(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info): void {
@ -36,6 +36,15 @@ export abstract class BaseLogger implements ILogger {
this._persistItem(item, undefined, false); this._persistItem(item, undefined, false);
} }
/** Prefer `run()` or `log()` above this method; only use it if you have a long-running operation
* *without* a single call stack that should be logged into one sub-tree.
* You need to call `finish()` on the returned item or it will stay open until the app unloads. */
child(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info, filterCreator?: FilterCreator): ILogItem {
const item = new DeferredPersistRootLogItem(labelOrValues, logLevel, this, filterCreator);
this._openItems.add(item);
return item;
}
/** if item is a log item, wrap the callback in a child of it, otherwise start a new root log item. */ /** if item is a log item, wrap the callback in a child of it, otherwise start a new root log item. */
wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T { wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T {
if (item) { if (item) {
@ -70,10 +79,10 @@ export abstract class BaseLogger implements ILogger {
return this._run(item, callback, logLevel, true, filterCreator); return this._run(item, callback, logLevel, true, filterCreator);
} }
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: true, filterCreator?: FilterCreator): T; private _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: true, filterCreator?: FilterCreator): T;
// we don't return if we don't throw, as we don't have anything to return when an error is caught but swallowed for the fire-and-forget case. // we don't return if we don't throw, as we don't have anything to return when an error is caught but swallowed for the fire-and-forget case.
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: false, filterCreator?: FilterCreator): void; private _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: false, filterCreator?: FilterCreator): void;
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: boolean, filterCreator?: FilterCreator): T | void { private _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: boolean, filterCreator?: FilterCreator): T | void {
this._openItems.add(item); this._openItems.add(item);
const finishItem = () => { const finishItem = () => {
@ -125,9 +134,18 @@ export abstract class BaseLogger implements ILogger {
} }
} }
_finishOpenItems() { addReporter(reporter: ILogReporter): void {
reporter.setLogger(this);
this.reporters.push(reporter);
}
getOpenRootItems(): Iterable<ILogItem> {
return this._openItems;
}
forceFinish() {
for (const openItem of this._openItems) { for (const openItem of this._openItems) {
openItem.finish(); openItem.forceFinish();
try { try {
// for now, serialize with an all-permitting filter // for now, serialize with an all-permitting filter
// as the createFilter function would get a distorted image anyway // as the createFilter function would get a distorted image anyway
@ -141,20 +159,43 @@ export abstract class BaseLogger implements ILogger {
this._openItems.clear(); this._openItems.clear();
} }
abstract _persistItem(item: LogItem, filter?: LogFilter, forced?: boolean): void; /** @internal */
_removeItemFromOpenList(item: LogItem): void {
this._openItems.delete(item);
}
abstract export(): Promise<ILogExport | undefined>; /** @internal */
_persistItem(item: LogItem, filter?: LogFilter, forced?: boolean): void {
for (var i = 0; i < this.reporters.length; i += 1) {
this.reporters[i].reportItem(item, filter, forced);
}
}
// expose log level without needing // expose log level without needing
get level(): typeof LogLevel { get level(): typeof LogLevel {
return LogLevel; return LogLevel;
} }
/** @internal */
_now(): number { _now(): number {
return this._platform.clock.now(); return this._platform.clock.now();
} }
/** @internal */
_createRefId(): number { _createRefId(): number {
return Math.round(this._platform.random() * Number.MAX_SAFE_INTEGER); return Math.round(this._platform.random() * Number.MAX_SAFE_INTEGER);
} }
} }
class DeferredPersistRootLogItem extends LogItem {
finish() {
super.finish();
(this._logger as Logger)._persistItem(this, undefined, false);
(this._logger as Logger)._removeItemFromOpenList(this);
}
forceFinish() {
super.finish();
/// no need to persist when force-finishing as _finishOpenItems above will do it
}
}

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {LogLevel} from "./LogFilter"; import {LogLevel} from "./LogFilter";
import type {ILogger, ILogExport, ILogItem, LabelOrValues, LogCallback, LogItemValues} from "./types"; import type {ILogger, ILogItem, ILogReporter, LabelOrValues, LogCallback, LogItemValues} from "./types";
function noop (): void {} function noop (): void {}
@ -23,6 +23,22 @@ export class NullLogger implements ILogger {
log(): void {} log(): void {}
addReporter() {}
get reporters(): ReadonlyArray<ILogReporter> {
return [];
}
getOpenRootItems(): Iterable<ILogItem> {
return [];
}
forceFinish(): void {}
child(): ILogItem {
return this.item;
}
run<T>(_, callback: LogCallback<T>): T { run<T>(_, callback: LogCallback<T>): T {
return callback(this.item); return callback(this.item);
} }
@ -40,10 +56,6 @@ export class NullLogger implements ILogger {
return this.item; return this.item;
} }
async export(): Promise<ILogExport | undefined> {
return undefined;
}
get level(): typeof LogLevel { get level(): typeof LogLevel {
return LogLevel; return LogLevel;
} }
@ -61,12 +73,18 @@ export class NullLogItem implements ILogItem {
} }
wrap<T>(_: LabelOrValues, callback: LogCallback<T>): T { wrap<T>(_: LabelOrValues, callback: LogCallback<T>): T {
return this.run(callback);
}
run<T>(callback: LogCallback<T>): T {
return callback(this); return callback(this);
} }
log(): ILogItem { log(): ILogItem {
return this; return this;
} }
set(): ILogItem { return this; } set(): ILogItem { return this; }
runDetached(_: LabelOrValues, callback: LogCallback<unknown>): ILogItem { runDetached(_: LabelOrValues, callback: LogCallback<unknown>): ILogItem {
@ -99,6 +117,7 @@ export class NullLogItem implements ILogItem {
} }
finish(): void {} finish(): void {}
forceFinish(): void {}
serialize(): undefined { serialize(): undefined {
return undefined; return undefined;

View file

@ -16,7 +16,6 @@ limitations under the License.
*/ */
import {LogLevel, LogFilter} from "./LogFilter"; import {LogLevel, LogFilter} from "./LogFilter";
import type {BaseLogger} from "./BaseLogger";
import type {BlobHandle} from "../platform/web/dom/BlobHandle.js"; import type {BlobHandle} from "../platform/web/dom/BlobHandle.js";
export interface ISerializedItem { export interface ISerializedItem {
@ -40,8 +39,10 @@ export interface ILogItem {
readonly level: typeof LogLevel; readonly level: typeof LogLevel;
readonly end?: number; readonly end?: number;
readonly start?: number; readonly start?: number;
readonly values: LogItemValues; readonly values: Readonly<LogItemValues>;
wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T; wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
/*** This is sort of low-level, you probably want to use wrap. If you do use it, it should only be called once. */
run<T>(callback: LogCallback<T>): T;
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem; log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem;
set(key: string | object, value: unknown): ILogItem; set(key: string | object, value: unknown): ILogItem;
runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem; runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
@ -51,22 +52,41 @@ export interface ILogItem {
catch(err: Error): Error; catch(err: Error): Error;
serialize(filter: LogFilter, parentStartTime: number | undefined, forced: boolean): ISerializedItem | undefined; serialize(filter: LogFilter, parentStartTime: number | undefined, forced: boolean): ISerializedItem | undefined;
finish(): void; finish(): void;
forceFinish(): void;
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem; child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
} }
/*
extend both ILogger and ILogItem from this interface, but need to rename ILogger.run => wrap then. Or both to `span`?
export interface ILogItemCreator {
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
refDetached(logItem: ILogItem, logLevel?: LogLevel): void;
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem;
wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
get level(): typeof LogLevel;
}
*/
export interface ILogger { export interface ILogger {
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): void; log(labelOrValues: LabelOrValues, logLevel?: LogLevel): void;
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T; wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
runDetached<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem; runDetached<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
run<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T; run<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
export(): Promise<ILogExport | undefined>;
get level(): typeof LogLevel; get level(): typeof LogLevel;
getOpenRootItems(): Iterable<ILogItem>;
addReporter(reporter: ILogReporter): void;
get reporters(): ReadonlyArray<ILogReporter>;
/**
* force-finishes any open items and passes them to the reporter, with the forced flag set.
* Good think to do when the page is being closed to not lose any logs.
**/
forceFinish(): void;
} }
export interface ILogExport { export interface ILogReporter {
get count(): number; setLogger(logger: ILogger): void;
removeFromStore(): Promise<void>; reportItem(item: ILogItem, filter?: LogFilter, forced?: boolean): void;
asBlob(): BlobHandle;
} }
export type LogItemValues = { export type LogItemValues = {

View file

@ -18,7 +18,7 @@ limitations under the License.
import {createEnum} from "../utils/enum"; import {createEnum} from "../utils/enum";
import {lookupHomeserver} from "./well-known.js"; import {lookupHomeserver} from "./well-known.js";
import {AbortableOperation} from "../utils/AbortableOperation"; import {AbortableOperation} from "../utils/AbortableOperation";
import {ObservableValue} from "../observable/ObservableValue"; import {ObservableValue} from "../observable/value/ObservableValue";
import {HomeServerApi} from "./net/HomeServerApi"; import {HomeServerApi} from "./net/HomeServerApi";
import {Reconnector, ConnectionStatus} from "./net/Reconnector"; import {Reconnector, ConnectionStatus} from "./net/Reconnector";
import {ExponentialRetryDelay} from "./net/ExponentialRetryDelay"; import {ExponentialRetryDelay} from "./net/ExponentialRetryDelay";

View file

@ -16,12 +16,15 @@ limitations under the License.
import {OLM_ALGORITHM} from "./e2ee/common.js"; import {OLM_ALGORITHM} from "./e2ee/common.js";
import {countBy, groupBy} from "../utils/groupBy"; import {countBy, groupBy} from "../utils/groupBy";
import {LRUCache} from "../utils/LRUCache";
export class DeviceMessageHandler { export class DeviceMessageHandler {
constructor({storage}) { constructor({storage, callHandler}) {
this._storage = storage; this._storage = storage;
this._olmDecryption = null; this._olmDecryption = null;
this._megolmDecryption = null; this._megolmDecryption = null;
this._callHandler = callHandler;
this._senderDeviceCache = new LRUCache(10, di => di.curve25519Key);
} }
enableEncryption({olmDecryption, megolmDecryption}) { enableEncryption({olmDecryption, megolmDecryption}) {
@ -49,6 +52,11 @@ export class DeviceMessageHandler {
log.child("decrypt_error").catch(err); log.child("decrypt_error").catch(err);
} }
const newRoomKeys = this._megolmDecryption.roomKeysFromDeviceMessages(olmDecryptChanges.results, log); const newRoomKeys = this._megolmDecryption.roomKeysFromDeviceMessages(olmDecryptChanges.results, log);
// TODO: somehow include rooms that received a call to_device message in the sync state?
// or have updates flow through event emitter?
// well, we don't really need to update the room other then when a call starts or stops
// any changes within the call will be emitted on the call object?
return new SyncPreparation(olmDecryptChanges, newRoomKeys); return new SyncPreparation(olmDecryptChanges, newRoomKeys);
} }
} }
@ -58,7 +66,38 @@ export class DeviceMessageHandler {
// write olm changes // write olm changes
prep.olmDecryptChanges.write(txn); prep.olmDecryptChanges.write(txn);
const didWriteValues = await Promise.all(prep.newRoomKeys.map(key => this._megolmDecryption.writeRoomKey(key, txn))); const didWriteValues = await Promise.all(prep.newRoomKeys.map(key => this._megolmDecryption.writeRoomKey(key, txn)));
return didWriteValues.some(didWrite => !!didWrite); const hasNewRoomKeys = didWriteValues.some(didWrite => !!didWrite);
return {
hasNewRoomKeys,
decryptionResults: prep.olmDecryptChanges.results
};
}
async afterSyncCompleted(decryptionResults, deviceTracker, hsApi, log) {
// if we don't have a device, we need to fetch the device keys the message claims
// and check the keys, and we should only do network requests during
// sync processing in the afterSyncCompleted step.
const callMessages = decryptionResults.filter(dr => this._callHandler.handlesDeviceMessageEventType(dr.event?.type));
if (callMessages.length) {
await log.wrap("process call signalling messages", async log => {
for (const dr of callMessages) {
// serialize device loading, so subsequent messages for the same device take advantage of the cache
const device = await deviceTracker.deviceForId(dr.event.sender, dr.event.content.device_id, hsApi, log);
dr.setDevice(device);
if (dr.isVerified) {
this._callHandler.handleDeviceMessage(dr.event, dr.userId, dr.deviceId, log);
} else {
log.log({
l: "could not verify olm fingerprint key matches, ignoring",
ed25519Key: dr.device.ed25519Key,
claimedEd25519Key: dr.claimedEd25519Key,
deviceId: device.deviceId,
userId: device.userId,
});
}
}
});
}
} }
} }

View file

@ -21,7 +21,7 @@ import {RoomStatus} from "./room/common";
import {RoomBeingCreated} from "./room/RoomBeingCreated"; import {RoomBeingCreated} from "./room/RoomBeingCreated";
import {Invite} from "./room/Invite.js"; import {Invite} from "./room/Invite.js";
import {Pusher} from "./push/Pusher"; import {Pusher} from "./push/Pusher";
import { ObservableMap } from "../observable/index.js"; import { ObservableMap } from "../observable/index";
import {User} from "./User.js"; import {User} from "./User.js";
import {DeviceMessageHandler} from "./DeviceMessageHandler.js"; import {DeviceMessageHandler} from "./DeviceMessageHandler.js";
import {Account as E2EEAccount} from "./e2ee/Account.js"; import {Account as E2EEAccount} from "./e2ee/Account.js";
@ -45,7 +45,10 @@ import {
keyFromDehydratedDeviceKey as createSSSSKeyFromDehydratedDeviceKey keyFromDehydratedDeviceKey as createSSSSKeyFromDehydratedDeviceKey
} from "./ssss/index"; } from "./ssss/index";
import {SecretStorage} from "./ssss/SecretStorage"; import {SecretStorage} from "./ssss/SecretStorage";
import {ObservableValue, RetainedObservableValue} from "../observable/ObservableValue"; import {ObservableValue} from "../observable/value/ObservableValue";
import {RetainedObservableValue} from "../observable/value/RetainedObservableValue";
import {CallHandler} from "./calls/CallHandler";
import {RoomStateHandlerSet} from "./room/state/RoomStateHandlerSet";
const PICKLE_KEY = "DEFAULT_KEY"; const PICKLE_KEY = "DEFAULT_KEY";
const PUSHER_KEY = "pusher"; const PUSHER_KEY = "pusher";
@ -73,7 +76,39 @@ export class Session {
}; };
this._roomsBeingCreated = new ObservableMap(); this._roomsBeingCreated = new ObservableMap();
this._user = new User(sessionInfo.userId); this._user = new User(sessionInfo.userId);
this._deviceMessageHandler = new DeviceMessageHandler({storage}); this._callHandler = new CallHandler({
clock: this._platform.clock,
hsApi: this._hsApi,
encryptDeviceMessage: async (roomId, userId, deviceId, message, log) => {
if (!this._deviceTracker || !this._olmEncryption) {
log.set("encryption_disabled", true);
return;
}
const device = await log.wrap("get device key", async log => {
const device = this._deviceTracker.deviceForId(userId, deviceId, this._hsApi, log);
if (!device) {
log.set("not_found", true);
}
return device;
});
if (device) {
const encryptedMessages = await this._olmEncryption.encrypt(message.type, message.content, [device], this._hsApi, log);
return encryptedMessages;
}
},
storage: this._storage,
webRTC: this._platform.webRTC,
ownDeviceId: sessionInfo.deviceId,
ownUserId: sessionInfo.userId,
logger: this._platform.logger,
turnServers: [{
urls: ["stun:turn.matrix.org"],
}],
forceTURN: false,
});
this._roomStateHandler = new RoomStateHandlerSet();
this.observeRoomState(this._callHandler);
this._deviceMessageHandler = new DeviceMessageHandler({storage, callHandler: this._callHandler});
this._olm = olm; this._olm = olm;
this._olmUtil = null; this._olmUtil = null;
this._e2eeAccount = null; this._e2eeAccount = null;
@ -102,6 +137,14 @@ export class Session {
this.needsKeyBackup = new ObservableValue(false); this.needsKeyBackup = new ObservableValue(false);
} }
get hsApi() {
return this._hsApi;
}
get sessionInfo() {
return this._sessionInfo;
}
get fingerprintKey() { get fingerprintKey() {
return this._e2eeAccount?.identityKeys.ed25519; return this._e2eeAccount?.identityKeys.ed25519;
} }
@ -118,6 +161,10 @@ export class Session {
return this._sessionInfo.userId; return this._sessionInfo.userId;
} }
get callHandler() {
return this._callHandler;
}
// called once this._e2eeAccount is assigned // called once this._e2eeAccount is assigned
_setupEncryption() { _setupEncryption() {
// TODO: this should all go in a wrapper in e2ee/ that is bootstrapped by passing in the account // TODO: this should all go in a wrapper in e2ee/ that is bootstrapped by passing in the account
@ -562,7 +609,8 @@ export class Session {
pendingEvents, pendingEvents,
user: this._user, user: this._user,
createRoomEncryption: this._createRoomEncryption, createRoomEncryption: this._createRoomEncryption,
platform: this._platform platform: this._platform,
roomStateHandler: this._roomStateHandler
}); });
} }
@ -649,7 +697,9 @@ export class Session {
async writeSync(syncResponse, syncFilterId, preparation, txn, log) { async writeSync(syncResponse, syncFilterId, preparation, txn, log) {
const changes = { const changes = {
syncInfo: null, syncInfo: null,
e2eeAccountChanges: null e2eeAccountChanges: null,
hasNewRoomKeys: false,
deviceMessageDecryptionResults: null,
}; };
const syncToken = syncResponse.next_batch; const syncToken = syncResponse.next_batch;
if (syncToken !== this.syncToken) { if (syncToken !== this.syncToken) {
@ -670,7 +720,9 @@ export class Session {
} }
if (preparation) { if (preparation) {
changes.hasNewRoomKeys = await log.wrap("deviceMsgs", log => this._deviceMessageHandler.writeSync(preparation, txn, log)); const {hasNewRoomKeys, decryptionResults} = await log.wrap("deviceMsgs", log => this._deviceMessageHandler.writeSync(preparation, txn, log));
changes.hasNewRoomKeys = hasNewRoomKeys;
changes.deviceMessageDecryptionResults = decryptionResults;
} }
// store account data // store account data
@ -711,6 +763,9 @@ export class Session {
if (changes.hasNewRoomKeys) { if (changes.hasNewRoomKeys) {
this._keyBackup.get()?.flush(log); this._keyBackup.get()?.flush(log);
} }
if (changes.deviceMessageDecryptionResults) {
await this._deviceMessageHandler.afterSyncCompleted(changes.deviceMessageDecryptionResults, this._deviceTracker, this._hsApi, log);
}
} }
_tryReplaceRoomBeingCreated(roomId, log) { _tryReplaceRoomBeingCreated(roomId, log) {
@ -904,6 +959,10 @@ export class Session {
return observable; return observable;
} }
observeRoomState(roomStateHandler) {
return this._roomStateHandler.subscribe(roomStateHandler);
}
/** /**
Creates an empty (summary isn't loaded) the archived room if it isn't Creates an empty (summary isn't loaded) the archived room if it isn't
loaded already, assuming sync will either remove it (when rejoining) or loaded already, assuming sync will either remove it (when rejoining) or
@ -983,9 +1042,18 @@ export function tests() {
return { return {
"session data is not modified until after sync": async (assert) => { "session data is not modified until after sync": async (assert) => {
const session = new Session({storage: createStorageMock({ const storage = createStorageMock({
sync: {token: "a", filterId: 5} sync: {token: "a", filterId: 5}
}), sessionInfo: {userId: ""}}); });
const session = new Session({
storage,
sessionInfo: {userId: ""},
platform: {
clock: {
createTimeout: () => undefined
}
}
});
await session.load(); await session.load();
let syncSet = false; let syncSet = false;
const syncTxn = { const syncTxn = {

View file

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {ObservableValue} from "../observable/ObservableValue"; import {ObservableValue} from "../observable/value/ObservableValue";
import {createEnum} from "../utils/enum"; import {createEnum} from "../utils/enum";
const INCREMENTAL_TIMEOUT = 30000; const INCREMENTAL_TIMEOUT = 30000;
@ -160,7 +160,7 @@ export class Sync {
const isCatchupSync = this._status.get() === SyncStatus.CatchupSync; const isCatchupSync = this._status.get() === SyncStatus.CatchupSync;
const sessionPromise = (async () => { const sessionPromise = (async () => {
try { try {
await log.wrap("session", log => this._session.afterSyncCompleted(sessionChanges, isCatchupSync, log), log.level.Detail); await log.wrap("session", log => this._session.afterSyncCompleted(sessionChanges, isCatchupSync, log));
} catch (err) {} // error is logged, but don't fail sessionPromise } catch (err) {} // error is logged, but don't fail sessionPromise
})(); })();
@ -224,6 +224,7 @@ export class Sync {
_openPrepareSyncTxn() { _openPrepareSyncTxn() {
const storeNames = this._storage.storeNames; const storeNames = this._storage.storeNames;
return this._storage.readTxn([ return this._storage.readTxn([
storeNames.deviceIdentities, // to read device from olm messages
storeNames.olmSessions, storeNames.olmSessions,
storeNames.inboundGroupSessions, storeNames.inboundGroupSessions,
// to read fragments when loading sync writer when rejoining archived room // to read fragments when loading sync writer when rejoining archived room
@ -343,6 +344,7 @@ export class Sync {
// to decrypt and store new room keys // to decrypt and store new room keys
storeNames.olmSessions, storeNames.olmSessions,
storeNames.inboundGroupSessions, storeNames.inboundGroupSessions,
storeNames.calls,
]); ]);
} }

View file

@ -0,0 +1,246 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableMap} from "../../observable/map/ObservableMap";
import {WebRTC, PeerConnection} from "../../platform/types/WebRTC";
import {MediaDevices, Track} from "../../platform/types/MediaDevices";
import {handlesEventType} from "./PeerCall";
import {EventType, CallIntent} from "./callEventTypes";
import {GroupCall} from "./group/GroupCall";
import {makeId} from "../common";
import {CALL_LOG_TYPE} from "./common";
import {EVENT_TYPE as MEMBER_EVENT_TYPE, RoomMember} from "../room/members/RoomMember";
import type {LocalMedia} from "./LocalMedia";
import type {Room} from "../room/Room";
import type {MemberChange} from "../room/members/RoomMember";
import type {StateEvent} from "../storage/types";
import type {ILogItem, ILogger} from "../../logging/types";
import type {Platform} from "../../platform/web/Platform";
import type {BaseObservableMap} from "../../observable/map/BaseObservableMap";
import type {SignallingMessage, MGroupCallBase} from "./callEventTypes";
import type {Options as GroupCallOptions} from "./group/GroupCall";
import type {Transaction} from "../storage/idb/Transaction";
import type {CallEntry} from "../storage/idb/stores/CallStore";
import type {Clock} from "../../platform/web/dom/Clock";
import type {RoomStateHandler} from "../room/state/types";
import type {MemberSync} from "../room/timeline/persistence/MemberWriter";
export type Options = Omit<GroupCallOptions, "emitUpdate" | "createTimeout"> & {
clock: Clock
};
function getRoomMemberKey(roomId: string, userId: string): string {
return JSON.stringify(roomId)+`,`+JSON.stringify(userId);
}
export class CallHandler implements RoomStateHandler {
// group calls by call id
private readonly _calls: ObservableMap<string, GroupCall> = new ObservableMap<string, GroupCall>();
// map of `"roomId","userId"` to set of conf_id's they are in
private roomMemberToCallIds: Map<string, Set<string>> = new Map();
private groupCallOptions: GroupCallOptions;
private sessionId = makeId("s");
constructor(private readonly options: Options) {
this.groupCallOptions = Object.assign({}, this.options, {
emitUpdate: (groupCall, params) => this._calls.update(groupCall.id, params),
createTimeout: this.options.clock.createTimeout,
sessionId: this.sessionId
});
}
async loadCalls(intent: CallIntent = CallIntent.Ring) {
const txn = await this._getLoadTxn();
const callEntries = await txn.calls.getByIntent(intent);
this._loadCallEntries(callEntries, txn);
}
async loadCallsForRoom(intent: CallIntent, roomId: string) {
const txn = await this._getLoadTxn();
const callEntries = await txn.calls.getByIntentAndRoom(intent, roomId);
this._loadCallEntries(callEntries, txn);
}
private async _getLoadTxn(): Promise<Transaction> {
const names = this.options.storage.storeNames;
const txn = await this.options.storage.readTxn([
names.calls,
names.roomState,
]);
return txn;
}
private async _loadCallEntries(callEntries: CallEntry[], txn: Transaction): Promise<void> {
return this.options.logger.run({l: "loading calls", t: CALL_LOG_TYPE}, async log => {
log.set("entries", callEntries.length);
await Promise.all(callEntries.map(async callEntry => {
if (this._calls.get(callEntry.callId)) {
return;
}
const event = await txn.roomState.get(callEntry.roomId, EventType.GroupCall, callEntry.callId);
if (event) {
const call = new GroupCall(event.event.state_key, false, event.event.content, event.roomId, this.groupCallOptions);
this._calls.set(call.id, call);
}
}));
const roomIds = Array.from(new Set(callEntries.map(e => e.roomId)));
await Promise.all(roomIds.map(async roomId => {
// TODO: don't load all members until we need them
const callsMemberEvents = await txn.roomState.getAllForType(roomId, EventType.GroupCallMember);
await Promise.all(callsMemberEvents.map(async entry => {
const userId = entry.event.sender;
const roomMemberState = await txn.roomState.get(roomId, MEMBER_EVENT_TYPE, userId);
let roomMember;
if (roomMemberState) {
roomMember = RoomMember.fromMemberEvent(roomMemberState.event);
}
if (!roomMember) {
// we'll be missing the member here if we received a call and it's members
// as pre-gap state and the members weren't active in the timeline we got.
roomMember = RoomMember.fromUserId(roomId, userId, "join");
}
this.handleCallMemberEvent(entry.event, roomMember, roomId, log);
}));
}));
log.set("newSize", this._calls.size);
});
}
async createCall(roomId: string, type: "m.video" | "m.voice", name: string, intent: CallIntent = CallIntent.Ring): Promise<GroupCall> {
const call = new GroupCall(makeId("conf-"), true, {
"m.name": name,
"m.intent": intent
}, roomId, this.groupCallOptions);
this._calls.set(call.id, call);
try {
await call.create(type);
// store call info so it will ring again when reopening the app
const txn = await this.options.storage.readWriteTxn([this.options.storage.storeNames.calls]);
txn.calls.add({
intent: call.intent,
callId: call.id,
timestamp: this.options.clock.now(),
roomId: roomId
});
await txn.complete();
} catch (err) {
//if (err.name === "ConnectionError") {
// if we're offline, give up and remove the call again
this._calls.remove(call.id);
//}
throw err;
}
return call;
}
get calls(): BaseObservableMap<string, GroupCall> { return this._calls; }
// TODO: check and poll turn server credentials here
/** @internal */
async handleRoomState(room: Room, event: StateEvent, memberSync: MemberSync, txn: Transaction, log: ILogItem) {
if (event.type === EventType.GroupCall) {
this.handleCallEvent(event, room.id, txn, log);
}
if (event.type === EventType.GroupCallMember) {
let member = await memberSync.lookupMemberAtEvent(event.sender, event, txn);
if (!member) {
// we'll be missing the member here if we received a call and it's members
// as pre-gap state and the members weren't active in the timeline we got.
member = RoomMember.fromUserId(room.id, event.sender, "join");
}
this.handleCallMemberEvent(event, member, room.id, log);
}
}
/** @internal */
updateRoomMembers(room: Room, memberChanges: Map<string, MemberChange>) {
// TODO: also have map for roomId to calls, so we can easily update members
// we will also need this to get the call for a room
for (const call of this._calls.values()) {
if (call.roomId === room.id) {
call.updateRoomMembers(memberChanges);
}
}
}
/** @internal */
handlesDeviceMessageEventType(eventType: string): boolean {
return handlesEventType(eventType);
}
/** @internal */
handleDeviceMessage(message: SignallingMessage<MGroupCallBase>, userId: string, deviceId: string, log: ILogItem) {
// TODO: buffer messages for calls we haven't received the state event for yet?
const call = this._calls.get(message.content.conf_id);
call?.handleDeviceMessage(message, userId, deviceId, log);
}
private handleCallEvent(event: StateEvent, roomId: string, txn: Transaction, log: ILogItem) {
const callId = event.state_key;
let call = this._calls.get(callId);
if (call) {
call.updateCallEvent(event.content, log);
if (call.isTerminated) {
call.disconnect(log);
this._calls.remove(call.id);
txn.calls.remove(call.intent, roomId, call.id);
}
} else {
call = new GroupCall(event.state_key, false, event.content, roomId, this.groupCallOptions);
this._calls.set(call.id, call);
txn.calls.add({
intent: call.intent,
callId: call.id,
timestamp: event.origin_server_ts,
roomId: roomId
});
}
}
private handleCallMemberEvent(event: StateEvent, member: RoomMember, roomId: string, log: ILogItem) {
const userId = event.state_key;
const roomMemberKey = getRoomMemberKey(roomId, userId)
const calls = event.content["m.calls"] ?? [];
const eventTimestamp = event.origin_server_ts;
for (const call of calls) {
const callId = call["m.call_id"];
const groupCall = this._calls.get(callId);
// TODO: also check the member when receiving the m.call event
groupCall?.updateMembership(userId, member, call, eventTimestamp, log);
};
const newCallIdsMemberOf = new Set<string>(calls.map(call => call["m.call_id"]));
let previousCallIdsMemberOf = this.roomMemberToCallIds.get(roomMemberKey);
// remove user as member of any calls not present anymore
if (previousCallIdsMemberOf) {
for (const previousCallId of previousCallIdsMemberOf) {
if (!newCallIdsMemberOf.has(previousCallId)) {
const groupCall = this._calls.get(previousCallId);
groupCall?.removeMembership(userId, log);
}
}
}
if (newCallIdsMemberOf.size === 0) {
this.roomMemberToCallIds.delete(roomMemberKey);
} else {
this.roomMemberToCallIds.set(roomMemberKey, newCallIdsMemberOf);
}
}
}

View file

@ -0,0 +1,68 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {SDPStreamMetadataPurpose} from "./callEventTypes";
import {Stream} from "../../platform/types/MediaDevices";
import {SDPStreamMetadata} from "./callEventTypes";
import {getStreamVideoTrack, getStreamAudioTrack} from "./common";
export class LocalMedia {
constructor(
public readonly userMedia?: Stream,
public readonly screenShare?: Stream,
public readonly dataChannelOptions?: RTCDataChannelInit,
) {}
withUserMedia(stream: Stream) {
return new LocalMedia(stream, this.screenShare, this.dataChannelOptions);
}
withScreenShare(stream: Stream) {
return new LocalMedia(this.userMedia, stream, this.dataChannelOptions);
}
withDataChannel(options: RTCDataChannelInit): LocalMedia {
return new LocalMedia(this.userMedia, this.screenShare, options);
}
/** @internal */
replaceClone(oldClone: LocalMedia | undefined, oldOriginal: LocalMedia | undefined): LocalMedia {
const cloneOrAdoptStream = (oldOriginalStream: Stream | undefined, oldCloneStream: Stream | undefined, newStream: Stream | undefined): Stream | undefined => {
let stream;
if (oldOriginalStream?.id === newStream?.id) {
return oldCloneStream;
} else {
return newStream?.clone();
}
}
return new LocalMedia(
cloneOrAdoptStream(oldOriginal?.userMedia, oldClone?.userMedia, this.userMedia),
cloneOrAdoptStream(oldOriginal?.screenShare, oldClone?.screenShare, this.screenShare),
this.dataChannelOptions
);
}
/** @internal */
clone(): LocalMedia {
return new LocalMedia(this.userMedia?.clone(),this.screenShare?.clone(), this.dataChannelOptions);
}
dispose() {
getStreamAudioTrack(this.userMedia)?.stop();
getStreamVideoTrack(this.userMedia)?.stop();
getStreamVideoTrack(this.screenShare)?.stop();
}
}

1183
src/matrix/calls/PeerCall.ts Normal file

File diff suppressed because it is too large Load diff

225
src/matrix/calls/TODO.md Normal file
View file

@ -0,0 +1,225 @@
- relevant MSCs next to spec:
- https://github.com/matrix-org/matrix-doc/pull/2746 Improved Signalling for 1:1 VoIP
- https://github.com/matrix-org/matrix-doc/pull/2747 Transferring VoIP Calls
- https://github.com/matrix-org/matrix-doc/pull/3077 Support for multi-stream VoIP
- https://github.com/matrix-org/matrix-doc/pull/3086 Asserted identity on VoIP calls
- https://github.com/matrix-org/matrix-doc/pull/3291 Muting in VoIP calls
- https://github.com/matrix-org/matrix-doc/pull/3401 Native Group VoIP Signalling
## TODO
- DONE: implement receiving hangup
- DONE: implement cloning the localMedia so it works in safari?
- DONE: implement 3 retries per peer
- DONE: implement muting tracks with m.call.sdp_stream_metadata_changed
- DONE: implement renegotiation
- DONE: finish session id support
- call peers are essentially identified by (userid, deviceid, sessionid). If see a new session id, we first disconnect from the current member so we're ready to connect with a clean slate again (in a member event, also in to_device? no harm I suppose, given olm encryption ensures you can't spoof the deviceid).
- DONE: making logging better
- figure out why sometimes leave button does not work
- get correct members and avatars in call
- improve UI while in a call
- allow toggling audio
- support active speaker, sort speakers by last active
- close muted media stream after a while
- support highlight mode where we show active speaker and thumbnails for other participants
- better grid mode:
- we report the call view size to the view model with ResizeObserver, we calculate the A/R
- we calculate the grid based on view A/R, taking into account minimal stream size
- show name on stream view
- when you start a call, or join one, first you go to a SelectCallMedia screen where you can pick whether you want to use camera, audio or both:
- if you are joining a call, we'll default to the call intent
- if you are creating a call, we'll default to video
- when creating a call, adjust the navigation path to room/room_id/call
- when selecting a call, adjust the navigation path to room/room_id/call/call_id
- implement to_device messages arriving before m.call(.member) state event
- DONE for m.call.member, not for m.call and not for to_device other than m.call.invite arriving before invite
- reeable crypto & implement fetching olm keys before sending encrypted signalling message
- local echo for join/leave buttons?
- batch outgoing to_device messages in one request to homeserver for operations that will send out an event to all participants (e.g. mute)
- implement call ringing and rejecting a ringing call
- support screen sharing
- add button to enable, disable
- support showing stream view with large screen share video element and small camera video element (if present)
- don't load all members when loading calls to know whether they are ringing and joined by ourself
- only load our own member once, then have a way to load additional members on a call.
- see if we remove partyId entirely, it is only used for detecting remote echo which is not an issue for group calls? see https://github.com/matrix-org/matrix-spec-proposals/blob/dbkr/msc2746/proposals/2746-reliable-voip.md#add-party_id-to-all-voip-events
- remove PeerCall.waitForState ?
- invite glare is completely untested, does it work?
- how to remove call from m.call.member when just closing client?
- when closing client and still in call, tell service worker to send event on our behalf?
```js
// dispose when leaving call
this.track(platform.registerExitHandler(unloadActions => {
// batch requests will resolve immediately,
// so we can reuse the same send code that does awaits without awaiting?
const batch = new RequestBatch();
const hsApi = this.hsApi.withBatch(batch);
// _leaveCallMemberContent will need to become sync,
// so we'll need to keep track of own member event rather than rely on storage
hsApi.sendStateEvent("m.call.member", this._leaveCallMemberContent());
// does this internally: serviceWorkerHandler.trySend("sendRequestBatch", batch.toJSON());
unloadActions.sendRequestBatch(batch);
}));
```
## TODO (old)
- DONE: PeerCall
- send invite
- implement terminate
- implement waitForState
- find out if we need to do something different when renegotation is triggered (a subsequent onnegotiationneeded event) whether
we sent the invite/offer or answer. e.g. do we always do createOffer/setLocalDescription and then send it over a matrix negotiation event? even if we before called createAnswer.
- handle receiving offer and send anwser
- handle sending ice candidates
- handle ice candidates finished (iceGatheringState === 'complete')
- handle receiving ice candidates
- handle sending renegotiation
- handle receiving renegotiation
- reject call
- hangup call
- handle muting tracks
- handle remote track being muted
- handle adding/removing tracks to an ongoing call
- handle sdp metadata
- DONE: Participant
- handle glare
- encrypt to_device message with olm
- batch outgoing to_device messages in one request to homeserver for operations that will send out an event to all participants (e.g. mute)
- find out if we should start muted or not?
## Store ongoing calls
DONE: Add store with all ongoing calls so when we quit and start again, we don't have to go through all the past calls to know which ones might still be ongoing.
## Notes
we send m.call as state event in room
we add m.call.participant for our own device
we wait for other participants to add their user and device (in the sources)
for each (userid, deviceid)
- if userId < ourUserId
- get local media
- we setup a peer connection
- add local tracks
- we wait for negotation event to get sdp
- peerConn.createOffer
- peerConn.setLocalDescription
- we send an m.call.invite
- else
- wait for invite from other side
on local ice candidate:
- if we haven't ... sent invite yet? or received answer? buffer candidate
- otherwise send candidate (without buffering?)
on incoming call:
- ring, offer to answer
answering incoming call
- get local media
- peerConn.setRemoteDescription
- add local tracks to peerConn
- peerConn.createAnswer()
- peerConn.setLocalDescription
in some cases, we will actually send the invite to all devices (e.g. SFU), so
we probably still need to handle multiple anwsers?
so we would send an invite to multiple devices and pick the one for which we
received the anwser first. between invite and anwser, we could already receive
ice candidates that we need to buffer.
updating the metadata:
if we're renegotiating: use m.call.negotatie
if just muting: use m.call.sdp_stream_metadata_changed
party identification
- for 1:1 calls, we identify with a party_id
- for group calls, we identify with a device_id
## TODO
Build basic version of PeerCall
- add candidates code
DONE: Build basic version of GroupCall
- DONE: add state, block invalid actions
DONE: Make it possible to olm encrypt the messages
Do work needed for state events
- DONEish: receiving (almost done?)
- DONEish: sending
logging
DONE: Expose call objects
expose volume events from audiotrack to group call
DONE: Write view model
DONE: write view
- handle glare edge-cases (not yet sent): https://spec.matrix.org/latest/client-server-api/#glare
## Calls questions
- how do we handle glare between group calls (e.g. different state events with different call ids?)
- Split up DOM part into platform code? What abstractions to choose?
Does it make sense to come up with our own API very similar to DOM api?
- what code do we copy over vs what do we implement ourselves?
- MatrixCall: perhaps we can copy it over and modify it to our needs? Seems to have a lot of edge cases implemented.
- what is partyId about?
- CallFeed: I need better understand where it is used. It's basically a wrapper around a MediaStream with volume detection. Could it make sense to put this in platform for example?
- which parts of MSC2746 are still relevant for group calls?
- which parts of MSC2747 are still relevant for group calls? it seems mostly orthogonal?
- SOLVED: how does switching channels work? This was only enabled by MSC 2746
- you do getUserMedia()/getDisplayMedia() to get the stream(s)
- you call removeTrack/addTrack on the peerConnection
- you receive a negotiationneeded event
- you call createOffer
- you send m.call.negotiate
- SOLVED: wrt to MSC2746, is the screen share track and the audio track (and video track) part of the same stream? or do screen share tracks need to go in a different stream? it sounds incompatible with the MSC2746 requirement.
- SOLVED: how does muting work? MediaStreamTrack.enabled
- SOLVED: so, what's the difference between the call_id and the conf_id in group call events?
- call_id is the specific 1:1 call, conf_id is the thing in the m.call state event key
- so a group call has a conf_id with MxN peer calls, each having their call_id.
I think we need to synchronize the negotiation needed because we don't use a CallState to guard it...
## Thursday 3-3 notes
we probably best keep the perfect negotiation flags, as they are needed for both starting the call AND renegotiation? if only for the former, it would make sense as it is a step in setting up the call, but if the call is ongoing, does it make sense to have a MakingOffer state? it actually looks like they are only needed for renegotiation! for call setup we compare the call_ids. What does that mean for these flags?
## Peer call state transitions
FROM CALLER FROM CALLEE
Fledgling Fledgling
V `call()` V `handleInvite()`: setRemoteDescription(event.offer), add buffered candidates
V Ringing
V V `answer()`
CreateOffer V
V add local tracks V
V wait for negotionneeded events V add local tracks
V setLocalDescription() CreateAnswer
V send invite event V setLocalDescription(createAnswer())
InviteSent |
V receive anwser, setRemoteDescription() |
\___________________________________________________/
V
Connecting
V receive ice candidates and iceConnectionState becomes 'connected'
Connected
V `hangup()` or some terminate condition
Ended
so if we don't want to bother with having two call objects, we can make the existing call hangup his old call_id? That way we keep the old peerConnection.
when glare, won't we drop both calls? No: https://github.com/matrix-org/matrix-spec-proposals/pull/2746#discussion_r819388754

View file

@ -0,0 +1,227 @@
// allow non-camelcase as these are events type that go onto the wire
/* eslint-disable camelcase */
import type {StateEvent} from "../storage/types";
import type {SessionDescription} from "../../platform/types/WebRTC";
export enum EventType {
GroupCall = "org.matrix.msc3401.call",
GroupCallMember = "org.matrix.msc3401.call.member",
Invite = "m.call.invite",
Candidates = "m.call.candidates",
Answer = "m.call.answer",
Hangup = "m.call.hangup",
Reject = "m.call.reject",
SelectAnswer = "m.call.select_answer",
Negotiate = "m.call.negotiate",
SDPStreamMetadataChanged = "m.call.sdp_stream_metadata_changed",
SDPStreamMetadataChangedPrefix = "org.matrix.call.sdp_stream_metadata_changed",
Replaces = "m.call.replaces",
AssertedIdentity = "m.call.asserted_identity",
AssertedIdentityPrefix = "org.matrix.call.asserted_identity",
}
// TODO: Change to "sdp_stream_metadata" when MSC3077 is merged
export const SDPStreamMetadataKey = "org.matrix.msc3077.sdp_stream_metadata";
export interface CallDeviceMembership {
device_id: string,
session_id: string
}
export interface CallMembership {
["m.call_id"]: string,
["m.devices"]: CallDeviceMembership[]
}
export interface CallMemberContent {
["m.calls"]: CallMembership[];
}
export enum SDPStreamMetadataPurpose {
Usermedia = "m.usermedia",
Screenshare = "m.screenshare",
}
export interface SDPStreamMetadataObject {
purpose: SDPStreamMetadataPurpose;
audio_muted: boolean;
video_muted: boolean;
}
export interface SDPStreamMetadata {
[key: string]: SDPStreamMetadataObject;
}
export interface CallCapabilities {
'm.call.transferee': boolean;
'm.call.dtmf': boolean;
}
export interface CallReplacesTarget {
id: string;
display_name: string;
avatar_url: string;
}
export type MCallBase = {
call_id: string;
version: string | number;
}
export type MGroupCallBase = MCallBase & {
conf_id: string;
device_id: string;
sender_session_id: string;
dest_session_id: string;
party_id: string; // Should not need this?
seq: number;
}
export type MCallAnswer<Base extends MCallBase> = Base & {
answer: SessionDescription;
capabilities?: CallCapabilities;
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallSelectAnswer<Base extends MCallBase> = Base & {
selected_party_id: string;
}
export type MCallInvite<Base extends MCallBase> = Base & {
offer: SessionDescription;
lifetime: number;
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallNegotiate<Base extends MCallBase> = Base & {
description: SessionDescription;
lifetime: number;
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallSDPStreamMetadataChanged<Base extends MCallBase> = Base & {
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallReplacesEvent<Base extends MCallBase> = Base & {
replacement_id: string;
target_user: CallReplacesTarget;
create_call: string;
await_call: string;
target_room: string;
}
export type MCAllAssertedIdentity<Base extends MCallBase> = Base & {
asserted_identity: {
id: string;
display_name: string;
avatar_url: string;
};
}
export type MCallCandidates<Base extends MCallBase> = Base & {
candidates: RTCIceCandidate[];
}
export type MCallHangupReject<Base extends MCallBase> = Base & {
reason?: CallErrorCode;
}
export enum CallErrorCode {
/** The user chose to end the call */
UserHangup = 'user_hangup',
/** An error code when the local client failed to create an offer. */
LocalOfferFailed = 'local_offer_failed',
/**
* An error code when there is no local mic/camera to use. This may be because
* the hardware isn't plugged in, or the user has explicitly denied access.
*/
NoUserMedia = 'no_user_media',
/**
* Error code used when a call event failed to send
* because unknown devices were present in the room
*/
UnknownDevices = 'unknown_devices',
/**
* Error code used when we fail to send the invite
* for some reason other than there being unknown devices
*/
SendInvite = 'send_invite',
/**
* An answer could not be created
*/
CreateAnswer = 'create_answer',
/**
* Error code used when we fail to send the answer
* for some reason other than there being unknown devices
*/
SendAnswer = 'send_answer',
/**
* The session description from the other side could not be set
*/
SetRemoteDescription = 'set_remote_description',
/**
* The session description from this side could not be set
*/
SetLocalDescription = 'set_local_description',
/**
* A different device answered the call
*/
AnsweredElsewhere = 'answered_elsewhere',
/**
* No media connection could be established to the other party
*/
IceFailed = 'ice_failed',
/**
* The invite timed out whilst waiting for an answer
*/
InviteTimeout = 'invite_timeout',
/**
* The call was replaced by another call
*/
Replaced = 'replaced',
/**
* Signalling for the call could not be sent (other than the initial invite)
*/
SignallingFailed = 'signalling_timeout',
/**
* The remote party is busy
*/
UserBusy = 'user_busy',
/**
* We transferred the call off to somewhere else
*/
Transfered = 'transferred',
/**
* A call from the same user was found with a new session id
*/
NewSession = 'new_session',
}
export type SignallingMessage<Base extends MCallBase> =
{type: EventType.Invite, content: MCallInvite<Base>} |
{type: EventType.Negotiate, content: MCallNegotiate<Base>} |
{type: EventType.Answer, content: MCallAnswer<Base>} |
{type: EventType.SDPStreamMetadataChanged | EventType.SDPStreamMetadataChangedPrefix, content: MCallSDPStreamMetadataChanged<Base>} |
{type: EventType.Candidates, content: MCallCandidates<Base>} |
{type: EventType.Hangup | EventType.Reject, content: MCallHangupReject<Base>};
export enum CallIntent {
Ring = "m.ring",
Prompt = "m.prompt",
Room = "m.room",
};

View file

@ -0,0 +1,61 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {Track, Stream} from "../../platform/types/MediaDevices";
export function getStreamAudioTrack(stream: Stream | undefined): Track | undefined {
return stream?.getAudioTracks()[0];
}
export function getStreamVideoTrack(stream: Stream | undefined): Track | undefined {
return stream?.getVideoTracks()[0];
}
export class MuteSettings {
constructor (
private readonly isMicrophoneMuted: boolean = false,
private readonly isCameraMuted: boolean = false,
private hasMicrophoneTrack: boolean = false,
private hasCameraTrack: boolean = false,
) {}
updateTrackInfo(userMedia: Stream | undefined) {
this.hasMicrophoneTrack = !!getStreamAudioTrack(userMedia);
this.hasCameraTrack = !!getStreamVideoTrack(userMedia);
}
get microphone(): boolean {
return !this.hasMicrophoneTrack || this.isMicrophoneMuted;
}
get camera(): boolean {
return !this.hasCameraTrack || this.isCameraMuted;
}
toggleCamera(): MuteSettings {
return new MuteSettings(this.microphone, !this.camera, this.hasMicrophoneTrack, this.hasCameraTrack);
}
toggleMicrophone(): MuteSettings {
return new MuteSettings(!this.microphone, this.camera, this.hasMicrophoneTrack, this.hasCameraTrack);
}
equals(other: MuteSettings) {
return this.microphone === other.microphone && this.camera === other.camera;
}
}
export const CALL_LOG_TYPE = "call";

View file

@ -0,0 +1,543 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableMap} from "../../../observable/map/ObservableMap";
import {Member} from "./Member";
import {LocalMedia} from "../LocalMedia";
import {MuteSettings, CALL_LOG_TYPE} from "../common";
import {MemberChange, RoomMember} from "../../room/members/RoomMember";
import {EventEmitter} from "../../../utils/EventEmitter";
import {EventType, CallIntent} from "../callEventTypes";
import type {Options as MemberOptions} from "./Member";
import type {BaseObservableMap} from "../../../observable/map/BaseObservableMap";
import type {Track} from "../../../platform/types/MediaDevices";
import type {SignallingMessage, MGroupCallBase, CallMembership} from "../callEventTypes";
import type {Room} from "../../room/Room";
import type {StateEvent} from "../../storage/types";
import type {Platform} from "../../../platform/web/Platform";
import type {EncryptedMessage} from "../../e2ee/olm/Encryption";
import type {ILogItem, ILogger} from "../../../logging/types";
import type {Storage} from "../../storage/idb/Storage";
export enum GroupCallState {
Fledgling = "fledgling",
Creating = "creating",
Created = "created",
Joining = "joining",
Joined = "joined",
}
function getMemberKey(userId: string, deviceId: string) {
return JSON.stringify(userId)+`,`+JSON.stringify(deviceId);
}
function memberKeyIsForUser(key: string, userId: string) {
return key.startsWith(JSON.stringify(userId)+`,`);
}
function getDeviceFromMemberKey(key: string): string {
return JSON.parse(`[${key}]`)[1];
}
export type Options = Omit<MemberOptions, "emitUpdate" | "confId" | "encryptDeviceMessage"> & {
emitUpdate: (call: GroupCall, params?: any) => void;
encryptDeviceMessage: (roomId: string, userId: string, deviceId: string, message: SignallingMessage<MGroupCallBase>, log: ILogItem) => Promise<EncryptedMessage | undefined>,
storage: Storage,
logger: ILogger,
};
class JoinedData {
constructor(
public readonly logItem: ILogItem,
public readonly membersLogItem: ILogItem,
public localMedia: LocalMedia,
public localMuteSettings: MuteSettings
) {}
dispose() {
this.localMedia.dispose();
this.logItem.finish();
}
}
export class GroupCall extends EventEmitter<{change: never}> {
private readonly _members: ObservableMap<string, Member> = new ObservableMap();
private _memberOptions: MemberOptions;
private _state: GroupCallState;
private bufferedDeviceMessages = new Map<string, Set<SignallingMessage<MGroupCallBase>>>();
private joinedData?: JoinedData;
private _deviceIndex?: number;
private _eventTimestamp?: number;
constructor(
public readonly id: string,
newCall: boolean,
private callContent: Record<string, any>,
public readonly roomId: string,
private readonly options: Options,
) {
super();
this._state = newCall ? GroupCallState.Fledgling : GroupCallState.Created;
this._memberOptions = Object.assign({}, options, {
confId: this.id,
emitUpdate: member => this._members.update(getMemberKey(member.userId, member.deviceId), member),
encryptDeviceMessage: (userId: string, deviceId: string, message: SignallingMessage<MGroupCallBase>, log) => {
return this.options.encryptDeviceMessage(this.roomId, userId, deviceId, message, log);
}
});
}
get localMedia(): LocalMedia | undefined { return this.joinedData?.localMedia; }
get members(): BaseObservableMap<string, Member> { return this._members; }
get isTerminated(): boolean {
return this.callContent?.["m.terminated"] === true;
}
get isRinging(): boolean {
return this._state === GroupCallState.Created && this.intent === "m.ring" && !this.isMember(this.options.ownUserId);
}
get name(): string {
return this.callContent?.["m.name"];
}
get intent(): CallIntent {
return this.callContent?.["m.intent"];
}
get deviceIndex(): number | undefined {
return this._deviceIndex;
}
get eventTimestamp(): number | undefined {
return this._eventTimestamp;
}
/**
* Gives access the log item for this call while joined.
* Can be used for call diagnostics while in the call.
**/
get logItem(): ILogItem | undefined {
return this.joinedData?.logItem;
}
async join(localMedia: LocalMedia): Promise<void> {
if (this._state !== GroupCallState.Created || this.joinedData) {
return;
}
const logItem = this.options.logger.child({
l: "answer call",
t: CALL_LOG_TYPE,
id: this.id,
ownSessionId: this.options.sessionId
});
const membersLogItem = logItem.child("member connections");
const localMuteSettings = new MuteSettings();
localMuteSettings.updateTrackInfo(localMedia.userMedia);
const joinedData = new JoinedData(
logItem,
membersLogItem,
localMedia,
localMuteSettings
);
this.joinedData = joinedData;
await joinedData.logItem.wrap("join", async log => {
this._state = GroupCallState.Joining;
this.emitChange();
await log.wrap("update member state", async log => {
const memberContent = await this._createJoinPayload();
log.set("payload", memberContent);
// send m.call.member state event
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCallMember, this.options.ownUserId, memberContent, {log});
await request.response();
this.emitChange();
});
// send invite to all members that are < my userId
for (const [,member] of this._members) {
this.connectToMember(member, joinedData, log);
}
});
}
async setMedia(localMedia: LocalMedia): Promise<void> {
if ((this._state === GroupCallState.Joining || this._state === GroupCallState.Joined) && this.joinedData) {
const oldMedia = this.joinedData.localMedia;
this.joinedData.localMedia = localMedia;
// reflect the fact we gained or lost local tracks in the local mute settings
// and update the track info so PeerCall can use it to send up to date metadata,
this.joinedData.localMuteSettings.updateTrackInfo(localMedia.userMedia);
this.emitChange(); //allow listeners to see new media/mute settings
await Promise.all(Array.from(this._members.values()).map(m => {
return m.setMedia(localMedia, oldMedia);
}));
oldMedia?.dispose();
}
}
async setMuted(muteSettings: MuteSettings): Promise<void> {
const {joinedData} = this;
if (!joinedData) {
return;
}
const prevMuteSettings = joinedData.localMuteSettings;
// we still update the mute settings if nothing changed because
// you might be muted because you don't have a track or because
// you actively chosen to mute
// (which we want to respect in the future when you add a track)
joinedData.localMuteSettings = muteSettings;
joinedData.localMuteSettings.updateTrackInfo(joinedData.localMedia.userMedia);
if (!prevMuteSettings.equals(muteSettings)) {
await Promise.all(Array.from(this._members.values()).map(m => {
return m.setMuted(joinedData.localMuteSettings);
}));
this.emitChange();
}
}
get muteSettings(): MuteSettings | undefined {
return this.joinedData?.localMuteSettings;
}
get hasJoined() {
return this._state === GroupCallState.Joining || this._state === GroupCallState.Joined;
}
async leave(): Promise<void> {
const {joinedData} = this;
if (!joinedData) {
return;
}
await joinedData.logItem.wrap("leave", async log => {
try {
const memberContent = await this._leaveCallMemberContent();
// send m.call.member state event
if (memberContent) {
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCallMember, this.options.ownUserId, memberContent, {log});
await request.response();
// our own user isn't included in members, so not in the count
if (this.intent === CallIntent.Ring && this._members.size === 0) {
await this.terminate(log);
}
} else {
log.set("already_left", true);
}
} finally {
this.disconnect(log);
}
});
}
terminate(log?: ILogItem): Promise<void> {
return this.options.logger.wrapOrRun(log, {l: "terminate call", t: CALL_LOG_TYPE}, async log => {
if (this._state === GroupCallState.Fledgling) {
return;
}
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCall, this.id, Object.assign({}, this.callContent, {
"m.terminated": true
}), {log});
await request.response();
});
}
/** @internal */
create(type: "m.video" | "m.voice", log?: ILogItem): Promise<void> {
return this.options.logger.wrapOrRun(log, {l: "create call", t: CALL_LOG_TYPE}, async log => {
if (this._state !== GroupCallState.Fledgling) {
return;
}
this._state = GroupCallState.Creating;
this.emitChange();
this.callContent = Object.assign({
"m.type": type,
}, this.callContent);
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCall, this.id, this.callContent!, {log});
await request.response();
this._state = GroupCallState.Created;
this.emitChange();
});
}
/** @internal */
updateCallEvent(callContent: Record<string, any>, syncLog: ILogItem) {
syncLog.wrap({l: "update call", t: CALL_LOG_TYPE, id: this.id}, log => {
this.callContent = callContent;
if (this._state === GroupCallState.Creating) {
this._state = GroupCallState.Created;
}
log.set("status", this._state);
this.emitChange();
});
}
/** @internal */
updateRoomMembers(memberChanges: Map<string, MemberChange>) {
for (const change of memberChanges.values()) {
const {member} = change;
for (const callMember of this._members.values()) {
// find all call members for a room member (can be multiple, for every device)
if (callMember.userId === member.userId) {
callMember.updateRoomMember(member);
}
}
}
}
/** @internal */
updateMembership(userId: string, roomMember: RoomMember, callMembership: CallMembership, eventTimestamp: number, syncLog: ILogItem) {
syncLog.wrap({l: "update call membership", t: CALL_LOG_TYPE, id: this.id, userId}, log => {
const devices = callMembership["m.devices"];
const previousDeviceIds = this.getDeviceIdsForUserId(userId);
for (let deviceIndex = 0; deviceIndex < devices.length; deviceIndex++) {
const device = devices[deviceIndex];
const deviceId = device.device_id;
const memberKey = getMemberKey(userId, deviceId);
log.wrap({l: "update device membership", id: memberKey, sessionId: device.session_id}, log => {
if (userId === this.options.ownUserId && deviceId === this.options.ownDeviceId) {
this._deviceIndex = deviceIndex;
this._eventTimestamp = eventTimestamp;
if (this._state === GroupCallState.Joining) {
log.set("update_own", true);
this._state = GroupCallState.Joined;
this.emitChange();
}
} else {
let member = this._members.get(memberKey);
const sessionIdChanged = member && member.sessionId !== device.session_id;
if (member && !sessionIdChanged) {
log.set("update", true);
member.updateCallInfo(device, deviceIndex, eventTimestamp, log);
} else {
if (member && sessionIdChanged) {
log.set("removedSessionId", member.sessionId);
const disconnectLogItem = member.disconnect(false);
if (disconnectLogItem) {
log.refDetached(disconnectLogItem);
}
this._members.remove(memberKey);
member = undefined;
}
log.set("add", true);
member = new Member(
roomMember,
device, deviceIndex, eventTimestamp, this._memberOptions,
);
this._members.add(memberKey, member);
if (this.joinedData) {
this.connectToMember(member, this.joinedData, log);
}
}
// flush pending messages, either after having created the member,
// or updated the session id with updateCallInfo
this.flushPendingIncomingDeviceMessages(member, log);
}
});
}
const newDeviceIds = new Set<string>(devices.map(call => call.device_id));
// remove user as member of any calls not present anymore
for (const previousDeviceId of previousDeviceIds) {
if (!newDeviceIds.has(previousDeviceId)) {
this.removeMemberDevice(userId, previousDeviceId, log);
}
}
if (userId === this.options.ownUserId && !newDeviceIds.has(this.options.ownDeviceId)) {
this.removeOwnDevice(log);
}
});
}
/** @internal */
removeMembership(userId: string, syncLog: ILogItem) {
const deviceIds = this.getDeviceIdsForUserId(userId);
syncLog.wrap({
l: "remove call member",
t: CALL_LOG_TYPE,
id: this.id,
userId
}, log => {
for (const deviceId of deviceIds) {
this.removeMemberDevice(userId, deviceId, log);
}
if (userId === this.options.ownUserId) {
this.removeOwnDevice(log);
}
});
}
private flushPendingIncomingDeviceMessages(member: Member, log: ILogItem) {
const memberKey = getMemberKey(member.userId, member.deviceId);
const bufferedMessages = this.bufferedDeviceMessages.get(memberKey);
// check if we have any pending message for the member with (userid, deviceid, sessionid)
if (bufferedMessages) {
for (const message of bufferedMessages) {
if (message.content.sender_session_id === member.sessionId) {
member.handleDeviceMessage(message, log);
bufferedMessages.delete(message);
}
}
if (bufferedMessages.size === 0) {
this.bufferedDeviceMessages.delete(memberKey);
}
}
}
private getDeviceIdsForUserId(userId: string): string[] {
return Array.from(this._members.keys())
.filter(key => memberKeyIsForUser(key, userId))
.map(key => getDeviceFromMemberKey(key));
}
private isMember(userId: string): boolean {
return Array.from(this._members.keys()).some(key => memberKeyIsForUser(key, userId));
}
private removeOwnDevice(log: ILogItem) {
log.set("leave_own", true);
this.disconnect(log);
}
/** @internal */
disconnect(log: ILogItem) {
if (this._state === GroupCallState.Joined) {
for (const [,member] of this._members) {
const disconnectLogItem = member.disconnect(true);
if (disconnectLogItem) {
log.refDetached(disconnectLogItem);
}
}
this._state = GroupCallState.Created;
}
this.joinedData?.dispose();
this.joinedData = undefined;
this.emitChange();
}
/** @internal */
private removeMemberDevice(userId: string, deviceId: string, log: ILogItem) {
const memberKey = getMemberKey(userId, deviceId);
log.wrap({l: "remove device member", id: memberKey}, log => {
const member = this._members.get(memberKey);
if (member) {
log.set("leave", true);
this._members.remove(memberKey);
const disconnectLogItem = member.disconnect(false);
if (disconnectLogItem) {
log.refDetached(disconnectLogItem);
}
}
this.emitChange();
});
}
/** @internal */
handleDeviceMessage(message: SignallingMessage<MGroupCallBase>, userId: string, deviceId: string, syncLog: ILogItem) {
// TODO: return if we are not membering to the call
const key = getMemberKey(userId, deviceId);
let member = this._members.get(key);
if (member && message.content.sender_session_id === member.sessionId) {
member.handleDeviceMessage(message, syncLog);
} else {
const item = syncLog.log({
l: "call: buffering to_device message, member not found",
t: CALL_LOG_TYPE,
id: this.id,
userId,
deviceId,
sessionId: message.content.sender_session_id,
type: message.type
});
syncLog.refDetached(item);
// we haven't received the m.call.member yet for this caller (or with this session id).
// buffer the device messages or create the member/call as it should arrive in a moment
let messages = this.bufferedDeviceMessages.get(key);
if (!messages) {
messages = new Set();
this.bufferedDeviceMessages.set(key, messages);
}
messages.add(message);
}
}
private async _createJoinPayload() {
const {storage} = this.options;
const txn = await storage.readTxn([storage.storeNames.roomState]);
const stateEvent = await txn.roomState.get(this.roomId, EventType.GroupCallMember, this.options.ownUserId);
const stateContent = stateEvent?.event?.content ?? {
["m.calls"]: []
};
const callsInfo = stateContent["m.calls"];
let callInfo = callsInfo.find(c => c["m.call_id"] === this.id);
if (!callInfo) {
callInfo = {
["m.call_id"]: this.id,
["m.devices"]: []
};
callsInfo.push(callInfo);
}
callInfo["m.devices"] = callInfo["m.devices"].filter(d => d["device_id"] !== this.options.ownDeviceId);
callInfo["m.devices"].push({
["device_id"]: this.options.ownDeviceId,
["session_id"]: this.options.sessionId,
feeds: [{purpose: "m.usermedia"}]
});
this._deviceIndex = callInfo["m.devices"].length;
this._eventTimestamp = Date.now();
return stateContent;
}
private async _leaveCallMemberContent(): Promise<Record<string, any> | undefined> {
const {storage} = this.options;
const txn = await storage.readTxn([storage.storeNames.roomState]);
const stateEvent = await txn.roomState.get(this.roomId, EventType.GroupCallMember, this.options.ownUserId);
if (stateEvent) {
const content = stateEvent.event.content;
const callInfo = content["m.calls"]?.find(c => c["m.call_id"] === this.id);
if (callInfo) {
const devicesInfo = callInfo["m.devices"];
const deviceIndex = devicesInfo.findIndex(d => d["device_id"] === this.options.ownDeviceId);
if (deviceIndex !== -1) {
devicesInfo.splice(deviceIndex, 1);
return content;
}
}
}
}
private connectToMember(member: Member, joinedData: JoinedData, log: ILogItem) {
const memberKey = getMemberKey(member.userId, member.deviceId);
const logItem = joinedData.membersLogItem.child({l: "member", id: memberKey});
logItem.set("sessionId", member.sessionId);
log.wrap({l: "connect", id: memberKey}, log => {
const connectItem = member.connect(joinedData.localMedia, joinedData.localMuteSettings, logItem);
if (connectItem) {
log.refDetached(connectItem);
}
})
}
protected emitChange() {
this.emit("change");
this.options.emitUpdate(this);
}
}

View file

@ -0,0 +1,334 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {PeerCall, CallState} from "../PeerCall";
import {makeTxnId, makeId} from "../../common";
import {EventType, CallErrorCode} from "../callEventTypes";
import {formatToDeviceMessagesPayload} from "../../common";
import {sortedIndex} from "../../../utils/sortedIndex";
import type {MuteSettings} from "../common";
import type {Options as PeerCallOptions, RemoteMedia} from "../PeerCall";
import type {LocalMedia} from "../LocalMedia";
import type {HomeServerApi} from "../../net/HomeServerApi";
import type {MCallBase, MGroupCallBase, SignallingMessage, CallDeviceMembership} from "../callEventTypes";
import type {GroupCall} from "./GroupCall";
import type {RoomMember} from "../../room/members/RoomMember";
import type {EncryptedMessage} from "../../e2ee/olm/Encryption";
import type {ILogItem} from "../../../logging/types";
export type Options = Omit<PeerCallOptions, "emitUpdate" | "sendSignallingMessage"> & {
confId: string,
ownUserId: string,
ownDeviceId: string,
// local session id of our client
sessionId: string,
hsApi: HomeServerApi,
encryptDeviceMessage: (userId: string, deviceId: string, message: SignallingMessage<MGroupCallBase>, log: ILogItem) => Promise<EncryptedMessage | undefined>,
emitUpdate: (participant: Member, params?: any) => void,
}
const errorCodesWithoutRetry = [
CallErrorCode.UserHangup,
CallErrorCode.AnsweredElsewhere,
CallErrorCode.Replaced,
CallErrorCode.UserBusy,
CallErrorCode.Transfered,
CallErrorCode.NewSession
];
/** @internal */
class MemberConnection {
public retryCount: number = 0;
public peerCall?: PeerCall;
public lastProcessedSeqNr: number | undefined;
public queuedSignallingMessages: SignallingMessage<MGroupCallBase>[] = [];
public outboundSeqCounter: number = 0;
constructor(
public localMedia: LocalMedia,
public localMuteSettings: MuteSettings,
public readonly logItem: ILogItem
) {}
}
export class Member {
private connection?: MemberConnection;
constructor(
public member: RoomMember,
private callDeviceMembership: CallDeviceMembership,
private _deviceIndex: number,
private _eventTimestamp: number,
private readonly options: Options,
) {}
/**
* Gives access the log item for this item once joined to the group call.
* The signalling for this member will be log in this item.
* Can be used for call diagnostics while in the call.
**/
get logItem(): ILogItem | undefined {
return this.connection?.logItem;
}
get remoteMedia(): RemoteMedia | undefined {
return this.connection?.peerCall?.remoteMedia;
}
get remoteMuteSettings(): MuteSettings | undefined {
return this.connection?.peerCall?.remoteMuteSettings;
}
get isConnected(): boolean {
return this.connection?.peerCall?.state === CallState.Connected;
}
get userId(): string {
return this.member.userId;
}
get deviceId(): string {
return this.callDeviceMembership.device_id;
}
/** session id of the member */
get sessionId(): string {
return this.callDeviceMembership.session_id;
}
get dataChannel(): any | undefined {
return this.connection?.peerCall?.dataChannel;
}
get deviceIndex(): number {
return this._deviceIndex;
}
get eventTimestamp(): number {
return this._eventTimestamp;
}
/** @internal */
connect(localMedia: LocalMedia, localMuteSettings: MuteSettings, memberLogItem: ILogItem): ILogItem | undefined {
if (this.connection) {
return;
}
// Safari can't send a MediaStream to multiple sources, so clone it
const connection = new MemberConnection(localMedia.clone(), localMuteSettings, memberLogItem);
this.connection = connection;
let connectLogItem;
connection.logItem.wrap("connect", async log => {
connectLogItem = log;
await this.callIfNeeded(log);
});
return connectLogItem;
}
private callIfNeeded(log: ILogItem): Promise<void> {
return log.wrap("callIfNeeded", async log => {
// otherwise wait for it to connect
let shouldInitiateCall;
// the lexicographically lower side initiates the call
if (this.member.userId === this.options.ownUserId) {
shouldInitiateCall = this.deviceId > this.options.ownDeviceId;
} else {
shouldInitiateCall = this.member.userId > this.options.ownUserId;
}
if (shouldInitiateCall) {
const connection = this.connection!;
connection.peerCall = this._createPeerCall(makeId("c"));
await connection.peerCall.call(
connection.localMedia,
connection.localMuteSettings,
log
);
} else {
log.set("wait_for_invite", true);
}
});
}
/** @internal */
disconnect(hangup: boolean): ILogItem | undefined {
const {connection} = this;
if (!connection) {
return;
}
let disconnectLogItem;
connection.logItem.wrap("disconnect", async log => {
disconnectLogItem = log;
if (hangup) {
await connection.peerCall?.hangup(CallErrorCode.UserHangup, log);
} else {
await connection.peerCall?.close(undefined, log);
}
connection.peerCall?.dispose();
connection.localMedia?.dispose();
this.connection = undefined;
});
connection.logItem.finish();
return disconnectLogItem;
}
/** @internal */
updateCallInfo(
callDeviceMembership: CallDeviceMembership,
deviceIndex: number,
eventTimestamp: number,
causeItem: ILogItem
) {
this.callDeviceMembership = callDeviceMembership;
this._deviceIndex = deviceIndex;
this._eventTimestamp = eventTimestamp;
if (this.connection) {
this.connection.logItem.refDetached(causeItem);
}
}
/** @internal */
updateRoomMember(roomMember: RoomMember) {
this.member = roomMember;
// TODO: this emits an update during the writeSync phase, which we usually try to avoid
this.options.emitUpdate(this);
}
/** @internal */
emitUpdateFromPeerCall = (peerCall: PeerCall, params: any, log: ILogItem): void => {
const connection = this.connection!;
if (peerCall.state === CallState.Ringing) {
connection.logItem.wrap("ringing, answer peercall", answerLog => {
log.refDetached(answerLog);
return peerCall.answer(connection.localMedia, connection.localMuteSettings, answerLog);
});
}
else if (peerCall.state === CallState.Ended) {
const hangupReason = peerCall.hangupReason;
peerCall.dispose();
connection.peerCall = undefined;
if (hangupReason && !errorCodesWithoutRetry.includes(hangupReason)) {
connection.retryCount += 1;
const {retryCount} = connection;
connection.logItem.wrap({l: "retry connection", retryCount}, async retryLog => {
log.refDetached(retryLog);
if (retryCount <= 3) {
await this.callIfNeeded(retryLog);
} else {
const disconnectLogItem = this.disconnect(false);
if (disconnectLogItem) {
retryLog.refDetached(disconnectLogItem);
}
}
});
}
}
this.options.emitUpdate(this, params);
}
/** @internal */
sendSignallingMessage = async (message: SignallingMessage<MCallBase>, log: ILogItem): Promise<void> => {
const groupMessage = message as SignallingMessage<MGroupCallBase>;
groupMessage.content.seq = this.connection!.outboundSeqCounter++;
groupMessage.content.conf_id = this.options.confId;
groupMessage.content.device_id = this.options.ownDeviceId;
groupMessage.content.party_id = this.options.ownDeviceId;
groupMessage.content.sender_session_id = this.options.sessionId;
groupMessage.content.dest_session_id = this.sessionId;
let payload;
let type: string = message.type;
const encryptedMessages = await this.options.encryptDeviceMessage(this.member.userId, this.deviceId, groupMessage, log);
if (encryptedMessages) {
payload = formatToDeviceMessagesPayload(encryptedMessages);
type = "m.room.encrypted";
} else {
// device needs deviceId and userId
payload = formatToDeviceMessagesPayload([{content: groupMessage.content, device: this}]);
}
// TODO: remove this for release
log.set("payload", groupMessage.content);
const request = this.options.hsApi.sendToDevice(
type,
payload,
makeTxnId(),
{log}
);
await request.response();
}
/** @internal */
handleDeviceMessage(message: SignallingMessage<MGroupCallBase>, syncLog: ILogItem): void{
const {connection} = this;
if (connection) {
const destSessionId = message.content.dest_session_id;
if (destSessionId !== this.options.sessionId) {
const logItem = connection.logItem.log({l: "ignoring to_device event with wrong session_id", destSessionId, type: message.type});
syncLog.refDetached(logItem);
return;
}
if (message.type === EventType.Invite && !connection.peerCall) {
connection.peerCall = this._createPeerCall(message.content.call_id);
}
const idx = sortedIndex(connection.queuedSignallingMessages, message, (a, b) => a.content.seq - b.content.seq);
connection.queuedSignallingMessages.splice(idx, 0, message);
let hasBeenDequeued = false;
if (connection.peerCall) {
while (
connection.queuedSignallingMessages.length && (
connection.lastProcessedSeqNr === undefined ||
connection.queuedSignallingMessages[0].content.seq === connection.lastProcessedSeqNr + 1
)
) {
const dequeuedMessage = connection.queuedSignallingMessages.shift()!;
if (dequeuedMessage === message) {
hasBeenDequeued = true;
}
const item = connection.peerCall!.handleIncomingSignallingMessage(dequeuedMessage, this.deviceId, connection.logItem);
syncLog.refDetached(item);
connection.lastProcessedSeqNr = dequeuedMessage.content.seq;
}
}
if (!hasBeenDequeued) {
syncLog.refDetached(connection.logItem.log({l: "queued signalling message", type: message.type, seq: message.content.seq}));
}
} else {
syncLog.log({l: "member not connected", userId: this.userId, deviceId: this.deviceId});
}
}
/** @internal */
async setMedia(localMedia: LocalMedia, previousMedia: LocalMedia): Promise<void> {
const {connection} = this;
if (connection) {
connection.localMedia = localMedia.replaceClone(connection.localMedia, previousMedia);
await connection.peerCall?.setMedia(connection.localMedia, connection.logItem);
}
}
async setMuted(muteSettings: MuteSettings): Promise<void> {
const {connection} = this;
if (connection) {
connection.localMuteSettings = muteSettings;
await connection.peerCall?.setMuted(muteSettings, connection.logItem);
}
}
private _createPeerCall(callId: string): PeerCall {
return new PeerCall(callId, Object.assign({}, this.options, {
emitUpdate: this.emitUpdateFromPeerCall,
sendSignallingMessage: this.sendSignallingMessage
}), this.connection!.logItem);
}
}

View file

@ -15,16 +15,37 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {groupBy} from "../utils/groupBy";
export function makeTxnId() { export function makeTxnId() {
return makeId("t");
}
export function makeId(prefix) {
const n = Math.floor(Math.random() * Number.MAX_SAFE_INTEGER); const n = Math.floor(Math.random() * Number.MAX_SAFE_INTEGER);
const str = n.toString(16); const str = n.toString(16);
return "t" + "0".repeat(14 - str.length) + str; return prefix + "0".repeat(14 - str.length) + str;
} }
export function isTxnId(txnId) { export function isTxnId(txnId) {
return txnId.startsWith("t") && txnId.length === 15; return txnId.startsWith("t") && txnId.length === 15;
} }
export function formatToDeviceMessagesPayload(messages) {
const messagesByUser = groupBy(messages, message => message.device.userId);
const payload = {
messages: Array.from(messagesByUser.entries()).reduce((userMap, [userId, messages]) => {
userMap[userId] = messages.reduce((deviceMap, message) => {
deviceMap[message.device.deviceId] = message.content;
return deviceMap;
}, {});
return userMap;
}, {})
};
return payload;
}
export function tests() { export function tests() {
return { return {
"isTxnId succeeds on result of makeTxnId": assert => { "isTxnId succeeds on result of makeTxnId": assert => {

View file

@ -69,6 +69,14 @@ export class DecryptionResult {
} }
} }
get userId(): string | undefined {
return this.device?.userId;
}
get deviceId(): string | undefined {
return this.device?.deviceId;
}
get isVerificationUnknown(): boolean { get isVerificationUnknown(): boolean {
// verification is unknown if we haven't yet fetched the devices for the room // verification is unknown if we haven't yet fetched the devices for the room
return !this.device && !this.roomTracked; return !this.device && !this.roomTracked;

View file

@ -309,6 +309,7 @@ export class DeviceTracker {
return await this._devicesForUserIds(roomId, userIds, txn, hsApi, log); return await this._devicesForUserIds(roomId, userIds, txn, hsApi, log);
} }
/** gets devices for the given user ids that are in the given room */
async devicesForRoomMembers(roomId, userIds, hsApi, log) { async devicesForRoomMembers(roomId, userIds, hsApi, log) {
const txn = await this._storage.readTxn([ const txn = await this._storage.readTxn([
this._storage.storeNames.userIdentities, this._storage.storeNames.userIdentities,
@ -316,6 +317,60 @@ export class DeviceTracker {
return await this._devicesForUserIds(roomId, userIds, txn, hsApi, log); return await this._devicesForUserIds(roomId, userIds, txn, hsApi, log);
} }
/** gets a single device */
async deviceForId(userId, deviceId, hsApi, log) {
const txn = await this._storage.readTxn([
this._storage.storeNames.deviceIdentities,
]);
let device = await txn.deviceIdentities.get(userId, deviceId);
if (device) {
log.set("existingDevice", true);
} else {
//// BEGIN EXTRACT (deviceKeysMap)
const deviceKeyResponse = await hsApi.queryKeys({
"timeout": 10000,
"device_keys": {
[userId]: [deviceId]
},
"token": this._getSyncToken()
}, {log}).response();
// verify signature
const verifiedKeysPerUser = log.wrap("verify", log => this._filterVerifiedDeviceKeys(deviceKeyResponse["device_keys"], log));
//// END EXTRACT
const verifiedKeys = verifiedKeysPerUser
.find(vkpu => vkpu.userId === userId).verifiedKeys
.find(vk => vk["device_id"] === deviceId);
// user hasn't uploaded keys for device?
if (!verifiedKeys) {
return undefined;
}
device = deviceKeysAsDeviceIdentity(verifiedKeys);
const txn = await this._storage.readWriteTxn([
this._storage.storeNames.deviceIdentities,
]);
// check again we don't have the device already.
// when updating all keys for a user we allow updating the
// device when the key hasn't changed so the device display name
// can be updated, but here we don't.
const existingDevice = await txn.deviceIdentities.get(userId, deviceId);
if (existingDevice) {
device = existingDevice;
log.set("existingDeviceAfterFetch", true);
} else {
try {
txn.deviceIdentities.set(device);
log.set("newDevice", true);
} catch (err) {
txn.abort();
throw err;
}
await txn.complete();
}
}
return device;
}
/** /**
* @param {string} roomId [description] * @param {string} roomId [description]
* @param {Array<string>} userIds a set of user ids to try and find the identity for. Will be check to belong to roomId. * @param {Array<string>} userIds a set of user ids to try and find the identity for. Will be check to belong to roomId.

View file

@ -41,5 +41,8 @@ Runs before any room.prepareSync, so the new room keys can be passed to each roo
- e2ee account - e2ee account
- generate more otks if needed - generate more otks if needed
- upload new otks if needed or device keys if not uploaded before - upload new otks if needed or device keys if not uploaded before
- device message handler:
- fetch keys we don't know about yet for (call) to_device messages identity
- pass signalling messages to call handler
- rooms - rooms
- share new room keys if needed - share new room keys if needed

View file

@ -18,7 +18,7 @@ import {MEGOLM_ALGORITHM, DecryptionSource} from "./common.js";
import {groupEventsBySession} from "./megolm/decryption/utils"; import {groupEventsBySession} from "./megolm/decryption/utils";
import {mergeMap} from "../../utils/mergeMap"; import {mergeMap} from "../../utils/mergeMap";
import {groupBy} from "../../utils/groupBy"; import {groupBy} from "../../utils/groupBy";
import {makeTxnId} from "../common.js"; import {makeTxnId, formatToDeviceMessagesPayload} from "../common.js";
const ENCRYPTED_TYPE = "m.room.encrypted"; const ENCRYPTED_TYPE = "m.room.encrypted";
// how often ensureMessageKeyIsShared can check if it needs to // how often ensureMessageKeyIsShared can check if it needs to
@ -386,6 +386,7 @@ export class RoomEncryption {
await writeTxn.complete(); await writeTxn.complete();
} }
// TODO: make this use _sendMessagesToDevices
async _sendSharedMessageToDevices(type, message, devices, hsApi, log) { async _sendSharedMessageToDevices(type, message, devices, hsApi, log) {
const devicesByUser = groupBy(devices, device => device.userId); const devicesByUser = groupBy(devices, device => device.userId);
const payload = { const payload = {
@ -403,16 +404,7 @@ export class RoomEncryption {
async _sendMessagesToDevices(type, messages, hsApi, log) { async _sendMessagesToDevices(type, messages, hsApi, log) {
log.set("messages", messages.length); log.set("messages", messages.length);
const messagesByUser = groupBy(messages, message => message.device.userId); const payload = formatToDeviceMessagesPayload(messages);
const payload = {
messages: Array.from(messagesByUser.entries()).reduce((userMap, [userId, messages]) => {
userMap[userId] = messages.reduce((deviceMap, message) => {
deviceMap[message.device.deviceId] = message.content;
return deviceMap;
}, {});
return userMap;
}, {})
};
const txnId = makeTxnId(); const txnId = makeTxnId();
await hsApi.sendToDevice(type, payload, txnId, {log}).response(); await hsApi.sendToDevice(type, payload, txnId, {log}).response();
} }

View file

@ -19,7 +19,7 @@ import {StoredRoomKey, keyFromBackup} from "../decryption/RoomKey";
import {MEGOLM_ALGORITHM} from "../../common"; import {MEGOLM_ALGORITHM} from "../../common";
import * as Curve25519 from "./Curve25519"; import * as Curve25519 from "./Curve25519";
import {AbortableOperation} from "../../../../utils/AbortableOperation"; import {AbortableOperation} from "../../../../utils/AbortableOperation";
import {ObservableValue} from "../../../../observable/ObservableValue"; import {ObservableValue} from "../../../../observable/value/ObservableValue";
import {SetAbortableFn} from "../../../../utils/AbortableOperation"; import {SetAbortableFn} from "../../../../utils/AbortableOperation";
import type {BackupInfo, SessionData, SessionKeyInfo, SessionInfo, KeyBackupPayload} from "./types"; import type {BackupInfo, SessionData, SessionKeyInfo, SessionInfo, KeyBackupPayload} from "./types";

View file

@ -311,7 +311,7 @@ class EncryptionTarget {
} }
} }
class EncryptedMessage { export class EncryptedMessage {
constructor( constructor(
public readonly content: OlmEncryptedMessageContent, public readonly content: OlmEncryptedMessageContent,
public readonly device: DeviceIdentity public readonly device: DeviceIdentity

View file

@ -160,6 +160,10 @@ export class HomeServerApi {
return this._get(`/rooms/${encodeURIComponent(roomId)}/state/${encodeURIComponent(eventType)}/${encodeURIComponent(stateKey)}`, {}, undefined, options); return this._get(`/rooms/${encodeURIComponent(roomId)}/state/${encodeURIComponent(eventType)}/${encodeURIComponent(stateKey)}`, {}, undefined, options);
} }
sendState(roomId: string, eventType: string, stateKey: string, content: Record<string, any>, options?: BaseRequestOptions): IHomeServerRequest {
return this._put(`/rooms/${encodeURIComponent(roomId)}/state/${encodeURIComponent(eventType)}/${encodeURIComponent(stateKey)}`, {}, content, options);
}
getLoginFlows(): IHomeServerRequest { getLoginFlows(): IHomeServerRequest {
return this._unauthedRequest("GET", this._url("/login")); return this._unauthedRequest("GET", this._url("/login"));
} }
@ -267,6 +271,12 @@ export class HomeServerApi {
return this._post(`/join/${encodeURIComponent(roomIdOrAlias)}`, {}, {}, options); return this._post(`/join/${encodeURIComponent(roomIdOrAlias)}`, {}, {}, options);
} }
invite(roomId: string, userId: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/rooms/${encodeURIComponent(roomId)}/invite`, {}, {
user_id: userId
}, options);
}
leave(roomId: string, options?: BaseRequestOptions): IHomeServerRequest { leave(roomId: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/rooms/${encodeURIComponent(roomId)}/leave`, {}, {}, options); return this._post(`/rooms/${encodeURIComponent(roomId)}/leave`, {}, {}, options);
} }
@ -275,6 +285,27 @@ export class HomeServerApi {
return this._post(`/rooms/${encodeURIComponent(roomId)}/forget`, {}, {}, options); return this._post(`/rooms/${encodeURIComponent(roomId)}/forget`, {}, {}, options);
} }
kick(roomId: string, userId: string, reason?: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/rooms/${encodeURIComponent(roomId)}/kick`, {}, {
user_id: userId,
reason: reason,
}, options);
}
ban(roomId: string, userId: string, reason?: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/rooms/${encodeURIComponent(roomId)}/ban`, {}, {
user_id: userId,
reason: reason,
}, options);
}
unban(roomId: string, userId: string, reason?: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/rooms/${encodeURIComponent(roomId)}/unban`, {}, {
user_id: userId,
reason: reason,
}, options);
}
logout(options?: BaseRequestOptions): IHomeServerRequest { logout(options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/logout`, {}, {}, options); return this._post(`/logout`, {}, {}, options);
} }
@ -294,10 +325,25 @@ export class HomeServerApi {
return this._post(`/dehydrated_device/claim`, {}, {device_id: deviceId}, options); return this._post(`/dehydrated_device/claim`, {}, {device_id: deviceId}, options);
} }
searchProfile(searchTerm: string, limit?: number, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/user_directory/search`, {}, {
limit: limit ?? 10,
search_term: searchTerm,
}, options);
}
profile(userId: string, options?: BaseRequestOptions): IHomeServerRequest { profile(userId: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._get(`/profile/${encodeURIComponent(userId)}`); return this._get(`/profile/${encodeURIComponent(userId)}`);
} }
setProfileDisplayName(userId, displayName, options?: BaseRequestOptions): IHomeServerRequest {
return this._put(`/profile/${encodeURIComponent(userId)}/displayname`, {}, { displayname: displayName }, options);
}
setProfileAvatarUrl(userId, avatarUrl, options?: BaseRequestOptions): IHomeServerRequest {
return this._put(`/profile/${encodeURIComponent(userId)}/avatar_url`, {}, { avatar_url: avatarUrl }, options);
}
createRoom(payload: Record<string, any>, options?: BaseRequestOptions): IHomeServerRequest { createRoom(payload: Record<string, any>, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/createRoom`, {}, payload, options); return this._post(`/createRoom`, {}, payload, options);
} }

View file

@ -29,32 +29,31 @@ export class MediaRepository {
this._platform = platform; this._platform = platform;
} }
mxcUrlThumbnail(url: string, width: number, height: number, method: "crop" | "scale"): string | null { mxcUrlThumbnail(url: string, width: number, height: number, method: "crop" | "scale"): string | undefined {
const parts = this._parseMxcUrl(url); const parts = this._parseMxcUrl(url);
if (parts) { if (parts) {
const [serverName, mediaId] = parts; const [serverName, mediaId] = parts;
const httpUrl = `${this._homeserver}/_matrix/media/r0/thumbnail/${encodeURIComponent(serverName)}/${encodeURIComponent(mediaId)}`; const httpUrl = `${this._homeserver}/_matrix/media/r0/thumbnail/${encodeURIComponent(serverName)}/${encodeURIComponent(mediaId)}`;
return httpUrl + "?" + encodeQueryParams({width: Math.round(width), height: Math.round(height), method}); return httpUrl + "?" + encodeQueryParams({width: Math.round(width), height: Math.round(height), method});
} }
return null; return undefined;
} }
mxcUrl(url: string): string | null { mxcUrl(url: string): string | undefined {
const parts = this._parseMxcUrl(url); const parts = this._parseMxcUrl(url);
if (parts) { if (parts) {
const [serverName, mediaId] = parts; const [serverName, mediaId] = parts;
return `${this._homeserver}/_matrix/media/r0/download/${encodeURIComponent(serverName)}/${encodeURIComponent(mediaId)}`; return `${this._homeserver}/_matrix/media/r0/download/${encodeURIComponent(serverName)}/${encodeURIComponent(mediaId)}`;
} else {
return null;
} }
return undefined;
} }
private _parseMxcUrl(url: string): string[] | null { private _parseMxcUrl(url: string): string[] | undefined {
const prefix = "mxc://"; const prefix = "mxc://";
if (url.startsWith(prefix)) { if (url.startsWith(prefix)) {
return url.substr(prefix.length).split("/", 2); return url.substr(prefix.length).split("/", 2);
} else { } else {
return null; return undefined;
} }
} }

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {ObservableValue} from "../../observable/ObservableValue"; import {ObservableValue} from "../../observable/value/ObservableValue";
import type {ExponentialRetryDelay} from "./ExponentialRetryDelay"; import type {ExponentialRetryDelay} from "./ExponentialRetryDelay";
import type {TimeMeasure} from "../../platform/web/dom/Clock.js"; import type {TimeMeasure} from "../../platform/web/dom/Clock.js";
import type {OnlineStatus} from "../../platform/web/dom/OnlineStatus.js"; import type {OnlineStatus} from "../../platform/web/dom/OnlineStatus.js";

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {reduceStateEvents} from "./RoomSummary.js"; import {iterateResponseStateEvents} from "./common";
import {BaseRoom} from "./BaseRoom.js"; import {BaseRoom} from "./BaseRoom.js";
import {RoomMember, EVENT_TYPE as MEMBER_EVENT_TYPE} from "./members/RoomMember.js"; import {RoomMember, EVENT_TYPE as MEMBER_EVENT_TYPE} from "./members/RoomMember.js";
@ -173,15 +173,15 @@ export class ArchivedRoom extends BaseRoom {
} }
function findKickDetails(roomResponse, ownUserId) { function findKickDetails(roomResponse, ownUserId) {
const kickEvent = reduceStateEvents(roomResponse, (kickEvent, event) => { let kickEvent;
iterateResponseStateEvents(roomResponse, event => {
if (event.type === MEMBER_EVENT_TYPE) { if (event.type === MEMBER_EVENT_TYPE) {
// did we get kicked? // did we get kicked?
if (event.state_key === ownUserId && event.sender !== event.state_key) { if (event.state_key === ownUserId && event.sender !== event.state_key) {
kickEvent = event; kickEvent = event;
} }
} }
return kickEvent; });
}, null);
if (kickEvent) { if (kickEvent) {
return { return {
// this is different from the room membership in the sync section, which can only be leave // this is different from the room membership in the sync section, which can only be leave

View file

@ -29,8 +29,10 @@ import {ObservedEventMap} from "./ObservedEventMap.js";
import {DecryptionSource} from "../e2ee/common.js"; import {DecryptionSource} from "../e2ee/common.js";
import {ensureLogItem} from "../../logging/utils"; import {ensureLogItem} from "../../logging/utils";
import {PowerLevels} from "./PowerLevels.js"; import {PowerLevels} from "./PowerLevels.js";
import {RetainedObservableValue} from "../../observable/ObservableValue"; import {RetainedObservableValue} from "../../observable/value/RetainedObservableValue";
import {TimelineReader} from "./timeline/persistence/TimelineReader"; import {TimelineReader} from "./timeline/persistence/TimelineReader";
import {ObservedStateTypeMap} from "./state/ObservedStateTypeMap";
import {ObservedStateKeyValue} from "./state/ObservedStateKeyValue";
const EVENT_ENCRYPTED_TYPE = "m.room.encrypted"; const EVENT_ENCRYPTED_TYPE = "m.room.encrypted";
@ -45,6 +47,7 @@ export class BaseRoom extends EventEmitter {
this._fragmentIdComparer = new FragmentIdComparer([]); this._fragmentIdComparer = new FragmentIdComparer([]);
this._emitCollectionChange = emitCollectionChange; this._emitCollectionChange = emitCollectionChange;
this._timeline = null; this._timeline = null;
this._openTimelinePromise = null;
this._user = user; this._user = user;
this._changedMembersDuringSync = null; this._changedMembersDuringSync = null;
this._memberList = null; this._memberList = null;
@ -53,11 +56,40 @@ export class BaseRoom extends EventEmitter {
this._getSyncToken = getSyncToken; this._getSyncToken = getSyncToken;
this._platform = platform; this._platform = platform;
this._observedEvents = null; this._observedEvents = null;
this._roomStateObservers = new Set();
this._powerLevels = null; this._powerLevels = null;
this._powerLevelLoading = null; this._powerLevelLoading = null;
this._observedMembers = null; this._observedMembers = null;
} }
async observeStateType(type, txn = undefined) {
const map = new ObservedStateTypeMap(type);
await this._addStateObserver(map, txn);
return map;
}
async observeStateTypeAndKey(type, stateKey, txn = undefined) {
const value = new ObservedStateKeyValue(type, stateKey);
await this._addStateObserver(value, txn);
return value;
}
async getStateEvent(type, key = '') {
const txn = await this._storage.readTxn(['roomState']);
return txn.roomState.get(this.id, type, key);
}
async _addStateObserver(stateObserver, txn) {
if (!txn) {
txn = await this._storage.readTxn([this._storage.storeNames.roomState]);
}
await stateObserver.load(this.id, txn);
this._roomStateObservers.add(stateObserver);
stateObserver.setRemoveCallback(() => {
this._roomStateObservers.delete(stateObserver);
});
}
async _eventIdsToEntries(eventIds, txn) { async _eventIdsToEntries(eventIds, txn) {
const retryEntries = []; const retryEntries = [];
await Promise.all(eventIds.map(async eventId => { await Promise.all(eventIds.map(async eventId => {
@ -383,6 +415,10 @@ export class BaseRoom extends EventEmitter {
return this._roomId; return this._roomId;
} }
get type() {
return this._summary.data.type ?? undefined;
}
get lastMessageTimestamp() { get lastMessageTimestamp() {
return this._summary.data.lastMessageTimestamp; return this._summary.data.lastMessageTimestamp;
} }
@ -420,6 +456,14 @@ export class BaseRoom extends EventEmitter {
return this._summary.data.membership; return this._summary.data.membership;
} }
get isDirectMessage() {
return this._summary.data.isDirectMessage;
}
get user() {
return this._user;
}
isDirectMessageForUserId(userId) { isDirectMessageForUserId(userId) {
if (this._summary.data.dmUserId === userId) { if (this._summary.data.dmUserId === userId) {
return true; return true;
@ -499,7 +543,8 @@ export class BaseRoom extends EventEmitter {
/** @public */ /** @public */
openTimeline(log = null) { openTimeline(log = null) {
return this._platform.logger.wrapOrRun(log, "open timeline", async log => { if (this._openTimelinePromise) return this._openTimelinePromise;
this._openTimelinePromise = this._platform.logger.wrapOrRun(log, "open timeline", async log => {
log.set("id", this.id); log.set("id", this.id);
if (this._timeline) { if (this._timeline) {
throw new Error("not dealing with load race here for now"); throw new Error("not dealing with load race here for now");
@ -511,6 +556,7 @@ export class BaseRoom extends EventEmitter {
pendingEvents: this._getPendingEvents(), pendingEvents: this._getPendingEvents(),
closeCallback: () => { closeCallback: () => {
this._timeline = null; this._timeline = null;
this._openTimelinePromise = null;
if (this._roomEncryption) { if (this._roomEncryption) {
this._roomEncryption.notifyTimelineClosed(); this._roomEncryption.notifyTimelineClosed();
} }
@ -532,6 +578,7 @@ export class BaseRoom extends EventEmitter {
} }
return this._timeline; return this._timeline;
}); });
return this._openTimelinePromise;
} }
/* allow subclasses to provide an observable list with pending events when opening the timeline */ /* allow subclasses to provide an observable list with pending events when opening the timeline */

View file

@ -61,6 +61,10 @@ export class Invite extends EventEmitter {
return this._inviteData.avatarColorId || this.id; return this._inviteData.avatarColorId || this.id;
} }
get type() {
return this._inviteData.type ?? undefined;
}
get timestamp() { get timestamp() {
return this._inviteData.timestamp; return this._inviteData.timestamp;
} }
@ -89,7 +93,7 @@ export class Invite extends EventEmitter {
await this._platform.logger.wrapOrRun(log, "acceptInvite", async log => { await this._platform.logger.wrapOrRun(log, "acceptInvite", async log => {
this._accepting = true; this._accepting = true;
this._emitChange("accepting"); this._emitChange("accepting");
await this._hsApi.join(this._roomId, {log}).response(); await this._hsApi.joinIdOrAlias(this.canonicalAlias ?? this._roomId, {log}).response();
}); });
} }
@ -189,7 +193,7 @@ export class Invite extends EventEmitter {
roomId: this.id, roomId: this.id,
isEncrypted: !!summaryData.encryption, isEncrypted: !!summaryData.encryption,
isDirectMessage: summaryData.isDirectMessage, isDirectMessage: summaryData.isDirectMessage,
// type: type: summaryData.type,
name, name,
avatarUrl, avatarUrl,
avatarColorId, avatarColorId,

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseObservableValue} from "../../observable/ObservableValue"; import {BaseObservableValue} from "../../observable/value/BaseObservableValue";
export class ObservedEventMap { export class ObservedEventMap {
constructor(notifyEmpty) { constructor(notifyEmpty) {

View file

@ -23,6 +23,7 @@ import {WrappedError} from "../error.js"
import {Heroes} from "./members/Heroes.js"; import {Heroes} from "./members/Heroes.js";
import {AttachmentUpload} from "./AttachmentUpload.js"; import {AttachmentUpload} from "./AttachmentUpload.js";
import {DecryptionSource} from "../e2ee/common.js"; import {DecryptionSource} from "../e2ee/common.js";
import {iterateResponseStateEvents} from "./common";
import {PowerLevels, EVENT_TYPE as POWERLEVELS_EVENT_TYPE } from "./PowerLevels.js"; import {PowerLevels, EVENT_TYPE as POWERLEVELS_EVENT_TYPE } from "./PowerLevels.js";
const EVENT_ENCRYPTED_TYPE = "m.room.encrypted"; const EVENT_ENCRYPTED_TYPE = "m.room.encrypted";
@ -30,6 +31,7 @@ const EVENT_ENCRYPTED_TYPE = "m.room.encrypted";
export class Room extends BaseRoom { export class Room extends BaseRoom {
constructor(options) { constructor(options) {
super(options); super(options);
this._roomStateHandler = options.roomStateHandler;
// TODO: pass pendingEvents to start like pendingOperations? // TODO: pass pendingEvents to start like pendingOperations?
const {pendingEvents} = options; const {pendingEvents} = options;
const relationWriter = new RelationWriter({ const relationWriter = new RelationWriter({
@ -121,7 +123,7 @@ export class Room extends BaseRoom {
txn.roomState.removeAllForRoom(this.id); txn.roomState.removeAllForRoom(this.id);
txn.roomMembers.removeAllForRoom(this.id); txn.roomMembers.removeAllForRoom(this.id);
} }
const {entries: newEntries, updatedEntries, newLiveKey, memberChanges} = const {entries: newEntries, updatedEntries, newLiveKey, memberChanges, memberSync} =
await log.wrap("syncWriter", log => this._syncWriter.writeSync( await log.wrap("syncWriter", log => this._syncWriter.writeSync(
roomResponse, isRejoin, summaryChanges.hasFetchedMembers, txn, log), log.level.Detail); roomResponse, isRejoin, summaryChanges.hasFetchedMembers, txn, log), log.level.Detail);
if (decryptChanges) { if (decryptChanges) {
@ -178,7 +180,9 @@ export class Room extends BaseRoom {
removedPendingEvents = await this._sendQueue.removeRemoteEchos(roomResponse.timeline.events, txn, log); removedPendingEvents = await this._sendQueue.removeRemoteEchos(roomResponse.timeline.events, txn, log);
} }
const powerLevelsEvent = this._getPowerLevelsEvent(roomResponse); const powerLevelsEvent = this._getPowerLevelsEvent(roomResponse);
await this._runRoomStateHandlers(roomResponse, memberSync, txn, log);
return { return {
roomResponse,
summaryChanges, summaryChanges,
roomEncryption, roomEncryption,
newEntries, newEntries,
@ -201,7 +205,7 @@ export class Room extends BaseRoom {
const { const {
summaryChanges, newEntries, updatedEntries, newLiveKey, summaryChanges, newEntries, updatedEntries, newLiveKey,
removedPendingEvents, memberChanges, powerLevelsEvent, removedPendingEvents, memberChanges, powerLevelsEvent,
heroChanges, roomEncryption heroChanges, roomEncryption, roomResponse
} = changes; } = changes;
log.set("id", this.id); log.set("id", this.id);
this._syncWriter.afterSync(newLiveKey); this._syncWriter.afterSync(newLiveKey);
@ -215,6 +219,7 @@ export class Room extends BaseRoom {
if (this._memberList) { if (this._memberList) {
this._memberList.afterSync(memberChanges); this._memberList.afterSync(memberChanges);
} }
this._roomStateHandler.updateRoomMembers(this, memberChanges);
if (this._observedMembers) { if (this._observedMembers) {
this._updateObservedMembers(memberChanges); this._updateObservedMembers(memberChanges);
} }
@ -260,6 +265,7 @@ export class Room extends BaseRoom {
if (removedPendingEvents) { if (removedPendingEvents) {
this._sendQueue.emitRemovals(removedPendingEvents); this._sendQueue.emitRemovals(removedPendingEvents);
} }
this._emitSyncRoomState(roomResponse);
} }
_updateObservedMembers(memberChanges) { _updateObservedMembers(memberChanges) {
@ -272,8 +278,13 @@ export class Room extends BaseRoom {
} }
_getPowerLevelsEvent(roomResponse) { _getPowerLevelsEvent(roomResponse) {
const isPowerlevelEvent = event => event.state_key === "" && event.type === POWERLEVELS_EVENT_TYPE; let powerLevelEvent;
const powerLevelEvent = roomResponse.timeline?.events.find(isPowerlevelEvent) ?? roomResponse.state?.events.find(isPowerlevelEvent); iterateResponseStateEvents(roomResponse, event => {
if(event.state_key === "" && event.type === POWERLEVELS_EVENT_TYPE) {
powerLevelEvent = event;
}
});
return powerLevelEvent; return powerLevelEvent;
} }
@ -442,6 +453,24 @@ export class Room extends BaseRoom {
return this._sendQueue.pendingEvents; return this._sendQueue.pendingEvents;
} }
/** global room state handlers, run during writeSync step */
_runRoomStateHandlers(roomResponse, memberSync, txn, log) {
const promises = [];
iterateResponseStateEvents(roomResponse, event => {
promises.push(this._roomStateHandler.handleRoomState(this, event, memberSync, txn, log));
});
return Promise.all(promises);
}
/** local room state observers, run during afterSync step */
_emitSyncRoomState(roomResponse) {
iterateResponseStateEvents(roomResponse, event => {
for (const handler of this._roomStateObservers) {
handler.handleStateEvent(event);
}
});
}
/** @package */ /** @package */
writeIsTrackingMembers(value, txn) { writeIsTrackingMembers(value, txn) {
return this._summary.writeIsTrackingMembers(value, txn); return this._summary.writeIsTrackingMembers(value, txn);

View file

@ -20,7 +20,7 @@ import {MediaRepository} from "../net/MediaRepository";
import {EventEmitter} from "../../utils/EventEmitter"; import {EventEmitter} from "../../utils/EventEmitter";
import {AttachmentUpload} from "./AttachmentUpload"; import {AttachmentUpload} from "./AttachmentUpload";
import {loadProfiles, Profile, UserIdProfile} from "../profile"; import {loadProfiles, Profile, UserIdProfile} from "../profile";
import {RoomType} from "./common"; import {RoomType, RoomVisibility} from "./common";
import type {HomeServerApi} from "../net/HomeServerApi"; import type {HomeServerApi} from "../net/HomeServerApi";
import type {ILogItem} from "../../logging/types"; import type {ILogItem} from "../../logging/types";
@ -36,8 +36,9 @@ type CreateRoomPayload = {
topic?: string; topic?: string;
invite?: string[]; invite?: string[];
room_alias_name?: string; room_alias_name?: string;
creation_content?: {"m.federate": boolean}; creation_content?: {"m.federate"?: boolean, type?: string};
initial_state: {type: string; state_key: string; content: Record<string, any>}[] initial_state: {type: string; state_key: string; content: Record<string, any>}[]
power_level_content_override?: any;
} }
type ImageInfo = { type ImageInfo = {
@ -54,7 +55,8 @@ type Avatar = {
} }
type Options = { type Options = {
type: RoomType; type?: RoomType;
visibility: RoomVisibility;
isEncrypted?: boolean; isEncrypted?: boolean;
isFederationDisabled?: boolean; isFederationDisabled?: boolean;
name?: string; name?: string;
@ -62,25 +64,27 @@ type Options = {
invites?: string[]; invites?: string[];
avatar?: Avatar; avatar?: Avatar;
alias?: string; alias?: string;
powerLevelContentOverride?: any;
initialState?: any[];
} }
function defaultE2EEStatusForType(type: RoomType): boolean { function defaultE2EEStatusForType(type: RoomVisibility): boolean {
switch (type) { switch (type) {
case RoomType.DirectMessage: case RoomVisibility.DirectMessage:
case RoomType.Private: case RoomVisibility.Private:
return true; return true;
case RoomType.Public: case RoomVisibility.Public:
return false; return false;
} }
} }
function presetForType(type: RoomType): string { function presetForType(type: RoomVisibility): string {
switch (type) { switch (type) {
case RoomType.DirectMessage: case RoomVisibility.DirectMessage:
return "trusted_private_chat"; return "trusted_private_chat";
case RoomType.Private: case RoomVisibility.Private:
return "private_chat"; return "private_chat";
case RoomType.Public: case RoomVisibility.Public:
return "public_chat"; return "public_chat";
} }
} }
@ -103,7 +107,7 @@ export class RoomBeingCreated extends EventEmitter<{change: never}> {
log: ILogItem log: ILogItem
) { ) {
super(); super();
this.isEncrypted = options.isEncrypted === undefined ? defaultE2EEStatusForType(options.type) : options.isEncrypted; this.isEncrypted = options.isEncrypted === undefined ? defaultE2EEStatusForType(options.visibility) : options.isEncrypted;
if (options.name) { if (options.name) {
this._calculatedName = options.name; this._calculatedName = options.name;
} else { } else {
@ -130,8 +134,8 @@ export class RoomBeingCreated extends EventEmitter<{change: never}> {
attachment.applyToContent("url", avatarEventContent); attachment.applyToContent("url", avatarEventContent);
} }
const createOptions: CreateRoomPayload = { const createOptions: CreateRoomPayload = {
is_direct: this.options.type === RoomType.DirectMessage, is_direct: this.options.visibility === RoomVisibility.DirectMessage,
preset: presetForType(this.options.type), preset: presetForType(this.options.visibility),
initial_state: [] initial_state: []
}; };
if (this.options.name) { if (this.options.name) {
@ -146,10 +150,15 @@ export class RoomBeingCreated extends EventEmitter<{change: never}> {
if (this.options.alias) { if (this.options.alias) {
createOptions.room_alias_name = this.options.alias; createOptions.room_alias_name = this.options.alias;
} }
if (this.options.type !== undefined) {
let type: string | undefined = undefined;
if (this.options.type === RoomType.World) type = "org.matrix.msc3815.world";
if (this.options.type === RoomType.Profile) type = "org.matrix.msc3815.profile";
createOptions.creation_content = { type };
}
if (this.options.isFederationDisabled === true) { if (this.options.isFederationDisabled === true) {
createOptions.creation_content = { if (createOptions.creation_content === undefined) createOptions.creation_content = {};
"m.federate": false createOptions.creation_content["m.federate"] = false;
};
} }
if (this.isEncrypted) { if (this.isEncrypted) {
createOptions.initial_state.push(createRoomEncryptionEvent()); createOptions.initial_state.push(createRoomEncryptionEvent());
@ -161,6 +170,14 @@ export class RoomBeingCreated extends EventEmitter<{change: never}> {
content: avatarEventContent content: avatarEventContent
}); });
} }
if (this.options.powerLevelContentOverride) {
createOptions.power_level_content_override = this.options.powerLevelContentOverride;
}
if (this.options.initialState) {
createOptions.initial_state.push(...this.options.initialState);
}
const response = await hsApi.createRoom(createOptions, {log}).response(); const response = await hsApi.createRoom(createOptions, {log}).response();
this._roomId = response["room_id"]; this._roomId = response["room_id"];
} catch (err) { } catch (err) {
@ -221,7 +238,7 @@ export class RoomBeingCreated extends EventEmitter<{change: never}> {
} }
async adjustDirectMessageMapIfNeeded(user: User, storage: Storage, hsApi: HomeServerApi, log: ILogItem): Promise<void> { async adjustDirectMessageMapIfNeeded(user: User, storage: Storage, hsApi: HomeServerApi, log: ILogItem): Promise<void> {
if (!this.options.invites || this.options.type !== RoomType.DirectMessage) { if (!this.options.invites || this.options.visibility !== RoomVisibility.DirectMessage) {
return; return;
} }
const userId = this.options.invites[0]; const userId = this.options.invites[0];

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {MEGOLM_ALGORITHM} from "../e2ee/common.js"; import {MEGOLM_ALGORITHM} from "../e2ee/common.js";
import {iterateResponseStateEvents} from "./common";
function applyTimelineEntries(data, timelineEntries, isInitialSync, canMarkUnread, ownUserId) { function applyTimelineEntries(data, timelineEntries, isInitialSync, canMarkUnread, ownUserId) {
if (timelineEntries.length) { if (timelineEntries.length) {
@ -27,25 +27,6 @@ function applyTimelineEntries(data, timelineEntries, isInitialSync, canMarkUnrea
return data; return data;
} }
export function reduceStateEvents(roomResponse, callback, value) {
const stateEvents = roomResponse?.state?.events;
// state comes before timeline
if (Array.isArray(stateEvents)) {
value = stateEvents.reduce(callback, value);
}
const timelineEvents = roomResponse?.timeline?.events;
// and after that state events in the timeline
if (Array.isArray(timelineEvents)) {
value = timelineEvents.reduce((data, event) => {
if (typeof event.state_key === "string") {
value = callback(value, event);
}
return value;
}, value);
}
return value;
}
function applySyncResponse(data, roomResponse, membership, ownUserId) { function applySyncResponse(data, roomResponse, membership, ownUserId) {
if (roomResponse.summary) { if (roomResponse.summary) {
data = updateSummary(data, roomResponse.summary); data = updateSummary(data, roomResponse.summary);
@ -60,7 +41,9 @@ function applySyncResponse(data, roomResponse, membership, ownUserId) {
// process state events in state and in timeline. // process state events in state and in timeline.
// non-state events are handled by applyTimelineEntries // non-state events are handled by applyTimelineEntries
// so decryption is handled properly // so decryption is handled properly
data = reduceStateEvents(roomResponse, (data, event) => processStateEvent(data, event, ownUserId), data); iterateResponseStateEvents(roomResponse, event => {
data = processStateEvent(data, event, ownUserId);
});
const unreadNotifications = roomResponse.unread_notifications; const unreadNotifications = roomResponse.unread_notifications;
if (unreadNotifications) { if (unreadNotifications) {
data = processNotificationCounts(data, unreadNotifications); data = processNotificationCounts(data, unreadNotifications);
@ -99,6 +82,7 @@ export function processStateEvent(data, event, ownUserId) {
if (event.type === "m.room.create") { if (event.type === "m.room.create") {
data = data.cloneIfNeeded(); data = data.cloneIfNeeded();
data.lastMessageTimestamp = event.origin_server_ts; data.lastMessageTimestamp = event.origin_server_ts;
data.type = event.content?.type ?? null;
} else if (event.type === "m.room.encryption") { } else if (event.type === "m.room.encryption") {
const algorithm = event.content?.algorithm; const algorithm = event.content?.algorithm;
if (!data.encryption && algorithm === MEGOLM_ALGORITHM) { if (!data.encryption && algorithm === MEGOLM_ALGORITHM) {
@ -184,6 +168,7 @@ export class SummaryData {
constructor(copy, roomId) { constructor(copy, roomId) {
this.roomId = copy ? copy.roomId : roomId; this.roomId = copy ? copy.roomId : roomId;
this.name = copy ? copy.name : null; this.name = copy ? copy.name : null;
this.type = copy ? copy.type : null;
this.lastMessageTimestamp = copy ? copy.lastMessageTimestamp : null; this.lastMessageTimestamp = copy ? copy.lastMessageTimestamp : null;
this.isUnread = copy ? copy.isUnread : false; this.isUnread = copy ? copy.isUnread : false;
this.encryption = copy ? copy.encryption : null; this.encryption = copy ? copy.encryption : null;

View file

@ -14,6 +14,12 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import type {Room} from "./Room";
import type {StateEvent, TimelineEvent} from "../storage/types";
import type {Transaction} from "../storage/idb/Transaction";
import type {ILogItem} from "../../logging/types";
import type {MemberChange} from "./members/RoomMember";
export function getPrevContentFromStateEvent(event) { export function getPrevContentFromStateEvent(event) {
// where to look for prev_content is a bit of a mess, // where to look for prev_content is a bit of a mess,
// see https://matrix.to/#/!NasysSDfxKxZBzJJoE:matrix.org/$DvrAbZJiILkOmOIuRsNoHmh2v7UO5CWp_rYhlGk34fQ?via=matrix.org&via=pixie.town&via=amorgan.xyz // see https://matrix.to/#/!NasysSDfxKxZBzJJoE:matrix.org/$DvrAbZJiILkOmOIuRsNoHmh2v7UO5CWp_rYhlGk34fQ?via=matrix.org&via=pixie.town&via=amorgan.xyz
@ -35,8 +41,82 @@ export enum RoomStatus {
Archived = 1 << 5, Archived = 1 << 5,
} }
export enum RoomType { export enum RoomVisibility {
DirectMessage, DirectMessage,
Private, Private,
Public Public,
}
export enum RoomType {
World,
Profile,
}
type RoomResponse = {
state?: {
events?: Array<StateEvent>
},
timeline?: {
events?: Array<StateEvent>
}
}
/** iterates over any state events in a sync room response, in the order that they should be applied (from older to younger events) */
export function iterateResponseStateEvents(roomResponse: RoomResponse, callback: (StateEvent) => void) {
// first iterate over state events, they precede the timeline
const stateEvents = roomResponse.state?.events;
if (stateEvents) {
for (let i = 0; i < stateEvents.length; i++) {
callback(stateEvents[i]);
}
}
// now see if there are any state events within the timeline
let timelineEvents = roomResponse.timeline?.events;
if (timelineEvents) {
for (let i = 0; i < timelineEvents.length; i++) {
const event = timelineEvents[i];
if (typeof event.state_key === "string") {
callback(event);
}
}
}
}
export function tests() {
return {
"test iterateResponseStateEvents with both state and timeline sections": assert => {
const roomResponse = {
state: {
events: [
{type: "m.room.member", state_key: "1"},
{type: "m.room.member", state_key: "2", content: {a: 1}},
]
},
timeline: {
events: [
{type: "m.room.message"},
{type: "m.room.member", state_key: "3"},
{type: "m.room.message"},
{type: "m.room.member", state_key: "2", content: {a: 2}},
]
}
} as unknown as RoomResponse;
const expectedStateKeys = ["1", "2", "3", "2"];
const expectedAForMember2 = [1, 2];
iterateResponseStateEvents(roomResponse, event => {
assert.strictEqual(event.type, "m.room.member");
assert.strictEqual(expectedStateKeys.shift(), event.state_key);
if (event.state_key === "2") {
assert.strictEqual(expectedAForMember2.shift(), event.content.a);
}
});
assert.strictEqual(expectedStateKeys.length, 0);
assert.strictEqual(expectedAForMember2.length, 0);
},
"test iterateResponseStateEvents with empty response": assert => {
iterateResponseStateEvents({}, () => {
assert.fail("no events expected");
});
}
}
} }

View file

@ -0,0 +1,104 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {StateObserver} from "./types";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import {BaseObservableValue} from "../../../observable/value/BaseObservableValue";
/**
* Observable value for a state event with a given type and state key.
* Unsubscribes when last subscription is removed */
export class ObservedStateKeyValue extends BaseObservableValue<StateEvent | undefined> implements StateObserver {
private event?: StateEvent;
private removeCallback?: () => void;
constructor(private readonly type: string, private readonly stateKey: string) {
super();
}
/** @internal */
async load(roomId: string, txn: Transaction): Promise<void> {
this.event = (await txn.roomState.get(roomId, this.type, this.stateKey))?.event;
}
/** @internal */
handleStateEvent(event: StateEvent) {
if (event.type === this.type && event.state_key === this.stateKey) {
this.event = event;
this.emit(this.get());
}
}
get(): StateEvent | undefined {
return this.event;
}
setRemoveCallback(callback: () => void) {
this.removeCallback = callback;
}
onUnsubscribeLast() {
this.removeCallback?.();
}
}
import {createMockStorage} from "../../../mocks/Storage";
export async function tests() {
return {
"test load and update": async assert => {
const storage = await createMockStorage();
const writeTxn = await storage.readWriteTxn([storage.storeNames.roomState]);
writeTxn.roomState.set("!abc", {
event_id: "$abc",
type: "m.room.member",
state_key: "@alice",
sender: "@alice",
origin_server_ts: 5,
content: {}
});
await writeTxn.complete();
const txn = await storage.readTxn([storage.storeNames.roomState]);
const value = new ObservedStateKeyValue("m.room.member", "@alice");
await value.load("!abc", txn);
const updates: Array<StateEvent | undefined> = [];
assert.strictEqual(value.get()?.origin_server_ts, 5);
const unsubscribe = value.subscribe(value => updates.push(value));
value.handleStateEvent({
event_id: "$abc",
type: "m.room.member",
state_key: "@bob",
sender: "@alice",
origin_server_ts: 10,
content: {}
});
assert.strictEqual(updates.length, 0);
value.handleStateEvent({
event_id: "$abc",
type: "m.room.member",
state_key: "@alice",
sender: "@alice",
origin_server_ts: 10,
content: {}
});
assert.strictEqual(updates.length, 1);
assert.strictEqual(updates[0]?.origin_server_ts, 10);
let removed = false;
value.setRemoveCallback(() => removed = true);
unsubscribe();
assert(removed);
}
}
}

View file

@ -0,0 +1,53 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {StateObserver} from "./types";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import {ObservableMap} from "../../../observable/map/ObservableMap";
/**
* Observable map for a given type with state keys as map keys.
* Unsubscribes when last subscription is removed */
export class ObservedStateTypeMap extends ObservableMap<string, StateEvent> implements StateObserver {
private removeCallback?: () => void;
constructor(private readonly type: string) {
super();
}
/** @internal */
async load(roomId: string, txn: Transaction): Promise<void> {
const events = await txn.roomState.getAllForType(roomId, this.type);
for (let i = 0; i < events.length; ++i) {
const {event} = events[i];
this.add(event.state_key, event);
}
}
/** @internal */
handleStateEvent(event: StateEvent) {
if (event.type === this.type) {
this.set(event.state_key, event);
}
}
setRemoveCallback(callback: () => void) {
this.removeCallback = callback;
}
onUnsubscribeLast() {
this.removeCallback?.();
}
}

View file

@ -0,0 +1,40 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {ILogItem} from "../../../logging/types";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import type {Room} from "../Room";
import type {MemberChange} from "../members/RoomMember";
import type {RoomStateHandler} from "./types";
import type {MemberSync} from "../timeline/persistence/MemberWriter.js";
import {BaseObservable} from "../../../observable/BaseObservable";
/** keeps track of all handlers registered with Session.observeRoomState */
export class RoomStateHandlerSet extends BaseObservable<RoomStateHandler> implements RoomStateHandler {
async handleRoomState(room: Room, stateEvent: StateEvent, memberSync: MemberSync, txn: Transaction, log: ILogItem): Promise<void> {
const promises: Promise<void>[] = [];
for(let h of this._handlers) {
promises.push(h.handleRoomState(room, stateEvent, memberSync, txn, log));
}
await Promise.all(promises);
}
updateRoomMembers(room: Room, memberChanges: Map<string, MemberChange>) {
for(let h of this._handlers) {
h.updateRoomMembers(room, memberChanges);
}
}
}

View file

@ -0,0 +1,39 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {Room} from "../Room";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import type {ILogItem} from "../../../logging/types";
import type {MemberChange} from "../members/RoomMember";
import type {MemberSync} from "../timeline/persistence/MemberWriter";
/** used for Session.observeRoomState, which observes in all room, but without loading from storage
* It receives the sync write transaction, so other stores can be updated as part of the same transaction. */
export interface RoomStateHandler {
handleRoomState(room: Room, stateEvent: StateEvent, memberSync: MemberSync, syncWriteTxn: Transaction, log: ILogItem): Promise<void>;
updateRoomMembers(room: Room, memberChanges: Map<string, MemberChange>): void;
}
/**
* used for Room.observeStateType and Room.observeStateTypeAndKey
* @internal
* */
export interface StateObserver {
handleStateEvent(event: StateEvent);
load(roomId: string, txn: Transaction): Promise<void>;
setRemoveCallback(callback: () => void);
}

View file

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {SortedArray, AsyncMappedList, ConcatList, ObservableArray} from "../../../observable/index.js"; import {SortedArray, AsyncMappedList, ConcatList, ObservableArray} from "../../../observable/index";
import {Disposables} from "../../../utils/Disposables"; import {Disposables} from "../../../utils/Disposables";
import {Direction} from "./Direction"; import {Direction} from "./Direction";
import {TimelineReader} from "./persistence/TimelineReader.js"; import {TimelineReader} from "./persistence/TimelineReader.js";

View file

@ -56,7 +56,11 @@ export class MemberWriter {
} }
} }
class MemberSync { /** Represents the member changes in a given sync.
* Used to write the changes to storage and historical member
* information for events in the same sync.
**/
export class MemberSync {
constructor(memberWriter, stateEvents, timelineEvents, hasFetchedMembers) { constructor(memberWriter, stateEvents, timelineEvents, hasFetchedMembers) {
this._memberWriter = memberWriter; this._memberWriter = memberWriter;
this._timelineEvents = timelineEvents; this._timelineEvents = timelineEvents;

View file

@ -244,7 +244,7 @@ export class SyncWriter {
const {currentKey, entries, updatedEntries} = const {currentKey, entries, updatedEntries} =
await this._writeTimeline(timelineEvents, timeline, memberSync, this._lastLiveKey, txn, log); await this._writeTimeline(timelineEvents, timeline, memberSync, this._lastLiveKey, txn, log);
const memberChanges = await memberSync.write(txn); const memberChanges = await memberSync.write(txn);
return {entries, updatedEntries, newLiveKey: currentKey, memberChanges}; return {entries, updatedEntries, newLiveKey: currentKey, memberChanges, memberSync};
} }
afterSync(newLiveKey) { afterSync(newLiveKey) {

View file

@ -33,6 +33,7 @@ export enum StoreNames {
groupSessionDecryptions = "groupSessionDecryptions", groupSessionDecryptions = "groupSessionDecryptions",
operations = "operations", operations = "operations",
accountData = "accountData", accountData = "accountData",
calls = "calls"
} }
export const STORE_NAMES: Readonly<StoreNames[]> = Object.values(StoreNames); export const STORE_NAMES: Readonly<StoreNames[]> = Object.values(StoreNames);

View file

@ -67,7 +67,7 @@ export class StorageFactory {
requestPersistedStorage().then(persisted => { requestPersistedStorage().then(persisted => {
// Firefox lies here though, and returns true even if the user denied the request // Firefox lies here though, and returns true even if the user denied the request
if (!persisted) { if (!persisted) {
console.warn("no persisted storage, database can be evicted by browser"); log.log("no persisted storage, database can be evicted by browser", log.level.Warn);
} }
}); });

View file

@ -36,6 +36,7 @@ import {OutboundGroupSessionStore} from "./stores/OutboundGroupSessionStore";
import {GroupSessionDecryptionStore} from "./stores/GroupSessionDecryptionStore"; import {GroupSessionDecryptionStore} from "./stores/GroupSessionDecryptionStore";
import {OperationStore} from "./stores/OperationStore"; import {OperationStore} from "./stores/OperationStore";
import {AccountDataStore} from "./stores/AccountDataStore"; import {AccountDataStore} from "./stores/AccountDataStore";
import {CallStore} from "./stores/CallStore";
import type {ILogger, ILogItem} from "../../../logging/types"; import type {ILogger, ILogItem} from "../../../logging/types";
export type IDBKey = IDBValidKey | IDBKeyRange; export type IDBKey = IDBValidKey | IDBKeyRange;
@ -168,6 +169,10 @@ export class Transaction {
return this._store(StoreNames.accountData, idbStore => new AccountDataStore(idbStore)); return this._store(StoreNames.accountData, idbStore => new AccountDataStore(idbStore));
} }
get calls(): CallStore {
return this._store(StoreNames.calls, idbStore => new CallStore(idbStore));
}
async complete(log?: ILogItem): Promise<void> { async complete(log?: ILogItem): Promise<void> {
try { try {
await txnAsPromise(this._txn); await txnAsPromise(this._txn);

View file

@ -34,7 +34,8 @@ export const schema: MigrationFunc[] = [
backupAndRestoreE2EEAccountToLocalStorage, backupAndRestoreE2EEAccountToLocalStorage,
clearAllStores, clearAllStores,
addInboundSessionBackupIndex, addInboundSessionBackupIndex,
migrateBackupStatus migrateBackupStatus,
createCallStore
]; ];
// TODO: how to deal with git merge conflicts of this array? // TODO: how to deal with git merge conflicts of this array?
@ -309,3 +310,8 @@ async function migrateBackupStatus(db: IDBDatabase, txn: IDBTransaction, localSt
log.set("countWithoutSession", countWithoutSession); log.set("countWithoutSession", countWithoutSession);
log.set("countWithSession", countWithSession); log.set("countWithSession", countWithSession);
} }
//v17 create calls store
function createCallStore(db: IDBDatabase) : void {
db.createObjectStore("calls", {keyPath: "key"});
}

View file

@ -0,0 +1,83 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {Store} from "../Store";
import {StateEvent} from "../../types";
import {MIN_UNICODE, MAX_UNICODE} from "./common";
function encodeKey(intent: string, roomId: string, callId: string) {
return `${intent}|${roomId}|${callId}`;
}
function decodeStorageEntry(storageEntry: CallStorageEntry): CallEntry {
const [intent, roomId, callId] = storageEntry.key.split("|");
return {intent, roomId, callId, timestamp: storageEntry.timestamp};
}
export interface CallEntry {
intent: string;
roomId: string;
callId: string;
timestamp: number;
}
type CallStorageEntry = {
key: string;
timestamp: number;
}
export class CallStore {
private _callStore: Store<CallStorageEntry>;
constructor(idbStore: Store<CallStorageEntry>) {
this._callStore = idbStore;
}
async getByIntent(intent: string): Promise<CallEntry[]> {
const range = this._callStore.IDBKeyRange.bound(
encodeKey(intent, MIN_UNICODE, MIN_UNICODE),
encodeKey(intent, MAX_UNICODE, MAX_UNICODE),
true,
true
);
const storageEntries = await this._callStore.selectAll(range);
return storageEntries.map(e => decodeStorageEntry(e));
}
async getByIntentAndRoom(intent: string, roomId: string): Promise<CallEntry[]> {
const range = this._callStore.IDBKeyRange.bound(
encodeKey(intent, roomId, MIN_UNICODE),
encodeKey(intent, roomId, MAX_UNICODE),
true,
true
);
const storageEntries = await this._callStore.selectAll(range);
return storageEntries.map(e => decodeStorageEntry(e));
}
add(entry: CallEntry) {
const storageEntry: CallStorageEntry = {
key: encodeKey(entry.intent, entry.roomId, entry.callId),
timestamp: entry.timestamp
};
this._callStore.add(storageEntry);
}
remove(intent: string, roomId: string, callId: string): void {
this._callStore.delete(encodeKey(intent, roomId, callId));
}
}

View file

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {MAX_UNICODE} from "./common"; import {MIN_UNICODE, MAX_UNICODE} from "./common";
import {Store} from "../Store"; import {Store} from "../Store";
import {StateEvent} from "../../types"; import {StateEvent} from "../../types";
@ -41,6 +41,16 @@ export class RoomStateStore {
return this._roomStateStore.get(key); return this._roomStateStore.get(key);
} }
getAllForType(roomId: string, type: string): Promise<RoomStateEntry[]> {
const range = this._roomStateStore.IDBKeyRange.bound(
encodeKey(roomId, type, MIN_UNICODE),
encodeKey(roomId, type, MAX_UNICODE),
true,
true
);
return this._roomStateStore.selectAll(range);
}
set(roomId: string, event: StateEvent): void { set(roomId: string, event: StateEvent): void {
const key = encodeKey(roomId, event.type, event.state_key); const key = encodeKey(roomId, event.type, event.state_key);
const entry = {roomId, event, key}; const entry = {roomId, event, key};

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {ObservableValue} from "../observable/ObservableValue"; import {ObservableValue} from "../observable/value/ObservableValue";
class Timeout { class Timeout {
constructor(elapsed, ms) { constructor(elapsed, ms) {

View file

@ -1,248 +0,0 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AbortError} from "../utils/error";
import {BaseObservable} from "./BaseObservable";
import type {SubscriptionHandle} from "./BaseObservable";
// like an EventEmitter, but doesn't have an event type
export abstract class BaseObservableValue<T> extends BaseObservable<(value: T) => void> {
emit(argument: T) {
for (const h of this._handlers) {
h(argument);
}
}
abstract get(): T;
waitFor(predicate: (value: T) => boolean): IWaitHandle<T> {
if (predicate(this.get())) {
return new ResolvedWaitForHandle(Promise.resolve(this.get()));
} else {
return new WaitForHandle(this, predicate);
}
}
flatMap<C>(mapper: (value: T) => (BaseObservableValue<C> | undefined)): BaseObservableValue<C | undefined> {
return new FlatMapObservableValue<T, C>(this, mapper);
}
}
interface IWaitHandle<T> {
promise: Promise<T>;
dispose(): void;
}
class WaitForHandle<T> implements IWaitHandle<T> {
private _promise: Promise<T>
private _reject: ((reason?: any) => void) | null;
private _subscription: (() => void) | null;
constructor(observable: BaseObservableValue<T>, predicate: (value: T) => boolean) {
this._promise = new Promise((resolve, reject) => {
this._reject = reject;
this._subscription = observable.subscribe(v => {
if (predicate(v)) {
this._reject = null;
resolve(v);
this.dispose();
}
});
});
}
get promise(): Promise<T> {
return this._promise;
}
dispose() {
if (this._subscription) {
this._subscription();
this._subscription = null;
}
if (this._reject) {
this._reject(new AbortError());
this._reject = null;
}
}
}
class ResolvedWaitForHandle<T> implements IWaitHandle<T> {
constructor(public promise: Promise<T>) {}
dispose() {}
}
export class ObservableValue<T> extends BaseObservableValue<T> {
private _value: T;
constructor(initialValue: T) {
super();
this._value = initialValue;
}
get(): T {
return this._value;
}
set(value: T): void {
if (value !== this._value) {
this._value = value;
this.emit(this._value);
}
}
}
export class RetainedObservableValue<T> extends ObservableValue<T> {
private _freeCallback: () => void;
constructor(initialValue: T, freeCallback: () => void) {
super(initialValue);
this._freeCallback = freeCallback;
}
onUnsubscribeLast() {
super.onUnsubscribeLast();
this._freeCallback();
}
}
export class FlatMapObservableValue<P, C> extends BaseObservableValue<C | undefined> {
private sourceSubscription?: SubscriptionHandle;
private targetSubscription?: SubscriptionHandle;
constructor(
private readonly source: BaseObservableValue<P>,
private readonly mapper: (value: P) => (BaseObservableValue<C> | undefined)
) {
super();
}
onUnsubscribeLast() {
super.onUnsubscribeLast();
this.sourceSubscription = this.sourceSubscription!();
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
onSubscribeFirst() {
super.onSubscribeFirst();
this.sourceSubscription = this.source.subscribe(() => {
this.updateTargetSubscription();
this.emit(this.get());
});
this.updateTargetSubscription();
}
private updateTargetSubscription() {
const sourceValue = this.source.get();
if (sourceValue) {
const target = this.mapper(sourceValue);
if (target) {
if (!this.targetSubscription) {
this.targetSubscription = target.subscribe(() => this.emit(this.get()));
}
return;
}
}
// if no sourceValue or target
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
get(): C | undefined {
const sourceValue = this.source.get();
if (!sourceValue) {
return undefined;
}
const mapped = this.mapper(sourceValue);
return mapped?.get();
}
}
export function tests() {
return {
"set emits an update": assert => {
const a = new ObservableValue<number>(0);
let fired = false;
const subscription = a.subscribe(v => {
fired = true;
assert.strictEqual(v, 5);
});
a.set(5);
assert(fired);
subscription();
},
"set doesn't emit if value hasn't changed": assert => {
const a = new ObservableValue(5);
let fired = false;
const subscription = a.subscribe(() => {
fired = true;
});
a.set(5);
a.set(5);
assert(!fired);
subscription();
},
"waitFor promise resolves on matching update": async assert => {
const a = new ObservableValue(5);
const handle = a.waitFor(v => v === 6);
Promise.resolve().then(() => {
a.set(6);
});
await handle.promise;
assert.strictEqual(a.get(), 6);
},
"waitFor promise rejects when disposed": async assert => {
const a = new ObservableValue<number>(0);
const handle = a.waitFor(() => false);
Promise.resolve().then(() => {
handle.dispose();
});
await assert.rejects(handle.promise, AbortError);
},
"flatMap.get": assert => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const countProxy = a.flatMap(a => a!.count);
assert.strictEqual(countProxy.get(), undefined);
const count = new ObservableValue<number>(0);
a.set({count});
assert.strictEqual(countProxy.get(), 0);
},
"flatMap update from source": assert => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
a.flatMap(a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
assert.deepEqual(updates, [0]);
},
"flatMap update from target": assert => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
a.flatMap(a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
count.set(5);
assert.deepEqual(updates, [0, 5]);
}
}
}

View file

@ -46,3 +46,12 @@ Object.assign(BaseObservableMap.prototype, {
return new JoinedMap([this].concat(otherMaps)); return new JoinedMap([this].concat(otherMaps));
} }
}); });
declare module "./map/BaseObservableMap" {
interface BaseObservableMap<K, V> {
sortValues(comparator: (a: V, b: V) => number): SortedMapList<V>;
mapValues<M>(mapper: (V, emitSpontaneousUpdate: (params: any) => void) => M, updater: (mappedValue: M, params: any, value: V) => void): MappedMap<K, M>;
filterValues(filter: (V, K) => boolean): FilteredMap<K, V>;
join(...otherMaps: BaseObservableMap<K, V>[]): JoinedMap<K, V>;
}
}

View file

@ -80,15 +80,15 @@ export class ObservableMap<K, V> extends BaseObservableMap<K, V> {
return this._values.size; return this._values.size;
} }
[Symbol.iterator](): Iterator<[K, V]> { [Symbol.iterator](): IterableIterator<[K, V]> {
return this._values.entries(); return this._values.entries();
} }
values(): Iterator<V> { values(): IterableIterator<V> {
return this._values.values(); return this._values.values();
} }
keys(): Iterator<K> { keys(): IterableIterator<K> {
return this._values.keys(); return this._values.keys();
} }
} }

View file

@ -0,0 +1,53 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableMap} from "./BaseObservableMap";
import {BaseObservableValue} from "../value/BaseObservableValue";
import {SubscriptionHandle} from "../BaseObservable";
export class ObservableValueMap<K, V> extends BaseObservableMap<K, V> {
private subscription?: SubscriptionHandle;
constructor(private readonly key: K, private readonly observableValue: BaseObservableValue<V>) {
super();
}
onSubscribeFirst() {
this.subscription = this.observableValue.subscribe(value => {
this.emitUpdate(this.key, value, undefined);
});
super.onSubscribeFirst();
}
onUnsubscribeLast() {
this.subscription!();
super.onUnsubscribeLast();
}
*[Symbol.iterator](): Iterator<[K, V]> {
yield [this.key, this.observableValue.get()];
}
get size(): number {
return 1;
}
get(key: K): V | undefined {
if (key == this.key) {
return this.observableValue.get();
}
}
}

View file

@ -0,0 +1,83 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AbortError} from "../../utils/error";
import {BaseObservable} from "../BaseObservable";
import type {SubscriptionHandle} from "../BaseObservable";
import {FlatMapObservableValue} from "./FlatMapObservableValue";
// like an EventEmitter, but doesn't have an event type
export abstract class BaseObservableValue<T> extends BaseObservable<(value: T) => void> {
emit(argument: T) {
for (const h of this._handlers) {
h(argument);
}
}
abstract get(): T;
waitFor(predicate: (value: T) => boolean): IWaitHandle<T> {
if (predicate(this.get())) {
return new ResolvedWaitForHandle(Promise.resolve(this.get()));
} else {
return new WaitForHandle(this, predicate);
}
}
}
interface IWaitHandle<T> {
promise: Promise<T>;
dispose(): void;
}
class WaitForHandle<T> implements IWaitHandle<T> {
private _promise: Promise<T>
private _reject: ((reason?: any) => void) | null;
private _subscription: (() => void) | null;
constructor(observable: BaseObservableValue<T>, predicate: (value: T) => boolean) {
this._promise = new Promise((resolve, reject) => {
this._reject = reject;
this._subscription = observable.subscribe(v => {
if (predicate(v)) {
this._reject = null;
resolve(v);
this.dispose();
}
});
});
}
get promise(): Promise<T> {
return this._promise;
}
dispose() {
if (this._subscription) {
this._subscription();
this._subscription = null;
}
if (this._reject) {
this._reject(new AbortError());
this._reject = null;
}
}
}
class ResolvedWaitForHandle<T> implements IWaitHandle<T> {
constructor(public promise: Promise<T>) {}
dispose() {}
}

View file

@ -0,0 +1,45 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./BaseObservableValue";
import {EventEmitter} from "../../utils/EventEmitter";
export class EventObservableValue<T, V extends EventEmitter<T>> extends BaseObservableValue<V> {
private eventSubscription: () => void;
constructor(
private readonly value: V,
private readonly eventName: keyof T
) {
super();
}
onSubscribeFirst(): void {
this.eventSubscription = this.value.disposableOn(this.eventName, () => {
this.emit(this.value);
});
super.onSubscribeFirst();
}
onUnsubscribeLast(): void {
this.eventSubscription!();
super.onUnsubscribeLast();
}
get(): V {
return this.value;
}
}

View file

@ -0,0 +1,109 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./BaseObservableValue";
import {SubscriptionHandle} from "../BaseObservable";
export class FlatMapObservableValue<P, C> extends BaseObservableValue<C | undefined> {
private sourceSubscription?: SubscriptionHandle;
private targetSubscription?: SubscriptionHandle;
constructor(
private readonly source: BaseObservableValue<P>,
private readonly mapper: (value: P) => (BaseObservableValue<C> | undefined)
) {
super();
}
onUnsubscribeLast() {
super.onUnsubscribeLast();
this.sourceSubscription = this.sourceSubscription!();
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
onSubscribeFirst() {
super.onSubscribeFirst();
this.sourceSubscription = this.source.subscribe(() => {
this.updateTargetSubscription();
this.emit(this.get());
});
this.updateTargetSubscription();
}
private updateTargetSubscription() {
const sourceValue = this.source.get();
if (sourceValue) {
const target = this.mapper(sourceValue);
if (target) {
if (!this.targetSubscription) {
this.targetSubscription = target.subscribe(() => this.emit(this.get()));
}
return;
}
}
// if no sourceValue or target
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
get(): C | undefined {
const sourceValue = this.source.get();
if (!sourceValue) {
return undefined;
}
const mapped = this.mapper(sourceValue);
return mapped?.get();
}
}
import {ObservableValue} from "./ObservableValue";
export function tests() {
return {
"flatMap.get": assert => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const countProxy = new FlatMapObservableValue(a, a => a!.count);
assert.strictEqual(countProxy.get(), undefined);
const count = new ObservableValue<number>(0);
a.set({count});
assert.strictEqual(countProxy.get(), 0);
},
"flatMap update from source": assert => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
new FlatMapObservableValue(a, a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
assert.deepEqual(updates, [0]);
},
"flatMap update from target": assert => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
new FlatMapObservableValue(a, a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
count.set(5);
assert.deepEqual(updates, [0, 5]);
}
}
}

View file

@ -0,0 +1,82 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AbortError} from "../../utils/error";
import {BaseObservableValue} from "./BaseObservableValue";
export class ObservableValue<T> extends BaseObservableValue<T> {
private _value: T;
constructor(initialValue: T) {
super();
this._value = initialValue;
}
get(): T {
return this._value;
}
set(value: T): void {
if (value !== this._value) {
this._value = value;
this.emit(this._value);
}
}
}
export function tests() {
return {
"set emits an update": assert => {
const a = new ObservableValue<number>(0);
let fired = false;
const subscription = a.subscribe(v => {
fired = true;
assert.strictEqual(v, 5);
});
a.set(5);
assert(fired);
subscription();
},
"set doesn't emit if value hasn't changed": assert => {
const a = new ObservableValue(5);
let fired = false;
const subscription = a.subscribe(() => {
fired = true;
});
a.set(5);
a.set(5);
assert(!fired);
subscription();
},
"waitFor promise resolves on matching update": async assert => {
const a = new ObservableValue(5);
const handle = a.waitFor(v => v === 6);
Promise.resolve().then(() => {
a.set(6);
});
await handle.promise;
assert.strictEqual(a.get(), 6);
},
"waitFor promise rejects when disposed": async assert => {
const a = new ObservableValue<number>(0);
const handle = a.waitFor(() => false);
Promise.resolve().then(() => {
handle.dispose();
});
await assert.rejects(handle.promise, AbortError);
}
}
}

View file

@ -0,0 +1,89 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./BaseObservableValue";
import {BaseObservableMap, IMapObserver} from "../map/BaseObservableMap";
import {SubscriptionHandle} from "../BaseObservable";
function pickLowestKey<K>(currentKey: K, newKey: K): boolean {
return newKey < currentKey;
}
export class PickMapObservableValue<K, V> extends BaseObservableValue<V | undefined> implements IMapObserver<K, V>{
private key?: K;
private mapSubscription?: SubscriptionHandle;
constructor(
private readonly map: BaseObservableMap<K, V>,
private readonly pickKey: (currentKey: K, newKey: K) => boolean = pickLowestKey
) {
super();
}
private updateKey(newKey: K): boolean {
if (this.key === undefined || this.pickKey(this.key, newKey)) {
this.key = newKey;
return true;
}
return false;
}
onReset(): void {
this.key = undefined;
this.emit(this.get());
}
onAdd(key: K, value:V): void {
if (this.updateKey(key)) {
this.emit(this.get());
}
}
onUpdate(key: K, value: V, params: any): void {
this.emit(this.get());
}
onRemove(key: K, value: V): void {
if (key === this.key) {
this.key = undefined;
// try to see if there is another key that fullfills pickKey
for (const [key] of this.map) {
this.updateKey(key);
}
this.emit(this.get());
}
}
onSubscribeFirst(): void {
this.mapSubscription = this.map.subscribe(this);
for (const [key] of this.map) {
this.updateKey(key);
}
}
onUnsubscribeLast(): void {
this.mapSubscription!();
this.key = undefined;
}
get(): V | undefined {
if (this.key !== undefined) {
return this.map.get(this.key);
}
return undefined;
}
}

View file

@ -0,0 +1,31 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableValue} from "./ObservableValue";
export class RetainedObservableValue<T> extends ObservableValue<T> {
private _freeCallback: () => void;
constructor(initialValue: T, freeCallback: () => void) {
super(initialValue);
this._freeCallback = freeCallback;
}
onUnsubscribeLast() {
super.onUnsubscribeLast();
this._freeCallback();
}
}

View file

@ -0,0 +1,81 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
export interface Event {}
export interface MediaDevices {
// filter out audiooutput
enumerate(): Promise<MediaDeviceInfo[]>;
// to assign to a video element, we downcast to WrappedTrack and use the stream property.
getMediaTracks(audio: true | MediaDeviceInfo, video: boolean | MediaDeviceInfo): Promise<Stream>;
getScreenShareTrack(): Promise<Stream | undefined>;
createVolumeMeasurer(stream: Stream, callback: () => void): VolumeMeasurer;
}
// Typescript definitions derived from https://github.com/microsoft/TypeScript/blob/main/lib/lib.dom.d.ts
/*! *****************************************************************************
Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at http://www.apache.org/licenses/LICENSE-2.0
THIS CODE IS PROVIDED ON AN *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,
MERCHANTABLITY OR NON-INFRINGEMENT.
See the Apache Version 2.0 License for specific language governing permissions
and limitations under the License.
***************************************************************************** */
export interface StreamTrackEvent extends Event {
readonly track: Track;
}
export interface StreamEventMap {
"addtrack": StreamTrackEvent;
"removetrack": StreamTrackEvent;
}
export interface Stream {
getTracks(): ReadonlyArray<Track>;
getAudioTracks(): ReadonlyArray<Track>;
getVideoTracks(): ReadonlyArray<Track>;
readonly id: string;
clone(): Stream;
addEventListener<K extends keyof StreamEventMap>(type: K, listener: (this: Stream, ev: StreamEventMap[K]) => any, options?: boolean | AddEventListenerOptions): void;
removeEventListener<K extends keyof StreamEventMap>(type: K, listener: (this: Stream, ev: StreamEventMap[K]) => any, options?: boolean | EventListenerOptions): void;
addTrack(track: Track);
removeTrack(track: Track);
}
export enum TrackKind {
Video = "video",
Audio = "audio"
}
export interface Track {
readonly kind: TrackKind;
readonly label: string;
readonly id: string;
enabled: boolean;
// getSettings(): MediaTrackSettings;
stop(): void;
}
export interface VolumeMeasurer {
get isSpeaking(): boolean;
setSpeakingThreshold(threshold: number): void;
stop();
}

View file

@ -0,0 +1,175 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {Track, Stream, Event} from "./MediaDevices";
import {SDPStreamMetadataPurpose} from "../../matrix/calls/callEventTypes";
export interface WebRTC {
createPeerConnection(forceTURN: boolean, turnServers: RTCIceServer[], iceCandidatePoolSize: number): PeerConnection;
prepareSenderForPurpose(peerConnection: PeerConnection, sender: Sender, purpose: SDPStreamMetadataPurpose): void;
}
// Typescript definitions derived from https://github.com/microsoft/TypeScript/blob/main/lib/lib.dom.d.ts
/*! *****************************************************************************
Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at http://www.apache.org/licenses/LICENSE-2.0
THIS CODE IS PROVIDED ON AN *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,
MERCHANTABLITY OR NON-INFRINGEMENT.
See the Apache Version 2.0 License for specific language governing permissions
and limitations under the License.
***************************************************************************** */
export interface DataChannelEventMap {
"bufferedamountlow": Event;
"close": Event;
"error": Event;
"message": MessageEvent;
"open": Event;
}
export interface DataChannel {
binaryType: BinaryType;
readonly id: number | null;
readonly label: string;
readonly negotiated: boolean;
readonly readyState: DataChannelState;
close(): void;
send(data: string): void;
send(data: Blob): void;
send(data: ArrayBuffer): void;
send(data: ArrayBufferView): void;
addEventListener<K extends keyof DataChannelEventMap>(type: K, listener: (this: DataChannel, ev: DataChannelEventMap[K]) => any, options?: boolean | AddEventListenerOptions): void;
removeEventListener<K extends keyof DataChannelEventMap>(type: K, listener: (this: DataChannel, ev: DataChannelEventMap[K]) => any, options?: boolean | EventListenerOptions): void;
}
export interface DataChannelInit {
id?: number;
maxPacketLifeTime?: number;
maxRetransmits?: number;
negotiated?: boolean;
ordered?: boolean;
protocol?: string;
}
export interface DataChannelEvent extends Event {
readonly channel: DataChannel;
}
export interface PeerConnectionIceEvent extends Event {
readonly candidate: RTCIceCandidate | null;
}
export interface TrackEvent extends Event {
readonly receiver: Receiver;
readonly streams: ReadonlyArray<Stream>;
readonly track: Track;
readonly transceiver: Transceiver;
}
export interface PeerConnectionEventMap {
"connectionstatechange": Event;
"datachannel": DataChannelEvent;
"icecandidate": PeerConnectionIceEvent;
"iceconnectionstatechange": Event;
"icegatheringstatechange": Event;
"negotiationneeded": Event;
"signalingstatechange": Event;
"track": TrackEvent;
}
export type DataChannelState = "closed" | "closing" | "connecting" | "open";
export type IceConnectionState = "checking" | "closed" | "completed" | "connected" | "disconnected" | "failed" | "new";
export type PeerConnectionState = "closed" | "connected" | "connecting" | "disconnected" | "failed" | "new";
export type SignalingState = "closed" | "have-local-offer" | "have-local-pranswer" | "have-remote-offer" | "have-remote-pranswer" | "stable";
export type IceGatheringState = "complete" | "gathering" | "new";
export type SdpType = "answer" | "offer" | "pranswer" | "rollback";
export type TransceiverDirection = "inactive" | "recvonly" | "sendonly" | "sendrecv" | "stopped";
export interface SessionDescription {
readonly sdp: string;
readonly type: SdpType;
}
export interface AnswerOptions {}
export interface OfferOptions {
iceRestart?: boolean;
offerToReceiveAudio?: boolean;
offerToReceiveVideo?: boolean;
}
export interface SessionDescriptionInit {
sdp?: string;
type: SdpType;
}
export interface LocalSessionDescriptionInit {
sdp?: string;
type?: SdpType;
}
/** A WebRTC connection between the local computer and a remote peer. It provides methods to connect to a remote peer, maintain and monitor the connection, and close the connection once it's no longer needed. */
export interface PeerConnection {
readonly connectionState: PeerConnectionState;
readonly iceConnectionState: IceConnectionState;
readonly iceGatheringState: IceGatheringState;
readonly localDescription: SessionDescription | null;
readonly remoteDescription: SessionDescription | null;
readonly signalingState: SignalingState;
addIceCandidate(candidate?: RTCIceCandidateInit): Promise<void>;
addTrack(track: Track, ...streams: Stream[]): Sender;
close(): void;
createAnswer(options?: AnswerOptions): Promise<SessionDescriptionInit>;
createDataChannel(label: string, dataChannelDict?: DataChannelInit): DataChannel;
createOffer(options?: OfferOptions): Promise<SessionDescriptionInit>;
getReceivers(): Receiver[];
getSenders(): Sender[];
getTransceivers(): Transceiver[];
removeTrack(sender: Sender): void;
restartIce(): void;
setLocalDescription(description?: LocalSessionDescriptionInit): Promise<void>;
setRemoteDescription(description: SessionDescriptionInit): Promise<void>;
addEventListener<K extends keyof PeerConnectionEventMap>(type: K, listener: (this: PeerConnection, ev: PeerConnectionEventMap[K]) => any, options?: boolean | AddEventListenerOptions): void;
removeEventListener<K extends keyof PeerConnectionEventMap>(type: K, listener: (this: PeerConnection, ev: PeerConnectionEventMap[K]) => any, options?: boolean | EventListenerOptions): void;
getStats(selector?: Track | null): Promise<StatsReport>;
}
interface StatsReport {
forEach(callbackfn: (value: any, key: string, parent: StatsReport) => void, thisArg?: any): void;
}
export interface Receiver {
readonly track: Track;
}
export interface Sender {
readonly track: Track | null;
replaceTrack(withTrack: Track | null): Promise<void>;
}
export interface Transceiver {
readonly currentDirection: TransceiverDirection | null;
direction: TransceiverDirection;
readonly mid: string | null;
readonly receiver: Receiver;
readonly sender: Sender;
stop(): void;
}

View file

@ -43,3 +43,11 @@ export type File = {
readonly name: string; readonly name: string;
readonly blob: IBlobHandle; readonly blob: IBlobHandle;
} }
export interface Timeout {
elapsed(): Promise<void>;
abort(): void;
dispose(): void;
};
export type TimeoutCreator = (timeout: number) => Timeout;

View file

@ -21,8 +21,9 @@ import {SessionInfoStorage} from "../../matrix/sessioninfo/localstorage/SessionI
import {SettingsStorage} from "./dom/SettingsStorage.js"; import {SettingsStorage} from "./dom/SettingsStorage.js";
import {Encoding} from "./utils/Encoding.js"; import {Encoding} from "./utils/Encoding.js";
import {OlmWorker} from "../../matrix/e2ee/OlmWorker.js"; import {OlmWorker} from "../../matrix/e2ee/OlmWorker.js";
import {IDBLogger} from "../../logging/IDBLogger"; import {IDBLogPersister} from "../../logging/IDBLogPersister";
import {ConsoleLogger} from "../../logging/ConsoleLogger"; import {ConsoleReporter} from "../../logging/ConsoleReporter";
import {Logger} from "../../logging/Logger";
import {RootView} from "./ui/RootView.js"; import {RootView} from "./ui/RootView.js";
import {Clock} from "./dom/Clock.js"; import {Clock} from "./dom/Clock.js";
import {ServiceWorkerHandler} from "./dom/ServiceWorkerHandler.js"; import {ServiceWorkerHandler} from "./dom/ServiceWorkerHandler.js";
@ -38,6 +39,8 @@ import {downloadInIframe} from "./dom/download.js";
import {Disposables} from "../../utils/Disposables"; import {Disposables} from "../../utils/Disposables";
import {parseHTML} from "./parsehtml.js"; import {parseHTML} from "./parsehtml.js";
import {handleAvatarError} from "./ui/avatar"; import {handleAvatarError} from "./ui/avatar";
import {MediaDevicesWrapper} from "./dom/MediaDevices";
import {DOMWebRTC} from "./dom/WebRTC";
import {ThemeLoader} from "./ThemeLoader"; import {ThemeLoader} from "./ThemeLoader";
function addScript(src) { function addScript(src) {
@ -127,7 +130,7 @@ function adaptUIOnVisualViewportResize(container) {
} }
export class Platform { export class Platform {
constructor({ container, assetPaths, config, configURL, options = null, cryptoExtras = null }) { constructor({ container, assetPaths, config, configURL, logger, options = null, cryptoExtras = null }) {
this._container = container; this._container = container;
this._assetPaths = assetPaths; this._assetPaths = assetPaths;
this._config = config; this._config = config;
@ -136,7 +139,7 @@ export class Platform {
this.clock = new Clock(); this.clock = new Clock();
this.encoding = new Encoding(); this.encoding = new Encoding();
this.random = Math.random; this.random = Math.random;
this._createLogger(options?.development); this.logger = logger ?? this._createLogger(options?.development);
this.history = new History(); this.history = new History();
this.onlineStatus = new OnlineStatus(); this.onlineStatus = new OnlineStatus();
this._serviceWorkerHandler = null; this._serviceWorkerHandler = null;
@ -165,6 +168,8 @@ export class Platform {
this._disposables = new Disposables(); this._disposables = new Disposables();
this._olmPromise = undefined; this._olmPromise = undefined;
this._workerPromise = undefined; this._workerPromise = undefined;
this.mediaDevices = new MediaDevicesWrapper(navigator.mediaDevices);
this.webRTC = new DOMWebRTC();
this._themeLoader = import.meta.env.DEV? null: new ThemeLoader(this); this._themeLoader = import.meta.env.DEV? null: new ThemeLoader(this);
} }
@ -202,6 +207,7 @@ export class Platform {
} }
_createLogger(isDevelopment) { _createLogger(isDevelopment) {
const logger = new Logger({platform: this});
// Make sure that loginToken does not end up in the logs // Make sure that loginToken does not end up in the logs
const transformer = (item) => { const transformer = (item) => {
if (item.e?.stack) { if (item.e?.stack) {
@ -209,11 +215,12 @@ export class Platform {
} }
return item; return item;
}; };
const logPersister = new IDBLogPersister({name: "hydrogen_logs", platform: this, serializedTransformer: transformer});
logger.addReporter(logPersister);
if (isDevelopment) { if (isDevelopment) {
this.logger = new ConsoleLogger({platform: this}); logger.addReporter(new ConsoleReporter());
} else {
this.logger = new IDBLogger({name: "hydrogen_logs", platform: this, serializedTransformer: transformer});
} }
return logger;
} }
get updateService() { get updateService() {
@ -358,24 +365,22 @@ import {LogItem} from "../../logging/LogItem";
export function tests() { export function tests() {
return { return {
"loginToken should not be in logs": (assert) => { "loginToken should not be in logs": (assert) => {
const transformer = (item) => { const logPersister = Object.create(IDBLogPersister.prototype);
logPersister._queuedItems = [];
logPersister.options = {
serializedTransformer: (item) => {
if (item.e?.stack) { if (item.e?.stack) {
item.e.stack = item.e.stack.replace(/(?<=\/\?loginToken=).+/, "<snip>"); item.e.stack = item.e.stack.replace(/(?<=\/\?loginToken=).+/, "<snip>");
} }
return item; return item;
}
}; };
const logger = { const logger = { _now() {return 5;} };
_queuedItems: [],
_serializedTransformer: transformer,
_now: () => {}
};
logger.persist = IDBLogger.prototype._persistItem.bind(logger);
const logItem = new LogItem("test", 1, logger); const logItem = new LogItem("test", 1, logger);
logItem.error = new Error(); logItem.error = new Error();
logItem.error.stack = "main http://localhost:3000/src/main.js:55\n<anonymous> http://localhost:3000/?loginToken=secret:26" logItem.error.stack = "main http://localhost:3000/src/main.js:55\n<anonymous> http://localhost:3000/?loginToken=secret:26"
logger.persist(logItem, null, false); logPersister.reportItem(logItem, null, false);
const item = logger._queuedItems.pop(); const item = logPersister._queuedItems.pop();
console.log(item);
assert.strictEqual(item.json.search("secret"), -1); assert.strictEqual(item.json.search("secret"), -1);
} }
}; };

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseObservableValue} from "../../../observable/ObservableValue"; import {BaseObservableValue} from "../../../observable/value/BaseObservableValue";
export class History extends BaseObservableValue { export class History extends BaseObservableValue {
handleEvent(event) { handleEvent(event) {

View file

@ -0,0 +1,184 @@
/*
Copyright 2021 Šimon Brandner <simon.bra.ag@gmail.com>
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {MediaDevices as IMediaDevices, Stream, Track, TrackKind, VolumeMeasurer} from "../../types/MediaDevices";
const POLLING_INTERVAL = 200; // ms
export const SPEAKING_THRESHOLD = -60; // dB
const SPEAKING_SAMPLE_COUNT = 8; // samples
export class MediaDevicesWrapper implements IMediaDevices {
constructor(private readonly mediaDevices: MediaDevices) {}
enumerate(): Promise<MediaDeviceInfo[]> {
return this.mediaDevices.enumerateDevices();
}
async getMediaTracks(audio: true | MediaDeviceInfo, video: boolean | MediaDeviceInfo): Promise<Stream> {
const stream = await this.mediaDevices.getUserMedia(this.getUserMediaContraints(audio, video));
stream.addEventListener("removetrack", evt => {
console.log(`removing track ${evt.track.id} (${evt.track.kind}) from stream ${stream.id}`);
});
return stream as Stream;
}
async getScreenShareTrack(): Promise<Stream | undefined> {
const stream = await this.mediaDevices.getDisplayMedia(this.getScreenshareContraints());
return stream as Stream;
}
private getUserMediaContraints(audio: boolean | MediaDeviceInfo, video: boolean | MediaDeviceInfo): MediaStreamConstraints {
const isWebkit = !!navigator["webkitGetUserMedia"];
return {
audio: audio
? {
deviceId: typeof audio !== "boolean" ? { ideal: audio.deviceId } : undefined,
}
: false,
video: video
? {
deviceId: typeof video !== "boolean" ? { ideal: video.deviceId } : undefined,
/* We want 640x360. Chrome will give it only if we ask exactly,
FF refuses entirely if we ask exactly, so have to ask for ideal
instead
XXX: Is this still true?
*/
width: isWebkit ? { exact: 640 } : { ideal: 640 },
height: isWebkit ? { exact: 360 } : { ideal: 360 },
}
: false,
};
}
private getScreenshareContraints(): DisplayMediaStreamConstraints {
return {
audio: false,
video: true,
};
}
createVolumeMeasurer(stream: Stream, callback: () => void): VolumeMeasurer {
return new WebAudioVolumeMeasurer(stream as MediaStream, callback);
}
}
export class WebAudioVolumeMeasurer implements VolumeMeasurer {
private measuringVolumeActivity = false;
private audioContext?: AudioContext;
private analyser: AnalyserNode;
private frequencyBinCount: Float32Array;
private speakingThreshold = SPEAKING_THRESHOLD;
private speaking = false;
private volumeLooperTimeout: number;
private speakingVolumeSamples: number[];
private callback: () => void;
private stream: MediaStream;
constructor(stream: MediaStream, callback: () => void) {
this.stream = stream;
this.callback = callback;
this.speakingVolumeSamples = new Array(SPEAKING_SAMPLE_COUNT).fill(-Infinity);
this.initVolumeMeasuring();
this.measureVolumeActivity(true);
}
get isSpeaking(): boolean { return this.speaking; }
/**
* Starts emitting volume_changed events where the emitter value is in decibels
* @param enabled emit volume changes
*/
private measureVolumeActivity(enabled: boolean): void {
if (enabled) {
if (!this.audioContext || !this.analyser || !this.frequencyBinCount) return;
this.measuringVolumeActivity = true;
this.volumeLooper();
} else {
this.measuringVolumeActivity = false;
this.speakingVolumeSamples.fill(-Infinity);
this.callback();
// this.emit(CallFeedEvent.VolumeChanged, -Infinity);
}
}
private initVolumeMeasuring(): void {
const AudioContext = window.AudioContext || window["webkitAudioContext"] as undefined | typeof window.AudioContext;
if (!AudioContext) return;
this.audioContext = new AudioContext();
this.analyser = this.audioContext.createAnalyser();
this.analyser.fftSize = 512;
this.analyser.smoothingTimeConstant = 0.1;
const mediaStreamAudioSourceNode = this.audioContext.createMediaStreamSource(this.stream);
mediaStreamAudioSourceNode.connect(this.analyser);
this.frequencyBinCount = new Float32Array(this.analyser.frequencyBinCount);
}
public setSpeakingThreshold(threshold: number) {
this.speakingThreshold = threshold;
}
private volumeLooper = () => {
if (!this.analyser) return;
if (!this.measuringVolumeActivity) return;
this.analyser.getFloatFrequencyData(this.frequencyBinCount);
let maxVolume = -Infinity;
for (let i = 0; i < this.frequencyBinCount.length; i++) {
if (this.frequencyBinCount[i] > maxVolume) {
maxVolume = this.frequencyBinCount[i];
}
}
this.speakingVolumeSamples.shift();
this.speakingVolumeSamples.push(maxVolume);
this.callback();
// this.emit(CallFeedEvent.VolumeChanged, maxVolume);
let newSpeaking = false;
for (let i = 0; i < this.speakingVolumeSamples.length; i++) {
const volume = this.speakingVolumeSamples[i];
if (volume > this.speakingThreshold) {
newSpeaking = true;
break;
}
}
if (this.speaking !== newSpeaking) {
this.speaking = newSpeaking;
this.callback();
// this.emit(CallFeedEvent.Speaking, this.speaking);
}
this.volumeLooperTimeout = setTimeout(this.volumeLooper, POLLING_INTERVAL) as unknown as number;
};
public stop(): void {
clearTimeout(this.volumeLooperTimeout);
this.analyser.disconnect();
this.audioContext?.close();
}
}

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseObservableValue} from "../../../observable/ObservableValue"; import {BaseObservableValue} from "../../../observable/value/BaseObservableValue";
export class OnlineStatus extends BaseObservableValue { export class OnlineStatus extends BaseObservableValue {
constructor() { constructor() {

View file

@ -0,0 +1,77 @@
/*
Copyright 2021 Šimon Brandner <simon.bra.ag@gmail.com>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {Stream, Track, TrackKind} from "../../types/MediaDevices";
import {WebRTC, Sender, PeerConnection} from "../../types/WebRTC";
import {SDPStreamMetadataPurpose} from "../../../matrix/calls/callEventTypes";
const POLLING_INTERVAL = 200; // ms
export const SPEAKING_THRESHOLD = -60; // dB
const SPEAKING_SAMPLE_COUNT = 8; // samples
export class DOMWebRTC implements WebRTC {
createPeerConnection(forceTURN: boolean, turnServers: RTCIceServer[], iceCandidatePoolSize): PeerConnection {
const peerConn = new RTCPeerConnection({
iceTransportPolicy: forceTURN ? 'relay' : undefined,
iceServers: turnServers,
iceCandidatePoolSize: iceCandidatePoolSize,
}) as PeerConnection;
return new Proxy(peerConn, {
get(target, prop, receiver) {
if (prop === "close") {
console.trace("calling peerConnection.close");
}
const value = target[prop];
if (typeof value === "function") {
return value.bind(target);
} else {
return value;
}
}
});
}
prepareSenderForPurpose(peerConnection: PeerConnection, sender: Sender, purpose: SDPStreamMetadataPurpose): void {
if (purpose === SDPStreamMetadataPurpose.Screenshare) {
this.getRidOfRTXCodecs(peerConnection as RTCPeerConnection, sender as RTCRtpSender);
}
}
private getRidOfRTXCodecs(peerConnection: RTCPeerConnection, sender: RTCRtpSender): void {
// RTCRtpReceiver.getCapabilities and RTCRtpSender.getCapabilities don't seem to be supported on FF
if (!RTCRtpReceiver.getCapabilities || !RTCRtpSender.getCapabilities) return;
const recvCodecs = RTCRtpReceiver.getCapabilities("video")?.codecs ?? [];
const sendCodecs = RTCRtpSender.getCapabilities("video")?.codecs ?? [];
const codecs = [...sendCodecs, ...recvCodecs];
for (const codec of codecs) {
if (codec.mimeType === "video/rtx") {
const rtxCodecIndex = codecs.indexOf(codec);
codecs.splice(rtxCodecIndex, 1);
}
}
const transceiver = peerConnection.getTransceivers().find(t => t.sender === sender);
if (transceiver && (
transceiver.sender.track?.kind === "video" ||
transceiver.receiver.track?.kind === "video"
)
) {
transceiver.setCodecPreferences(codecs);
}
}
}

View file

@ -46,6 +46,14 @@ limitations under the License.
font-size: calc(var(--avatar-size) * 0.6); font-size: calc(var(--avatar-size) * 0.6);
} }
.hydrogen .avatar.size-96 {
--avatar-size: 96px;
width: var(--avatar-size);
height: var(--avatar-size);
line-height: var(--avatar-size);
font-size: calc(var(--avatar-size) * 0.6);
}
.hydrogen .avatar.size-64 { .hydrogen .avatar.size-64 {
--avatar-size: 64px; --avatar-size: 64px;
width: var(--avatar-size); width: var(--avatar-size);

View file

@ -0,0 +1,219 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
.CallView {
height: 40vh;
display: grid;
}
.CallView > * {
grid-column: 1;
grid-row: 1;
}
.CallView_members {
display: grid;
gap: 12px;
background: var(--background-color-secondary--darker-60);
padding: 12px;
margin: 0;
min-height: 0;
list-style: none;
align-self: stretch;
}
.StreamView {
display: grid;
border-radius: 8px;
overflow: hidden;
background-color: black;
}
.StreamView > * {
grid-column: 1;
grid-row: 1;
}
.StreamView video {
width: 100%;
height: 100%;
min-height: 0;
object-fit: contain;
}
.StreamView_avatar {
align-self: center;
justify-self: center;
}
.StreamView_muteStatus {
align-self: start;
justify-self: end;
width: 24px;
height: 24px;
background-position: center;
background-repeat: no-repeat;
background-size: 14px;
display: block;
background-color: var(--text-color);
border-radius: 4px;
margin: 4px;
}
.StreamView_muteStatus.microphoneMuted {
background-image: url("./icons/mic-muted.svg?primary=text-color--lighter-80");
}
.StreamView_muteStatus.cameraMuted {
background-image: url("./icons/cam-muted.svg?primary=text-color--lighter-80");
}
.CallView_buttons {
align-self: end;
justify-self: center;
display: flex;
gap: 12px;
margin-bottom: 16px;
/** Chrome (v100) requires this to make the buttons clickable
* where they overlap with the video element, even though
* the buttons come later in the DOM. */
z-index: 1;
}
.CallView_buttons button {
border-radius: 100%;
width: 48px;
height: 48px;
border: none;
background-color: var(--accent-color);
background-position: center;
background-repeat: no-repeat;
}
.CallView_buttons button:disabled {
background-color: var(--accent-color--lighter-10);
}
.CallView_buttons .CallView_hangup {
background-color: var(--error-color);
background-image: url("./icons/hangup.svg?primary=background-color-primary");
}
.CallView_buttons .CallView_hangup:disabled {
background-color: var(--error-color--lighter-10);
}
.CallView_buttons .CallView_mutedMicrophone {
background-color: var(--background-color-primary);
background-image: url("./icons/mic-muted.svg?primary=text-color");
}
.CallView_buttons .CallView_unmutedMicrophone {
background-image: url("./icons/mic-unmuted.svg?primary=background-color-primary");
}
.CallView_buttons .CallView_mutedCamera {
background-color: var(--background-color-primary);
background-image: url("./icons/cam-muted.svg?primary=text-color");
}
.CallView_buttons .CallView_unmutedCamera {
background-image: url("./icons/cam-unmuted.svg?primary=background-color-primary");
}
.CallView_members.size1 {
grid-template-columns: 1fr;
grid-template-rows: 1fr;
}
.CallView_members.size2 {
grid-template-columns: 1fr;
grid-template-rows: repeat(2, 1fr);
}
/* square */
.CallView_members.square.size3,
.CallView_members.square.size4 {
grid-template-columns: repeat(2, 1fr);
grid-template-rows: repeat(2, 1fr);
}
.CallView_members.square.size5,
.CallView_members.square.size6 {
grid-template-columns: repeat(3, 1fr);
grid-template-rows: repeat(2, 1fr);
}
.CallView_members.square.size7,
.CallView_members.square.size8,
.CallView_members.square.size9 {
grid-template-columns: repeat(3, 1fr);
grid-template-rows: repeat(3, 1fr);
}
.CallView_members.square.size10 {
grid-template-columns: repeat(3, 1fr);
grid-template-rows: repeat(4, 1fr);
}
/** tall */
.CallView_members.tall.size3 {
grid-template-columns: 1fr;
grid-template-rows: repeat(3, 1fr);
}
.CallView_members.tall.size4 {
grid-template-columns: 1fr;
grid-template-rows: repeat(4, 1fr);
}
.CallView_members.tall.size5,
.CallView_members.tall.size6 {
grid-template-rows: repeat(3, 1fr);
grid-template-columns: repeat(2, 1fr);
}
.CallView_members.tall.size7,
.CallView_members.tall.size8 {
grid-template-rows: repeat(4, 1fr);
grid-template-columns: repeat(2, 1fr);
}
.CallView_members.tall.size9,
.CallView_members.tall.size10 {
grid-template-rows: repeat(5, 1fr);
grid-template-columns: repeat(2, 1fr);
}
/** wide */
.CallView_members.wide.size2 {
grid-template-columns: repeat(2, 1fr);
grid-template-rows: 1fr;
}
.CallView_members.wide.size3 {
grid-template-rows: 1fr;
grid-template-columns: repeat(3, 1fr);
}
.CallView_members.wide.size4 {
grid-template-rows: 1fr;
grid-template-columns: repeat(4, 1fr);
}
.CallView_members.wide.size5,
.CallView_members.wide.size6 {
grid-template-rows: repeat(2, 1fr);
grid-template-columns: repeat(3, 1fr);
}
.CallView_members.wide.size7,
.CallView_members.wide.size8 {
grid-template-rows: repeat(2, 1fr);
grid-template-columns: repeat(4, 1fr);
}
.CallView_members.wide.size9,
.CallView_members.wide.size10 {
grid-template-rows: repeat(3, 1fr);
grid-template-columns: repeat(4, 1fr);
}

View file

@ -0,0 +1 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M0.290472 1.37627C0.677768 0.985743 1.3057 0.985743 1.69299 1.37627L16.569 16.3762C16.9563 16.7668 16.9563 17.3999 16.569 17.7904C16.1817 18.181 15.5538 18.181 15.1665 17.7904L0.290472 2.79048C-0.096824 2.39995 -0.096824 1.76679 0.290472 1.37627Z" fill="#ff00ff"></path><path d="M0.597515 5.19186C0.323238 5.646 0.165249 6.17941 0.165249 6.75001V14.0833C0.165249 15.7402 1.49729 17.0833 3.14045 17.0833H12.363L0.639137 5.2371C0.624608 5.22242 0.610733 5.20733 0.597515 5.19186Z" fill="#ff00ff"></path><path d="M14.2148 6.75002V11.9031L6.14598 3.75002H11.2396C12.8828 3.75002 14.2148 5.09317 14.2148 6.75002Z" fill="#ff00ff"></path><path d="M18.3887 5.88312L15.8677 7.91669V12.9167L18.3887 14.9503C19.038 15.4741 19.9999 15.0079 19.9999 14.1694V6.66399C19.9999 5.82548 19.038 5.35931 18.3887 5.88312Z" fill="#ff00ff"></path></svg>

After

Width:  |  Height:  |  Size: 934 B

View file

@ -0,0 +1 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M2.50001 3.33334C1.11929 3.33334 0 4.45264 0 5.83336V14.1667C0 15.5474 1.11929 16.6667 2.50001 16.6667H11.6667C13.0474 16.6667 14.1667 15.5474 14.1667 14.1667V5.83336C14.1667 4.45264 13.0474 3.33334 11.6667 3.33334H2.50001Z" fill="#ff00ff"></path><path d="M18.6462 5.24983L15.8334 7.50004V12.5001L18.6462 14.7503C19.1918 15.1868 20.0001 14.7983 20.0001 14.0996V5.90056C20.0001 5.2018 19.1918 4.81332 18.6462 5.24983Z" fill="#ff00ff"></path></svg>

After

Width:  |  Height:  |  Size: 551 B

Some files were not shown because too many files have changed in this diff Show more