Merge branch 'master' into bwindels/log-signature-failure

This commit is contained in:
Bruno Windels 2021-12-09 18:34:36 +01:00
commit 8d315f2741
136 changed files with 2753 additions and 5969 deletions

View file

@ -13,5 +13,13 @@ module.exports = {
"no-empty": "off", "no-empty": "off",
"no-prototype-builtins": "off", "no-prototype-builtins": "off",
"no-unused-vars": "warn" "no-unused-vars": "warn"
},
"globals": {
"DEFINE_VERSION": "readonly",
"DEFINE_GLOBAL_HASH": "readonly",
// only available in sw.js
"DEFINE_UNHASHED_PRECACHED_ASSETS": "readonly",
"DEFINE_HASHED_PRECACHED_ASSETS": "readonly",
"DEFINE_HASHED_CACHED_ON_REQUEST_ASSETS": "readonly"
} }
}; };

View file

@ -8,3 +8,5 @@
otherwise it becomes hard to remember what was a default/named export otherwise it becomes hard to remember what was a default/named export
- should we return promises from storage mutation calls? probably not, as we don't await them anywhere. only read calls should return promises? - should we return promises from storage mutation calls? probably not, as we don't await them anywhere. only read calls should return promises?
- we don't anymore - we don't anymore
- don't use these features, as they are not widely enough supported.
- [lookbehind in regular expressions](https://caniuse.com/js-regexp-lookbehind)

View file

@ -1,7 +1,9 @@
# Typescript migration # Typescript style guide
## Introduce `abstract` & `override`
- find all methods and getters that throw or are empty in base classes and turn into abstract method or if all methods are abstract, into an interface. ## Use `type` rather than `interface` for named parameters and POJO return values.
- change child impls to not call super.method and to add override
- don't allow implicit override in ts config `type` and `interface` can be used somewhat interchangebly used, but let's use `type` to describe data and `interface` to describe (polymorphic) behaviour.
Good examples of data are option objects to have named parameters, and POJO (plain old javascript objects) without any methods, just fields.
Also see [this playground](https://www.typescriptlang.org/play?#code/C4TwDgpgBACghgJwgO2AeTMAlge2QZygF4oBvAKCiqmTgFsIAuKfYBLZAcwG5LqATCABs4IAPzNkAVzoAjCAl4BfcuVCQoAYQAWWIfwzY8hEvCSpDuAlABkZPlQDGOITgTNW7LstWOR+QjMUYHtqKGcCNilHYDcAChxMK3xmIIsk4wBKewcoFRVyPzgArV19KAgAD2AUfkDEYNDqCM9o2IQEjIJmHT0DLvxsijCw-ClIDsSjAkzeEebjEIYAuE5oEgADABJSKeSAOloGJSgsQh29433nVwQlDbnqfKA)

109
doc/impl-thoughts/SDK.md Normal file
View file

@ -0,0 +1,109 @@
SDK:
- we need to compile src/lib.ts to javascript, with a d.ts file generated as well. We need to compile to javascript once for cjs and once of es modules. The package.json looks like this:
```
"main": "./dist/index.cjs",
"exports": {
"import": "./dist/index.mjs",
"require": "./dist/index.cjs"
},
"types": "dist/index.d.ts",
```
we don't need to bundle for the sdk case! we might need to do some transpilation to just plain ES6 (e.g. don't assume ?. and ??) we could use a browserslist query for this e.g. `node 14`. esbuild seems to support this as well, tldraw uses esbuild for their build.
one advantage of not bundling the files for the sdk is that you can still use import overrides in the consuming project build settings. is that an idiomatic way of doing things though?
this way we will support typescript, non-esm javascript and esm javascript using libhydrogen as an SDK
got this from https://medium.com/dazn-tech/publishing-npm-packages-as-native-es-modules-41ffbc0a9dea
how about the assets?
we also need to build the app
we need to be able to version libhydrogen independently from hydrogen the app? as any api breaking changes will need a major version increase. we probably want to end up with a monorepo where the app uses the sdk as well and we just use the local code with yarn link?
## Assets
we want to provide scss/sass files, but also css that can be included
https://github.com/webpack/webpack/issues/7353 seems to imply that we just need to include the assets in the published files and from there on it is the consumer of libhydrogen's problem.
how does all of this tie in with vite?
we want to have hydrogenapp be a consumer of libhydrogen, potentially as two packages in a monorepo ... but we want the SDK to expose views and stylesheets... without having an index.html (which would be in hydrogenapp). this seems a bit odd...?
what would be in hydrogenapp actually? just an index.html file?
I'm not sure it makes sense to have them be 2 different packages in a monorepo, they should really be two artifacts from the same directory.
the stylesheets included in libhydrogen are from the same main.css file as is used in the app
https://www.freecodecamp.org/news/build-a-css-library-with-vitejs/
basically, we import the sass file from src/lib.ts so it is included in the assets there too, and we also create a plugin that emits a file for every sass file as suggested in the link above?
we probably want two different build commands for the app and the sdk though, we could have a parent vite config that both build configs extend from?
### Dependency assets
our dependencies should not be bundled for the SDK case. So if we import aesjs, it would be up to the build system of the consuming project to make that import work.
the paths.ts thingy ... we want to make it easy for people to setup the assets for our dependencies (olm), some assets are also part of the sdk itself. it might make sense to make all of the assets there part of the sdk (e.g. bundle olm.wasm and friends?) although shipping crypto, etc ...
perhaps we should have an include file per build system that treats own assets and dep assets the same by including the package name as wel for our own deps:
```js
import _downloadSandboxPath from "@matrix-org/hydrogen-sdk/download-sandbox.html?url";
import _serviceWorkerPath from "@matrix-org/hydrogen-sdk/sw.js?url"; // not yet sure this is the way to do it
import olmWasmPath from "@matrix-org/olm/olm.wasm?url";
import olmJsPath from "@matrix-org/olm/olm.js?url";
import olmLegacyJsPath from "@matrix-org/olm/olm_legacy.js?url";
export const olmPaths = {
wasm: olmWasmPath,
legacyBundle: olmLegacyJsPath,
wasmBundle: olmJsPath,
};
export const downloadSandboxPath = _downloadSandboxPath;
```
we could put this file per build system, as ESM, in dist as well so you can include it to get the paths
## Tooling
- `vite` a more high-level build tool that takes your index.html and turns it into optimized assets that you can host for production, as well as a very fast dev server. is used to have good default settings for our tools, typescript support, and also deals with asset compiling. good dev server. Would be nice to have the same tool for dev and prod. vite has good support for using `import` for anything that is not javascript, where we had an issue with `snowpack` (to get the prod path of an asset).
- `rollup`: inlines
- `lerna` is used to handle multi-package monorepos
- `esbuild`: a js/ts build tool that we could use for building the lower level sdk where no other assets are involved, `vite` uses it for fast dev builds (`rollup` for prod). For now we won't extract a lower level sdk though.
## TODO
- finish vite app build (without IE11 for now?)
- create vite config to build src/lib.ts in cjs and esm, inheriting from a common base config with the app config
- this will create a dist folder with
- the whole source tree in es and cjs format
- an es file to import get the asset paths as they are expected by Platform, per build system
- assets from hydrogen itself:
- css files and any resource used therein
- download-sandbox.html
- a type declaration file (index.d.ts)
## Questions
- can rollup not bundle the source tree and leave modules intact?
- if we can use a function that creates a chunk per file to pass to manualChunks and disable chunk hashing we can probably do this. See https://rollupjs.org/guide/en/#outputmanualchunks
looks like we should be able to disable chunk name hashing with chunkFileNames https://rollupjs.org/guide/en/#outputoptions-object
we should test this with a vite test config
we also need to compile down to ES6, both for the app and for the sdk

View file

@ -1,6 +1,6 @@
{ {
"name": "hydrogen-web", "name": "hydrogen-web",
"version": "0.2.21", "version": "0.2.22",
"description": "A javascript matrix client prototype, trying to minize RAM usage by offloading as much as possible to IndexedDB", "description": "A javascript matrix client prototype, trying to minize RAM usage by offloading as much as possible to IndexedDB",
"main": "src/lib.ts", "main": "src/lib.ts",
"directories": { "directories": {
@ -10,10 +10,9 @@
"lint": "eslint --cache src/", "lint": "eslint --cache src/",
"lint-ts": "eslint src/ -c .ts-eslintrc.js --ext .ts", "lint-ts": "eslint src/ -c .ts-eslintrc.js --ext .ts",
"lint-ci": "eslint src/", "lint-ci": "eslint src/",
"test": "impunity --entry-point src/main.js --force-esm-dirs lib/ src/", "test": "impunity --entry-point src/platform/web/main.js src/platform/web/Platform.js --force-esm-dirs lib/ src/ --root-dir src/",
"start": "snowpack dev --port 3000", "start": "vite --port 3000",
"build": "node --experimental-modules scripts/build.mjs", "build": "vite build"
"postinstall": "node ./scripts/post-install.js"
}, },
"repository": { "repository": {
"type": "git", "type": "git",
@ -26,46 +25,29 @@
}, },
"homepage": "https://github.com/vector-im/hydrogen-web/#readme", "homepage": "https://github.com/vector-im/hydrogen-web/#readme",
"devDependencies": { "devDependencies": {
"@babel/core": "^7.11.1",
"@babel/preset-env": "^7.11.0",
"@rollup/plugin-babel": "^5.1.0",
"@rollup/plugin-multi-entry": "^4.0.0",
"@typescript-eslint/eslint-plugin": "^4.29.2", "@typescript-eslint/eslint-plugin": "^4.29.2",
"@typescript-eslint/parser": "^4.29.2", "@typescript-eslint/parser": "^4.29.2",
"autoprefixer": "^10.2.6",
"cheerio": "^1.0.0-rc.3",
"commander": "^6.0.0",
"core-js": "^3.6.5", "core-js": "^3.6.5",
"eslint": "^7.32.0", "eslint": "^7.32.0",
"fake-indexeddb": "^3.1.2", "fake-indexeddb": "^3.1.2",
"finalhandler": "^1.1.1", "impunity": "^1.0.9",
"impunity": "^1.0.1",
"mdn-polyfills": "^5.20.0", "mdn-polyfills": "^5.20.0",
"node-html-parser": "^4.0.0", "node-html-parser": "^4.0.0",
"postcss": "^8.1.1", "postcss-css-variables": "^0.18.0",
"postcss-css-variables": "^0.17.0", "postcss-flexbugs-fixes": "^5.0.2",
"postcss-flexbugs-fixes": "^4.2.1",
"postcss-import": "^12.0.1",
"postcss-url": "^8.0.0",
"regenerator-runtime": "^0.13.7", "regenerator-runtime": "^0.13.7",
"rollup-plugin-cleanup": "^3.1.1",
"serve-static": "^1.13.2",
"snowpack": "^3.8.3",
"typescript": "^4.3.5", "typescript": "^4.3.5",
"vite": "^2.6.14",
"xxhashjs": "^0.2.2" "xxhashjs": "^0.2.2"
}, },
"dependencies": { "dependencies": {
"@matrix-org/olm": "https://gitlab.matrix.org/api/v4/projects/27/packages/npm/@matrix-org/olm/-/@matrix-org/olm-3.2.3.tgz", "@matrix-org/olm": "https://gitlab.matrix.org/api/v4/projects/27/packages/npm/@matrix-org/olm/-/@matrix-org/olm-3.2.3.tgz",
"@rollup/plugin-commonjs": "^15.0.0",
"@rollup/plugin-json": "^4.1.0",
"@rollup/plugin-node-resolve": "^9.0.0",
"aes-js": "^3.1.2", "aes-js": "^3.1.2",
"another-json": "^0.2.0", "another-json": "^0.2.0",
"base64-arraybuffer": "^0.2.0", "base64-arraybuffer": "^0.2.0",
"bs58": "^4.0.1", "bs58": "^4.0.1",
"dompurify": "^2.3.0", "dompurify": "^2.3.0",
"es6-promise": "https://github.com/bwindels/es6-promise.git#bwindels/expose-flush", "es6-promise": "https://github.com/bwindels/es6-promise.git#bwindels/expose-flush",
"rollup": "^2.26.4",
"text-encoding": "^0.7.0" "text-encoding": "^0.7.0"
} }
} }

View file

@ -0,0 +1,51 @@
const fs = require('fs/promises');
const path = require('path');
module.exports = function injectWebManifest(manifestFile) {
let root;
let base;
let manifestHref;
return {
name: "hydrogen:injectWebManifest",
apply: "build",
configResolved: config => {
root = config.root;
base = config.base;
},
transformIndexHtml: {
transform(html) {
return [{
tag: "link",
attrs: {rel: "manifest", href: manifestHref},
injectTo: "head"
}];
},
},
generateBundle: async function() {
const absoluteManifestFile = path.resolve(root, manifestFile);
const manifestDir = path.dirname(absoluteManifestFile);
const json = await fs.readFile(absoluteManifestFile, {encoding: "utf8"});
const manifest = JSON.parse(json);
for (const icon of manifest.icons) {
const iconFileName = path.resolve(manifestDir, icon.src);
const imgData = await fs.readFile(iconFileName);
const ref = this.emitFile({
type: "asset",
name: path.basename(iconFileName),
source: imgData
});
// we take the basename as getFileName gives the filename
// relative to the output dir, but the manifest is an asset
// just like they icon, so we assume they end up in the same dir
icon.src = path.basename(this.getFileName(ref));
}
const outputName = path.basename(absoluteManifestFile);
const manifestRef = this.emitFile({
type: "asset",
name: outputName,
source: JSON.stringify(manifest)
});
manifestHref = base + this.getFileName(manifestRef);
}
};
}

View file

@ -0,0 +1,154 @@
const fs = require('fs/promises');
const path = require('path');
const xxhash = require('xxhashjs');
function contentHash(str) {
var hasher = new xxhash.h32(0);
hasher.update(str);
return hasher.digest();
}
function injectServiceWorker(swFile, otherUnhashedFiles, placeholdersPerChunk) {
const swName = path.basename(swFile);
let root;
let version;
return {
name: "hydrogen:injectServiceWorker",
apply: "build",
enforce: "post",
buildStart() {
this.emitFile({
type: "chunk",
fileName: swName,
id: swFile,
});
},
configResolved: config => {
root = config.root;
version = JSON.parse(config.define.DEFINE_VERSION); // unquote
},
generateBundle: async function(options, bundle) {
const unhashedFilenames = [swName].concat(otherUnhashedFiles);
const unhashedFileContentMap = unhashedFilenames.reduce((map, fileName) => {
const chunkOrAsset = bundle[fileName];
if (!chunkOrAsset) {
throw new Error("could not get content for uncached asset or chunk " + fileName);
}
map[fileName] = chunkOrAsset.source || chunkOrAsset.code;
return map;
}, {});
const assets = Object.values(bundle);
const hashedFileNames = assets.map(o => o.fileName).filter(fileName => !unhashedFileContentMap[fileName]);
const globalHash = getBuildHash(hashedFileNames, unhashedFileContentMap);
const placeholderValues = {
DEFINE_GLOBAL_HASH: `"${globalHash}"`,
...getCacheFileNamePlaceholderValues(swName, unhashedFilenames, assets, placeholdersPerChunk)
};
replacePlaceholdersInChunks(assets, placeholdersPerChunk, placeholderValues);
console.log(`\nBuilt ${version} (${globalHash})`);
}
};
}
function getBuildHash(hashedFileNames, unhashedFileContentMap) {
const unhashedHashes = Object.entries(unhashedFileContentMap).map(([fileName, content]) => {
return `${fileName}-${contentHash(Buffer.from(content))}`;
});
const globalHashAssets = hashedFileNames.concat(unhashedHashes);
globalHashAssets.sort();
return contentHash(globalHashAssets.join(",")).toString();
}
const NON_PRECACHED_JS = [
"hydrogen-legacy",
"olm_legacy.js",
// most environments don't need the worker
"main.js"
];
function isPreCached(asset) {
const {name, fileName} = asset;
return name.endsWith(".svg") ||
name.endsWith(".png") ||
name.endsWith(".css") ||
name.endsWith(".wasm") ||
name.endsWith(".html") ||
// the index and vendor chunks don't have an extension in `name`, so check extension on `fileName`
fileName.endsWith(".js") && !NON_PRECACHED_JS.includes(path.basename(name));
}
function getCacheFileNamePlaceholderValues(swName, unhashedFilenames, assets) {
const unhashedPreCachedAssets = [];
const hashedPreCachedAssets = [];
const hashedCachedOnRequestAssets = [];
for (const asset of assets) {
const {name, fileName} = asset;
// the service worker should not be cached at all,
// it's how updates happen
if (fileName === swName) {
continue;
} else if (unhashedFilenames.includes(fileName)) {
unhashedPreCachedAssets.push(fileName);
} else if (isPreCached(asset)) {
hashedPreCachedAssets.push(fileName);
} else {
hashedCachedOnRequestAssets.push(fileName);
}
}
return {
DEFINE_UNHASHED_PRECACHED_ASSETS: JSON.stringify(unhashedPreCachedAssets),
DEFINE_HASHED_PRECACHED_ASSETS: JSON.stringify(hashedPreCachedAssets),
DEFINE_HASHED_CACHED_ON_REQUEST_ASSETS: JSON.stringify(hashedCachedOnRequestAssets)
}
}
function replacePlaceholdersInChunks(assets, placeholdersPerChunk, placeholderValues) {
for (const [name, placeholderMap] of Object.entries(placeholdersPerChunk)) {
const chunk = assets.find(a => a.type === "chunk" && a.name === name);
if (!chunk) {
throw new Error(`could not find chunk ${name} to replace placeholders`);
}
for (const [placeholderName, placeholderLiteral] of Object.entries(placeholderMap)) {
const replacedValue = placeholderValues[placeholderName];
const oldCode = chunk.code;
chunk.code = chunk.code.replaceAll(placeholderLiteral, replacedValue);
if (chunk.code === oldCode) {
throw new Error(`Could not replace ${placeholderName} in ${name}, looking for literal ${placeholderLiteral}:\n${chunk.code}`);
}
}
}
}
/** creates a value to be include in the `define` build settings,
* but can be replace at the end of the build in certain chunks.
* We need this for injecting the global build hash and the final
* filenames in the service worker and index chunk.
* These values are only known in the generateBundle step, so we
* replace them by unique strings wrapped in a prompt call so no
* transformation will touch them (minifying, ...) and we can do a
* string replacement still at the end of the build. */
function definePlaceholderValue(mode, name, devValue) {
if (mode === "production") {
// note that `prompt(...)` will never be in the final output, it's replaced by the final value
// once we know at the end of the build what it is and just used as a temporary value during the build
// as something that will not be transformed.
// I first considered Symbol but it's not inconceivable that babel would transform this.
return `prompt(${JSON.stringify(name)})`;
} else {
return JSON.stringify(devValue);
}
}
function createPlaceholderValues(mode) {
return {
DEFINE_GLOBAL_HASH: definePlaceholderValue(mode, "DEFINE_GLOBAL_HASH", null),
DEFINE_UNHASHED_PRECACHED_ASSETS: definePlaceholderValue(mode, "UNHASHED_PRECACHED_ASSETS", []),
DEFINE_HASHED_PRECACHED_ASSETS: definePlaceholderValue(mode, "HASHED_PRECACHED_ASSETS", []),
DEFINE_HASHED_CACHED_ON_REQUEST_ASSETS: definePlaceholderValue(mode, "HASHED_CACHED_ON_REQUEST_ASSETS", []),
};
}
module.exports = {injectServiceWorker, createPlaceholderValues};

View file

@ -1,578 +0,0 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {build as snowpackBuild, loadConfiguration} from "snowpack"
import cheerio from "cheerio";
import fsRoot from "fs";
const fs = fsRoot.promises;
import path from "path";
import xxhash from 'xxhashjs';
import { rollup } from 'rollup';
import postcss from "postcss";
import postcssImport from "postcss-import";
import { fileURLToPath } from 'url';
import { dirname } from 'path';
import commander from "commander";
// needed for legacy bundle
import babel from '@rollup/plugin-babel';
// needed to find the polyfill modules in the main-legacy.js bundle
import { nodeResolve } from '@rollup/plugin-node-resolve';
// needed because some of the polyfills are written as commonjs modules
import commonjs from '@rollup/plugin-commonjs';
// multi-entry plugin so we can add polyfill file to main
import multi from '@rollup/plugin-multi-entry';
import removeJsComments from 'rollup-plugin-cleanup';
// replace urls of asset names with content hashed version
import postcssUrl from "postcss-url";
import cssvariables from "postcss-css-variables";
import autoprefixer from "autoprefixer";
import flexbugsFixes from "postcss-flexbugs-fixes";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const projectDir = path.join(__dirname, "../");
const snowpackOutPath = path.join(projectDir, "snowpack-build-output");
const cssSrcDir = path.join(projectDir, "src/platform/web/ui/css/");
const snowpackConfig = await loadConfiguration({buildOptions: {out: snowpackOutPath}}, "snowpack.config.js");
const snowpackOutDir = snowpackConfig.buildOptions.out.substring(projectDir.length);
const srcDir = path.join(projectDir, `${snowpackOutDir}/src/`);
const isPathInSrcDir = path => path.startsWith(srcDir);
const parameters = new commander.Command();
parameters
.option("--modern-only", "don't make a legacy build")
.option("--override-imports <json file>", "pass in a file to override import paths, see doc/SKINNING.md")
.option("--override-css <main css file>", "pass in an alternative main css file")
parameters.parse(process.argv);
/**
* We use Snowpack to handle the translation of TypeScript
* into JavaScript. We thus can't bundle files straight from
* the src directory, since some of them are TypeScript, and since
* they may import Node modules. We thus bundle files after they
* have been processed by Snowpack. This function returns paths
* to the files that have already been pre-processed in this manner.
*/
function srcPath(src) {
return path.join(snowpackOutDir, 'src', src);
}
async function build({modernOnly, overrideImports, overrideCss}) {
await snowpackBuild({config: snowpackConfig});
// get version number
const version = JSON.parse(await fs.readFile(path.join(projectDir, "package.json"), "utf8")).version;
let importOverridesMap;
if (overrideImports) {
importOverridesMap = await readImportOverrides(overrideImports);
}
const devHtml = await fs.readFile(path.join(snowpackOutPath, "index.html"), "utf8");
const doc = cheerio.load(devHtml);
const themes = [];
findThemes(doc, themeName => {
themes.push(themeName);
});
// clear target dir
const targetDir = path.join(projectDir, "target/");
await removeDirIfExists(targetDir);
await createDirs(targetDir, themes);
const assets = new AssetMap(targetDir);
// copy olm assets
const olmAssets = await copyFolder(path.join(projectDir, "lib/olm/"), assets.directory);
assets.addSubMap(olmAssets);
await assets.write(`hydrogen.js`, await buildJs(srcPath("main.js"), [srcPath("platform/web/Platform.js")], importOverridesMap));
if (!modernOnly) {
await assets.write(`hydrogen-legacy.js`, await buildJsLegacy(srcPath("main.js"), [
srcPath('platform/web/legacy-polyfill.js'),
srcPath('platform/web/LegacyPlatform.js')
], importOverridesMap));
await assets.write(`worker.js`, await buildJsLegacy(srcPath("platform/web/worker/main.js"), [srcPath('platform/web/worker/polyfill.js')]));
}
// copy over non-theme assets
const baseConfig = JSON.parse(await fs.readFile(path.join(projectDir, "assets/config.json"), {encoding: "utf8"}));
const downloadSandbox = "download-sandbox.html";
let downloadSandboxHtml = await fs.readFile(path.join(projectDir, `assets/${downloadSandbox}`));
await assets.write(downloadSandbox, downloadSandboxHtml);
// creates the directories where the theme css bundles are placed in,
// and writes to assets, so the build bundles can translate them, so do it first
await copyThemeAssets(themes, assets);
await buildCssBundles(buildCssLegacy, themes, assets, overrideCss);
await buildManifest(assets);
// all assets have been added, create a hash from all assets name to cache unhashed files like index.html
assets.addToHashForAll("index.html", devHtml);
let swSource = await fs.readFile(path.join(snowpackOutPath, "sw.js"), "utf8");
assets.addToHashForAll("sw.js", swSource);
const globalHash = assets.hashForAll();
await buildServiceWorker(swSource, version, globalHash, assets);
await buildHtml(doc, version, baseConfig, globalHash, modernOnly, assets);
await removeDirIfExists(snowpackOutPath);
console.log(`built hydrogen ${version} (${globalHash}) successfully with ${assets.size} files`);
}
async function findThemes(doc, callback) {
doc("link[rel~=stylesheet][title]").each((i, el) => {
const theme = doc(el);
const href = theme.attr("href");
const themesPrefix = "/themes/";
const prefixIdx = href.indexOf(themesPrefix);
if (prefixIdx !== -1) {
const themeNameStart = prefixIdx + themesPrefix.length;
const themeNameEnd = href.indexOf("/", themeNameStart);
const themeName = href.substr(themeNameStart, themeNameEnd - themeNameStart);
callback(themeName, theme);
}
});
}
async function createDirs(targetDir, themes) {
await fs.mkdir(targetDir);
const themeDir = path.join(targetDir, "themes");
await fs.mkdir(themeDir);
for (const theme of themes) {
await fs.mkdir(path.join(themeDir, theme));
}
}
async function copyThemeAssets(themes, assets) {
for (const theme of themes) {
const themeDstFolder = path.join(assets.directory, `themes/${theme}`);
const themeSrcFolder = path.join(cssSrcDir, `themes/${theme}`);
const themeAssets = await copyFolder(themeSrcFolder, themeDstFolder, file => {
return !file.endsWith(".css");
});
assets.addSubMap(themeAssets);
}
return assets;
}
async function buildHtml(doc, version, baseConfig, globalHash, modernOnly, assets) {
// transform html file
// change path to main.css to css bundle
doc("link[rel=stylesheet]:not([title])").attr("href", assets.resolve(`hydrogen.css`));
// adjust file name of icon on iOS
doc("link[rel=apple-touch-icon]").attr("href", assets.resolve(`icon-maskable.png`));
// change paths to all theme stylesheets
findThemes(doc, (themeName, theme) => {
theme.attr("href", assets.resolve(`themes/${themeName}/bundle.css`));
});
const configJSON = JSON.stringify(Object.assign({}, baseConfig, {
worker: assets.has("worker.js") ? assets.resolve(`worker.js`) : null,
downloadSandbox: assets.resolve("download-sandbox.html"),
serviceWorker: "sw.js",
olm: {
wasm: assets.resolve("olm.wasm"),
legacyBundle: assets.resolve("olm_legacy.js"),
wasmBundle: assets.resolve("olm.js"),
}
}));
const modernScript = `import {main, Platform} from "./${assets.resolve(`hydrogen.js`)}"; main(new Platform(document.body, ${configJSON}));`;
const mainScripts = [
`<script type="module">${wrapWithLicenseComments(modernScript)}</script>`
];
if (!modernOnly) {
const legacyScript = `hydrogen.main(new hydrogen.Platform(document.body, ${configJSON}));`;
mainScripts.push(
`<script type="text/javascript" nomodule src="${assets.resolve(`hydrogen-legacy.js`)}"></script>`,
`<script type="text/javascript" nomodule>${wrapWithLicenseComments(legacyScript)}</script>`
);
}
doc("script#main").replaceWith(mainScripts.join(""));
const versionScript = doc("script#version");
versionScript.attr("type", "text/javascript");
let vSource = versionScript.contents().text();
vSource = vSource.replace(`"%%VERSION%%"`, `"${version}"`);
vSource = vSource.replace(`"%%GLOBAL_HASH%%"`, `"${globalHash}"`);
versionScript.text(wrapWithLicenseComments(vSource));
doc("head").append(`<link rel="manifest" href="${assets.resolve("manifest.json")}">`);
await assets.writeUnhashed("index.html", doc.html());
}
async function buildJs(mainFile, extraFiles, importOverrides) {
// create js bundle
const plugins = [multi(), removeJsComments({comments: "none"})];
if (importOverrides) {
plugins.push(overridesAsRollupPlugin(importOverrides));
}
const bundle = await rollup({
// for fake-indexeddb, so usage for tests only doesn't put it in bundle
treeshake: {moduleSideEffects: isPathInSrcDir},
input: extraFiles.concat(mainFile),
plugins
});
const {output} = await bundle.generate({
format: 'es',
// TODO: can remove this?
name: `hydrogen`
});
const code = output[0].code;
return wrapWithLicenseComments(code);
}
async function buildJsLegacy(mainFile, extraFiles, importOverrides) {
// compile down to whatever IE 11 needs
const babelPlugin = babel.babel({
babelHelpers: 'bundled',
exclude: 'node_modules/**',
presets: [
[
"@babel/preset-env",
{
useBuiltIns: "entry",
corejs: "3.4",
targets: "IE 11",
// we provide our own promise polyfill (es6-promise)
// with support for synchronous flushing of
// the queue for idb where needed
exclude: ["es.promise", "es.promise.all-settled", "es.promise.finally"]
}
]
]
});
const plugins = [multi(), commonjs()];
if (importOverrides) {
plugins.push(overridesAsRollupPlugin(importOverrides));
}
plugins.push(nodeResolve(), babelPlugin);
// create js bundle
const rollupConfig = {
// for fake-indexeddb, so usage for tests only doesn't put it in bundle
treeshake: {moduleSideEffects: isPathInSrcDir},
// important the extraFiles come first,
// so polyfills are available in the global scope
// if needed for the mainfile
input: extraFiles.concat(mainFile),
plugins
};
const bundle = await rollup(rollupConfig);
const {output} = await bundle.generate({
format: 'iife',
name: `hydrogen`
});
const code = output[0].code;
return wrapWithLicenseComments(code);
}
function wrapWithLicenseComments(code) {
// Add proper license comments to make GNU LibreJS accept the file
const start = '// @license magnet:?xt=urn:btih:8e4f440f4c65981c5bf93c76d35135ba5064d8b7&dn=apache-2.0.txt Apache-2.0';
const end = '// @license-end';
return `${start}\n${code}\n${end}`;
}
const NON_PRECACHED_JS = [
"hydrogen-legacy.js",
"olm_legacy.js",
"worker.js"
];
function isPreCached(asset) {
return asset.endsWith(".svg") ||
asset.endsWith(".png") ||
asset.endsWith(".css") ||
asset.endsWith(".wasm") ||
asset.endsWith(".html") ||
// most environments don't need the worker
asset.endsWith(".js") && !NON_PRECACHED_JS.includes(asset);
}
async function buildManifest(assets) {
const webManifest = JSON.parse(await fs.readFile(path.join(projectDir, "assets/manifest.json"), "utf8"));
// copy manifest icons
for (const icon of webManifest.icons) {
let iconData = await fs.readFile(path.join(projectDir, icon.src));
const iconTargetPath = path.basename(icon.src);
icon.src = await assets.write(iconTargetPath, iconData);
}
await assets.write("manifest.json", JSON.stringify(webManifest));
}
async function buildServiceWorker(swSource, version, globalHash, assets) {
const unhashedPreCachedAssets = ["index.html"];
const hashedPreCachedAssets = [];
const hashedCachedOnRequestAssets = [];
for (const [unresolved, resolved] of assets) {
if (unresolved === resolved) {
unhashedPreCachedAssets.push(resolved);
} else if (isPreCached(unresolved)) {
hashedPreCachedAssets.push(resolved);
} else {
hashedCachedOnRequestAssets.push(resolved);
}
}
const replaceArrayInSource = (name, value) => {
const newSource = swSource.replace(`${name} = []`, `${name} = ${JSON.stringify(value)}`);
if (newSource === swSource) {
throw new Error(`${name} was not found in the service worker source`);
}
return newSource;
};
const replaceStringInSource = (name, value) => {
const newSource = swSource.replace(new RegExp(`${name}\\s=\\s"[^"]*"`), `${name} = ${JSON.stringify(value)}`);
if (newSource === swSource) {
throw new Error(`${name} was not found in the service worker source`);
}
return newSource;
};
// write service worker
swSource = swSource.replace(`"%%VERSION%%"`, `"${version}"`);
swSource = swSource.replace(`"%%GLOBAL_HASH%%"`, `"${globalHash}"`);
swSource = replaceArrayInSource("UNHASHED_PRECACHED_ASSETS", unhashedPreCachedAssets);
swSource = replaceArrayInSource("HASHED_PRECACHED_ASSETS", hashedPreCachedAssets);
swSource = replaceArrayInSource("HASHED_CACHED_ON_REQUEST_ASSETS", hashedCachedOnRequestAssets);
swSource = replaceStringInSource("NOTIFICATION_BADGE_ICON", assets.resolve("icon.png"));
// service worker should not have a hashed name as it is polled by the browser for updates
await assets.writeUnhashed("sw.js", swSource);
}
async function buildCssBundles(buildFn, themes, assets, mainCssFile = null) {
if (!mainCssFile) {
mainCssFile = path.join(cssSrcDir, "main.css");
}
const bundleCss = await buildFn(mainCssFile);
await assets.write(`hydrogen.css`, bundleCss);
for (const theme of themes) {
const themeRelPath = `themes/${theme}/`;
const themeRoot = path.join(cssSrcDir, themeRelPath);
const assetUrlMapper = ({absolutePath}) => {
if (!absolutePath.startsWith(themeRoot)) {
throw new Error("resource is out of theme directory: " + absolutePath);
}
const relPath = absolutePath.substr(themeRoot.length);
const hashedDstPath = assets.resolve(path.join(themeRelPath, relPath));
if (hashedDstPath) {
return hashedDstPath.substr(themeRelPath.length);
}
};
const themeCss = await buildFn(path.join(themeRoot, `theme.css`), assetUrlMapper);
await assets.write(path.join(themeRelPath, `bundle.css`), themeCss);
}
}
// async function buildCss(entryPath, urlMapper = null) {
// const preCss = await fs.readFile(entryPath, "utf8");
// const options = [postcssImport];
// if (urlMapper) {
// options.push(postcssUrl({url: urlMapper}));
// }
// const cssBundler = postcss(options);
// const result = await cssBundler.process(preCss, {from: entryPath});
// return result.css;
// }
async function buildCssLegacy(entryPath, urlMapper = null) {
const preCss = await fs.readFile(entryPath, "utf8");
const options = [
postcssImport,
cssvariables({
preserve: (declaration) => {
return declaration.value.indexOf("var(--ios-") == 0;
}
}),
autoprefixer({overrideBrowserslist: ["IE 11"], grid: "no-autoplace"}),
flexbugsFixes()
];
if (urlMapper) {
options.push(postcssUrl({url: urlMapper}));
}
const cssBundler = postcss(options);
const result = await cssBundler.process(preCss, {from: entryPath});
return result.css;
}
async function removeDirIfExists(targetDir) {
try {
await fs.rmdir(targetDir, {recursive: true});
} catch (err) {
if (err.code !== "ENOENT") {
throw err;
}
}
}
async function copyFolder(srcRoot, dstRoot, filter, assets = null) {
assets = assets || new AssetMap(dstRoot);
const dirEnts = await fs.readdir(srcRoot, {withFileTypes: true});
for (const dirEnt of dirEnts) {
const dstPath = path.join(dstRoot, dirEnt.name);
const srcPath = path.join(srcRoot, dirEnt.name);
if (dirEnt.isDirectory()) {
await fs.mkdir(dstPath);
await copyFolder(srcPath, dstPath, filter, assets);
} else if ((dirEnt.isFile() || dirEnt.isSymbolicLink()) && (!filter || filter(srcPath))) {
const content = await fs.readFile(srcPath);
await assets.write(dstPath, content);
}
}
return assets;
}
function contentHash(str) {
var hasher = new xxhash.h32(0);
hasher.update(str);
return hasher.digest();
}
class AssetMap {
constructor(targetDir) {
// remove last / if any, so substr in create works well
this._targetDir = path.resolve(targetDir);
this._assets = new Map();
// hashes for unhashed resources so changes in these resources also contribute to the hashForAll
this._unhashedHashes = [];
}
_toRelPath(resourcePath) {
let relPath = resourcePath;
if (path.isAbsolute(resourcePath)) {
if (!resourcePath.startsWith(this._targetDir)) {
throw new Error(`absolute path ${resourcePath} that is not within target dir ${this._targetDir}`);
}
relPath = resourcePath.substr(this._targetDir.length + 1); // + 1 for the /
}
return relPath;
}
_create(resourcePath, content) {
const relPath = this._toRelPath(resourcePath);
const hash = contentHash(Buffer.from(content));
const dir = path.dirname(relPath);
const extname = path.extname(relPath);
const basename = path.basename(relPath, extname);
const dstRelPath = path.join(dir, `${basename}-${hash}${extname}`);
this._assets.set(relPath, dstRelPath);
return dstRelPath;
}
async write(resourcePath, content) {
const relPath = this._create(resourcePath, content);
const fullPath = path.join(this.directory, relPath);
if (typeof content === "string") {
await fs.writeFile(fullPath, content, "utf8");
} else {
await fs.writeFile(fullPath, content);
}
return relPath;
}
async writeUnhashed(resourcePath, content) {
const relPath = this._toRelPath(resourcePath);
this._assets.set(relPath, relPath);
const fullPath = path.join(this.directory, relPath);
if (typeof content === "string") {
await fs.writeFile(fullPath, content, "utf8");
} else {
await fs.writeFile(fullPath, content);
}
return relPath;
}
get directory() {
return this._targetDir;
}
resolve(resourcePath) {
const relPath = this._toRelPath(resourcePath);
const result = this._assets.get(relPath);
if (!result) {
throw new Error(`unknown path: ${relPath}, only know ${Array.from(this._assets.keys()).join(", ")}`);
}
return result;
}
addSubMap(assetMap) {
if (!assetMap.directory.startsWith(this.directory)) {
throw new Error(`map directory doesn't start with this directory: ${assetMap.directory} ${this.directory}`);
}
const relSubRoot = assetMap.directory.substr(this.directory.length + 1);
for (const [key, value] of assetMap._assets.entries()) {
this._assets.set(path.join(relSubRoot, key), path.join(relSubRoot, value));
}
}
[Symbol.iterator]() {
return this._assets.entries();
}
isUnhashed(relPath) {
const resolvedPath = this._assets.get(relPath);
if (!resolvedPath) {
throw new Error("Unknown asset: " + relPath);
}
return relPath === resolvedPath;
}
get size() {
return this._assets.size;
}
has(relPath) {
return this._assets.has(relPath);
}
hashForAll() {
const globalHashAssets = Array.from(this).map(([, resolved]) => resolved);
globalHashAssets.push(...this._unhashedHashes);
globalHashAssets.sort();
return contentHash(globalHashAssets.join(","));
}
addToHashForAll(resourcePath, content) {
this._unhashedHashes.push(`${resourcePath}-${contentHash(Buffer.from(content))}`);
}
}
async function readImportOverrides(filename) {
const json = await fs.readFile(filename, "utf8");
const mapping = new Map(Object.entries(JSON.parse(json)));
return {
basedir: path.dirname(path.resolve(filename))+path.sep,
mapping
};
}
function overridesAsRollupPlugin(importOverrides) {
const {mapping, basedir} = importOverrides;
return {
name: "rewrite-imports",
resolveId (source, importer) {
let file;
if (source.startsWith(path.sep)) {
file = source;
} else {
file = path.join(path.dirname(importer), source);
}
if (file.startsWith(basedir)) {
const searchPath = file.substr(basedir.length);
const replacingPath = mapping.get(searchPath);
if (replacingPath) {
console.info(`replacing ${searchPath} with ${replacingPath}`);
return path.join(basedir, replacingPath);
}
}
return null;
}
};
}
build(parameters).catch(err => console.error(err));

View file

@ -1,12 +0,0 @@
import fsRoot from "fs";
const fs = fsRoot.promises;
export async function removeDirIfExists(targetDir) {
try {
await fs.rmdir(targetDir, {recursive: true});
} catch (err) {
if (err.code !== "ENOENT") {
throw err;
}
}
}

View file

@ -1,6 +0,0 @@
git checkout gh-pages
cp -R target/* .
git add $(find . -maxdepth 1 -type f)
git add themes
git commit -m "update hydrogen"
git checkout master

View file

@ -1,6 +1,7 @@
module.exports = class Buffer { var Buffer = {
static isBuffer(array) {return array instanceof Uint8Array;} isBuffer: function(array) {return array instanceof Uint8Array;},
static from(arrayBuffer) {return arrayBuffer;} from: function(arrayBuffer) {return arrayBuffer;},
static allocUnsafe(size) {return Buffer.alloc(size);} allocUnsafe: function(size) {return Buffer.alloc(size);},
static alloc(size) {return new Uint8Array(size);} alloc: function(size) {return new Uint8Array(size);}
}; };
export default Buffer;

View file

@ -1,4 +0,0 @@
// we have our own main file for this module as we need both these symbols to
// be exported, and we also don't want to auto behaviour that modifies global vars
exports.FDBFactory = require("fake-indexeddb/lib/FDBFactory.js");
exports.FDBKeyRange = require("fake-indexeddb/lib/FDBKeyRange.js");

View file

@ -1 +1,2 @@
module.exports.Buffer = require("buffer"); import Buffer from "buffer";
export {Buffer};

View file

@ -1,132 +0,0 @@
/*
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
const fsRoot = require("fs");
const fs = fsRoot.promises;
const path = require("path");
const { rollup } = require('rollup');
const { fileURLToPath } = require('url');
const { dirname } = require('path');
// needed to translate commonjs modules to esm
const commonjs = require('@rollup/plugin-commonjs');
const json = require('@rollup/plugin-json');
const { nodeResolve } = require('@rollup/plugin-node-resolve');
const projectDir = path.join(__dirname, "../");
async function removeDirIfExists(targetDir) {
try {
await fs.rmdir(targetDir, {recursive: true});
} catch (err) {
if (err.code !== "ENOENT") {
throw err;
}
}
}
/** function used to resolve common-js require calls below. */
function packageIterator(request, start, defaultIterator) {
// this is just working for bs58, would need to tune it further for other dependencies
if (request === "safe-buffer") {
return [path.join(projectDir, "/scripts/package-overrides/safe-buffer")];
} else if (request === "buffer/") {
return [path.join(projectDir, "/scripts/package-overrides/buffer")];
} else {
return defaultIterator();
}
}
async function commonjsToESM(src, dst) {
// create js bundle
const bundle = await rollup({
treeshake: {moduleSideEffects: false},
input: src,
plugins: [commonjs(), json(), nodeResolve({
browser: true,
preferBuiltins: false,
customResolveOptions: {packageIterator}
})]
});
const {output} = await bundle.generate({
format: 'es'
});
const code = output[0].code;
await fs.writeFile(dst, code, "utf8");
}
async function populateLib() {
const libDir = path.join(projectDir, "lib/");
await removeDirIfExists(libDir);
await fs.mkdir(libDir);
const olmSrcDir = path.dirname(require.resolve("@matrix-org/olm"));
const olmDstDir = path.join(libDir, "olm/");
await fs.mkdir(olmDstDir);
for (const file of ["olm.js", "olm.wasm", "olm_legacy.js"]) {
await fs.copyFile(path.join(olmSrcDir, file), path.join(olmDstDir, file));
}
// transpile node-html-parser to esm
await fs.mkdir(path.join(libDir, "node-html-parser/"));
await commonjsToESM(
require.resolve('node-html-parser/dist/index.js'),
path.join(libDir, "node-html-parser/index.js")
);
// transpile another-json to esm
await fs.mkdir(path.join(libDir, "another-json/"));
await commonjsToESM(
require.resolve('another-json/another-json.js'),
path.join(libDir, "another-json/index.js")
);
// transpile bs58 to esm
await fs.mkdir(path.join(libDir, "bs58/"));
await commonjsToESM(
require.resolve('bs58/index.js'),
path.join(libDir, "bs58/index.js")
);
// transpile base64-arraybuffer to esm
await fs.mkdir(path.join(libDir, "base64-arraybuffer/"));
await commonjsToESM(
require.resolve('base64-arraybuffer/lib/base64-arraybuffer.js'),
path.join(libDir, "base64-arraybuffer/index.js")
);
// this probably should no go in here, we can just import "aes-js" from legacy-extras.js
// as that file is never loaded from a browser
// transpile aesjs to esm
await fs.mkdir(path.join(libDir, "aes-js/"));
await commonjsToESM(
require.resolve('aes-js/index.js'),
path.join(libDir, "aes-js/index.js")
);
// es6-promise is already written as an es module,
// but it does need to be babelified, and current we don't babelify
// anything in node_modules in the build script, so make a bundle that
// is conveniently not placed in node_modules rather than symlinking.
await fs.mkdir(path.join(libDir, "es6-promise/"));
await commonjsToESM(
require.resolve('es6-promise/lib/es6-promise/promise.js'),
path.join(libDir, "es6-promise/index.js")
);
// fake-indexeddb, used for tests (but unresolvable bare imports also makes the build complain)
// and might want to use it for in-memory storage too, although we probably do ts->es6 with esm
// directly rather than ts->es5->es6 as we do now. The bundle is 240K currently.
await fs.mkdir(path.join(libDir, "fake-indexeddb/"));
await commonjsToESM(
path.join(projectDir, "/scripts/package-overrides/fake-indexeddb.js"),
path.join(libDir, "fake-indexeddb/index.js")
);
}
populateLib();

View file

@ -1,43 +0,0 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
const finalhandler = require('finalhandler')
const http = require('http')
const serveStatic = require('serve-static')
const path = require('path');
// Serve up parent directory with cache disabled
const serve = serveStatic(
path.resolve(__dirname, "../"),
{
etag: false,
setHeaders: res => {
res.setHeader("Pragma", "no-cache");
res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
res.setHeader("Expires", "Wed, 21 Oct 2015 07:28:00 GMT");
},
index: ['index.html', 'index.htm']
}
);
// Create server
const server = http.createServer(function onRequest (req, res) {
console.log(req.method, req.url);
serve(req, res, finalhandler(req, res))
});
// Listen
server.listen(3000);

View file

@ -1,37 +0,0 @@
// Snowpack Configuration File
// See all supported options: https://www.snowpack.dev/reference/configuration
/** @type {import("snowpack").SnowpackUserConfig } */
module.exports = {
mount: {
// More specific paths before less specific paths (if they overlap)
"src/platform/web/docroot": "/",
"src": "/src",
"lib": {url: "/lib", static: true },
"assets": "/assets",
/* ... */
},
exclude: [
/* Avoid scanning scripts which use dev-dependencies and pull in babel, rollup, etc. */
'**/node_modules/**/*',
'**/scripts/**',
'**/target/**',
'**/prototypes/**',
'**/src/platform/web/legacy-polyfill.js',
'**/src/platform/web/worker/polyfill.js'
],
plugins: [
/* ... */
],
packageOptions: {
/* ... */
},
devOptions: {
open: "none",
hmr: false,
/* ... */
},
buildOptions: {
/* ... */
},
};

View file

@ -19,7 +19,7 @@ limitations under the License.
// we do need to return a disposable from EventEmitter.on, or at least have a method here to easily track a subscription to an EventEmitter // we do need to return a disposable from EventEmitter.on, or at least have a method here to easily track a subscription to an EventEmitter
import {EventEmitter} from "../utils/EventEmitter"; import {EventEmitter} from "../utils/EventEmitter";
import {Disposables} from "../utils/Disposables.js"; import {Disposables} from "../utils/Disposables";
export class ViewModel extends EventEmitter { export class ViewModel extends EventEmitter {
constructor(options = {}) { constructor(options = {}) {

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {ViewModel} from "../ViewModel.js"; import {ViewModel} from "../ViewModel.js";
import {createEnum} from "../../utils/enum.js"; import {createEnum} from "../../utils/enum";
import {ConnectionStatus} from "../../matrix/net/Reconnector.js"; import {ConnectionStatus} from "../../matrix/net/Reconnector.js";
import {SyncStatus} from "../../matrix/Sync.js"; import {SyncStatus} from "../../matrix/Sync.js";

View file

@ -184,11 +184,11 @@ import {Clock as MockClock} from "../../../../mocks/Clock.js";
import {createMockStorage} from "../../../../mocks/Storage"; import {createMockStorage} from "../../../../mocks/Storage";
import {ListObserver} from "../../../../mocks/ListObserver.js"; import {ListObserver} from "../../../../mocks/ListObserver.js";
import {createEvent, withTextBody, withContent} from "../../../../mocks/event.js"; import {createEvent, withTextBody, withContent} from "../../../../mocks/event.js";
import {NullLogItem, NullLogger} from "../../../../logging/NullLogger.js"; import {NullLogItem, NullLogger} from "../../../../logging/NullLogger";
import {HomeServer as MockHomeServer} from "../../../../mocks/HomeServer.js"; import {HomeServer as MockHomeServer} from "../../../../mocks/HomeServer.js";
// other imports // other imports
import {BaseMessageTile} from "./tiles/BaseMessageTile.js"; import {BaseMessageTile} from "./tiles/BaseMessageTile.js";
import {MappedList} from "../../../../observable/list/MappedList.js"; import {MappedList} from "../../../../observable/list/MappedList";
import {ObservableValue} from "../../../../observable/ObservableValue"; import {ObservableValue} from "../../../../observable/ObservableValue";
import {PowerLevels} from "../../../../matrix/room/PowerLevels.js"; import {PowerLevels} from "../../../../matrix/room/PowerLevels.js";

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {BaseObservableList} from "../../../../observable/list/BaseObservableList"; import {BaseObservableList} from "../../../../observable/list/BaseObservableList";
import {sortedIndex} from "../../../../utils/sortedIndex.js"; import {sortedIndex} from "../../../../utils/sortedIndex";
// maps 1..n entries to 0..1 tile. Entries are what is stored in the timeline, either an event or fragmentboundary // maps 1..n entries to 0..1 tile. Entries are what is stored in the timeline, either an event or fragmentboundary
// for now, tileCreator should be stable in whether it returns a tile or not. // for now, tileCreator should be stable in whether it returns a tile or not.
@ -253,7 +253,7 @@ export class TilesCollection extends BaseObservableList {
} }
} }
import {ObservableArray} from "../../../../observable/list/ObservableArray.js"; import {ObservableArray} from "../../../../observable/list/ObservableArray";
import {UpdateAction} from "./UpdateAction.js"; import {UpdateAction} from "./UpdateAction.js";
export function tests() { export function tests() {

View file

@ -352,9 +352,13 @@ export function parseHTMLBody(platform, mediaRepository, allowReplies, html) {
return new MessageBody(html, parts); return new MessageBody(html, parts);
} }
import parse from '../../../../../lib/node-html-parser/index.js';
export function tests() { export async function tests() {
// don't import node-html-parser until it's safe to assume we're actually in a unit test,
// as this is a devDependency
const nodeHtmlParser = await import("node-html-parser");
const {parse} = nodeHtmlParser.default;
class HTMLParseResult { class HTMLParseResult {
constructor(bodyNode) { constructor(bodyNode) {
this._bodyNode = bodyNode; this._bodyNode = bodyNode;

View file

@ -16,7 +16,7 @@ limitations under the License.
import {BaseMessageTile} from "./BaseMessageTile.js"; import {BaseMessageTile} from "./BaseMessageTile.js";
import {stringAsBody} from "../MessageBody.js"; import {stringAsBody} from "../MessageBody.js";
import {createEnum} from "../../../../../utils/enum.js"; import {createEnum} from "../../../../../utils/enum";
export const BodyFormat = createEnum("Plain", "Html"); export const BodyFormat = createEnum("Plain", "Html");

View file

@ -16,7 +16,7 @@ limitations under the License.
*/ */
import {BaseMessageTile} from "./BaseMessageTile.js"; import {BaseMessageTile} from "./BaseMessageTile.js";
import {formatSize} from "../../../../../utils/formatSize.js"; import {formatSize} from "../../../../../utils/formatSize";
import {SendStatus} from "../../../../../matrix/room/sending/PendingEvent.js"; import {SendStatus} from "../../../../../matrix/room/sending/PendingEvent.js";
export class FileTile extends BaseMessageTile { export class FileTile extends BaseMessageTile {

View file

@ -16,7 +16,7 @@ limitations under the License.
import {ViewModel} from "../../ViewModel.js"; import {ViewModel} from "../../ViewModel.js";
import {KeyType} from "../../../matrix/ssss/index.js"; import {KeyType} from "../../../matrix/ssss/index.js";
import {createEnum} from "../../../utils/enum.js"; import {createEnum} from "../../../utils/enum";
export const Status = createEnum("Enabled", "SetupKey", "SetupPhrase", "Pending"); export const Status = createEnum("Enabled", "SetupKey", "SetupPhrase", "Pending");

View file

@ -15,23 +15,29 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {LogItem} from "./LogItem.js"; import {LogItem} from "./LogItem";
import {LogLevel, LogFilter} from "./LogFilter.js"; import {LogLevel, LogFilter} from "./LogFilter";
import type {ILogger, ILogExport, FilterCreator, LabelOrValues, LogCallback, ILogItem, ISerializedItem} from "./types";
import type {Platform} from "../platform/web/Platform.js";
export class BaseLogger { export abstract class BaseLogger implements ILogger {
constructor({platform}) { protected _openItems: Set<LogItem> = new Set();
this._openItems = new Set(); protected _platform: Platform;
protected _serializedTransformer: (item: ISerializedItem) => ISerializedItem;
constructor({platform, serializedTransformer = (item: ISerializedItem) => item}) {
this._platform = platform; this._platform = platform;
this._serializedTransformer = serializedTransformer;
} }
log(labelOrValues, logLevel = LogLevel.Info) { log(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info): void {
const item = new LogItem(labelOrValues, logLevel, null, this); const item = new LogItem(labelOrValues, logLevel, this);
item._end = item._start; item.end = item.start;
this._persistItem(item, null, false); this._persistItem(item, undefined, false);
} }
/** if item is a log item, wrap the callback in a child of it, otherwise start a new root log item. */ /** if item is a log item, wrap the callback in a child of it, otherwise start a new root log item. */
wrapOrRun(item, labelOrValues, callback, logLevel = null, filterCreator = null) { wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T {
if (item) { if (item) {
return item.wrap(labelOrValues, callback, logLevel, filterCreator); return item.wrap(labelOrValues, callback, logLevel, filterCreator);
} else { } else {
@ -43,28 +49,31 @@ export class BaseLogger {
where the (async) result or errors are not propagated but still logged. where the (async) result or errors are not propagated but still logged.
Useful to pair with LogItem.refDetached. Useful to pair with LogItem.refDetached.
@return {LogItem} the log item added, useful to pass to LogItem.refDetached */ @return {ILogItem} the log item added, useful to pass to LogItem.refDetached */
runDetached(labelOrValues, callback, logLevel = null, filterCreator = null) { runDetached<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem {
if (logLevel === null) { if (!logLevel) {
logLevel = LogLevel.Info; logLevel = LogLevel.Info;
} }
const item = new LogItem(labelOrValues, logLevel, null, this); const item = new LogItem(labelOrValues, logLevel, this);
this._run(item, callback, logLevel, filterCreator, false /* don't throw, nobody is awaiting */); this._run(item, callback, logLevel, false /* don't throw, nobody is awaiting */, filterCreator);
return item; return item;
} }
/** run a callback wrapped in a log operation. /** run a callback wrapped in a log operation.
Errors and duration are transparently logged, also for async operations. Errors and duration are transparently logged, also for async operations.
Whatever the callback returns is returned here. */ Whatever the callback returns is returned here. */
run(labelOrValues, callback, logLevel = null, filterCreator = null) { run<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T {
if (logLevel === null) { if (logLevel === undefined) {
logLevel = LogLevel.Info; logLevel = LogLevel.Info;
} }
const item = new LogItem(labelOrValues, logLevel, null, this); const item = new LogItem(labelOrValues, logLevel, this);
return this._run(item, callback, logLevel, filterCreator, true); return this._run(item, callback, logLevel, true, filterCreator);
} }
_run(item, callback, logLevel, filterCreator, shouldThrow) { _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: true, filterCreator?: FilterCreator): T;
// we don't return if we don't throw, as we don't have anything to return when an error is caught but swallowed for the fire-and-forget case.
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: false, filterCreator?: FilterCreator): void;
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: boolean, filterCreator?: FilterCreator): T | void {
this._openItems.add(item); this._openItems.add(item);
const finishItem = () => { const finishItem = () => {
@ -88,24 +97,29 @@ export class BaseLogger {
}; };
try { try {
const result = item.run(callback); let result = item.run(callback);
if (result instanceof Promise) { if (result instanceof Promise) {
return result.then(promiseResult => { result = result.then(promiseResult => {
finishItem(); finishItem();
return promiseResult; return promiseResult;
}, err => { }, err => {
finishItem(); finishItem();
if (shouldThrow) { if (wantResult) {
throw err; throw err;
} }
}); }) as unknown as T;
if (wantResult) {
return result;
}
} else { } else {
finishItem(); finishItem();
return result; if(wantResult) {
return result;
}
} }
} catch (err) { } catch (err) {
finishItem(); finishItem();
if (shouldThrow) { if (wantResult) {
throw err; throw err;
} }
} }
@ -127,24 +141,20 @@ export class BaseLogger {
this._openItems.clear(); this._openItems.clear();
} }
_persistItem() { abstract _persistItem(item: LogItem, filter?: LogFilter, forced?: boolean): void;
throw new Error("not implemented");
}
async export() { abstract export(): Promise<ILogExport | undefined>;
throw new Error("not implemented");
}
// expose log level without needing // expose log level without needing
get level() { get level(): typeof LogLevel {
return LogLevel; return LogLevel;
} }
_now() { _now(): number {
return this._platform.clock.now(); return this._platform.clock.now();
} }
_createRefId() { _createRefId(): number {
return Math.round(this._platform.random() * Number.MAX_SAFE_INTEGER); return Math.round(this._platform.random() * Number.MAX_SAFE_INTEGER);
} }
} }

View file

@ -13,32 +13,35 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseLogger} from "./BaseLogger.js"; import {BaseLogger} from "./BaseLogger";
import {LogItem} from "./LogItem";
import type {ILogItem, LogItemValues, ILogExport} from "./types";
export class ConsoleLogger extends BaseLogger { export class ConsoleLogger extends BaseLogger {
_persistItem(item) { _persistItem(item: LogItem): void {
printToConsole(item); printToConsole(item);
} }
async export(): Promise<ILogExport | undefined> {
return undefined;
}
} }
const excludedKeysFromTable = ["l", "id"]; const excludedKeysFromTable = ["l", "id"];
function filterValues(values) { function filterValues(values: LogItemValues): LogItemValues | null {
if (!values) {
return null;
}
return Object.entries(values) return Object.entries(values)
.filter(([key]) => !excludedKeysFromTable.includes(key)) .filter(([key]) => !excludedKeysFromTable.includes(key))
.reduce((obj, [key, value]) => { .reduce((obj: LogItemValues, [key, value]) => {
obj = obj || {}; obj = obj || {};
obj[key] = value; obj[key] = value;
return obj; return obj;
}, null); }, null);
} }
function printToConsole(item) { function printToConsole(item: LogItem): void {
const label = `${itemCaption(item)} (${item.duration}ms)`; const label = `${itemCaption(item)} (${item.duration}ms)`;
const filteredValues = filterValues(item._values); const filteredValues = filterValues(item.values);
const shouldGroup = item._children || filteredValues; const shouldGroup = item.children || filteredValues;
if (shouldGroup) { if (shouldGroup) {
if (item.error) { if (item.error) {
console.group(label); console.group(label);
@ -58,8 +61,8 @@ function printToConsole(item) {
if (filteredValues) { if (filteredValues) {
console.table(filteredValues); console.table(filteredValues);
} }
if (item._children) { if (item.children) {
for(const c of item._children) { for(const c of item.children) {
printToConsole(c); printToConsole(c);
} }
} }
@ -68,18 +71,18 @@ function printToConsole(item) {
} }
} }
function itemCaption(item) { function itemCaption(item: ILogItem): string {
if (item._values.t === "network") { if (item.values.t === "network") {
return `${item._values.method} ${item._values.url}`; return `${item.values.method} ${item.values.url}`;
} else if (item._values.l && typeof item._values.id !== "undefined") { } else if (item.values.l && typeof item.values.id !== "undefined") {
return `${item._values.l} ${item._values.id}`; return `${item.values.l} ${item.values.id}`;
} else if (item._values.l && typeof item._values.status !== "undefined") { } else if (item.values.l && typeof item.values.status !== "undefined") {
return `${item._values.l} (${item._values.status})`; return `${item.values.l} (${item.values.status})`;
} else if (item._values.l && item.error) { } else if (item.values.l && item.error) {
return `${item._values.l} failed`; return `${item.values.l} failed`;
} else if (typeof item._values.ref !== "undefined") { } else if (typeof item.values.ref !== "undefined") {
return `ref ${item._values.ref}`; return `ref ${item.values.ref}`;
} else { } else {
return item._values.l || item._values.type; return item.values.l || item.values.type;
} }
} }

View file

@ -22,10 +22,25 @@ import {
iterateCursor, iterateCursor,
fetchResults, fetchResults,
} from "../matrix/storage/idb/utils"; } from "../matrix/storage/idb/utils";
import {BaseLogger} from "./BaseLogger.js"; import {BaseLogger} from "./BaseLogger";
import type {Interval} from "../platform/web/dom/Clock";
import type {Platform} from "../platform/web/Platform.js";
import type {BlobHandle} from "../platform/web/dom/BlobHandle.js";
import type {ILogItem, ILogExport, ISerializedItem} from "./types";
import type {LogFilter} from "./LogFilter";
type QueuedItem = {
json: string;
id?: number;
}
export class IDBLogger extends BaseLogger { export class IDBLogger extends BaseLogger {
constructor(options) { private readonly _name: string;
private readonly _limit: number;
private readonly _flushInterval: Interval;
private _queuedItems: QueuedItem[];
constructor(options: {name: string, flushInterval?: number, limit?: number, platform: Platform, serializedTransformer?: (item: ISerializedItem) => ISerializedItem}) {
super(options); super(options);
const {name, flushInterval = 60 * 1000, limit = 3000} = options; const {name, flushInterval = 60 * 1000, limit = 3000} = options;
this._name = name; this._name = name;
@ -36,18 +51,19 @@ export class IDBLogger extends BaseLogger {
this._flushInterval = this._platform.clock.createInterval(() => this._tryFlush(), flushInterval); this._flushInterval = this._platform.clock.createInterval(() => this._tryFlush(), flushInterval);
} }
dispose() { // TODO: move dispose to ILogger, listen to pagehide elsewhere and call dispose from there, which calls _finishAllAndFlush
dispose(): void {
window.removeEventListener("pagehide", this, false); window.removeEventListener("pagehide", this, false);
this._flushInterval.dispose(); this._flushInterval.dispose();
} }
handleEvent(evt) { handleEvent(evt: Event): void {
if (evt.type === "pagehide") { if (evt.type === "pagehide") {
this._finishAllAndFlush(); this._finishAllAndFlush();
} }
} }
async _tryFlush() { async _tryFlush(): Promise<void> {
const db = await this._openDB(); const db = await this._openDB();
try { try {
const txn = db.transaction(["logs"], "readwrite"); const txn = db.transaction(["logs"], "readwrite");
@ -77,13 +93,13 @@ export class IDBLogger extends BaseLogger {
} }
} }
_finishAllAndFlush() { _finishAllAndFlush(): void {
this._finishOpenItems(); this._finishOpenItems();
this.log({l: "pagehide, closing logs", t: "navigation"}); this.log({l: "pagehide, closing logs", t: "navigation"});
this._persistQueuedItems(this._queuedItems); this._persistQueuedItems(this._queuedItems);
} }
_loadQueuedItems() { _loadQueuedItems(): QueuedItem[] {
const key = `${this._name}_queuedItems`; const key = `${this._name}_queuedItems`;
try { try {
const json = window.localStorage.getItem(key); const json = window.localStorage.getItem(key);
@ -97,18 +113,21 @@ export class IDBLogger extends BaseLogger {
return []; return [];
} }
_openDB() { _openDB(): Promise<IDBDatabase> {
return openDatabase(this._name, db => db.createObjectStore("logs", {keyPath: "id", autoIncrement: true}), 1); return openDatabase(this._name, db => db.createObjectStore("logs", {keyPath: "id", autoIncrement: true}), 1);
} }
_persistItem(logItem, filter, forced) { _persistItem(logItem: ILogItem, filter: LogFilter, forced: boolean): void {
const serializedItem = logItem.serialize(filter, forced); const serializedItem = logItem.serialize(filter, undefined, forced);
this._queuedItems.push({ if (serializedItem) {
json: JSON.stringify(serializedItem) const transformedSerializedItem = this._serializedTransformer(serializedItem);
}); this._queuedItems.push({
json: JSON.stringify(transformedSerializedItem)
});
}
} }
_persistQueuedItems(items) { _persistQueuedItems(items: QueuedItem[]): void {
try { try {
window.localStorage.setItem(`${this._name}_queuedItems`, JSON.stringify(items)); window.localStorage.setItem(`${this._name}_queuedItems`, JSON.stringify(items));
} catch (e) { } catch (e) {
@ -116,12 +135,12 @@ export class IDBLogger extends BaseLogger {
} }
} }
async export() { async export(): Promise<ILogExport> {
const db = await this._openDB(); const db = await this._openDB();
try { try {
const txn = db.transaction(["logs"], "readonly"); const txn = db.transaction(["logs"], "readonly");
const logs = txn.objectStore("logs"); const logs = txn.objectStore("logs");
const storedItems = await fetchResults(logs.openCursor(), () => false); const storedItems: QueuedItem[] = await fetchResults(logs.openCursor(), () => false);
const allItems = storedItems.concat(this._queuedItems); const allItems = storedItems.concat(this._queuedItems);
return new IDBLogExport(allItems, this, this._platform); return new IDBLogExport(allItems, this, this._platform);
} finally { } finally {
@ -131,17 +150,20 @@ export class IDBLogger extends BaseLogger {
} }
} }
async _removeItems(items) { async _removeItems(items: QueuedItem[]): Promise<void> {
const db = await this._openDB(); const db = await this._openDB();
try { try {
const txn = db.transaction(["logs"], "readwrite"); const txn = db.transaction(["logs"], "readwrite");
const logs = txn.objectStore("logs"); const logs = txn.objectStore("logs");
for (const item of items) { for (const item of items) {
const queuedIdx = this._queuedItems.findIndex(i => i.id === item.id); if (typeof item.id === "number") {
if (queuedIdx === -1) {
logs.delete(item.id); logs.delete(item.id);
} else { } else {
this._queuedItems.splice(queuedIdx, 1); // assume the (non-persisted) object in each array will be the same
const queuedIdx = this._queuedItems.indexOf(item);
if (queuedIdx === -1) {
this._queuedItems.splice(queuedIdx, 1);
}
} }
} }
await txnAsPromise(txn); await txnAsPromise(txn);
@ -153,33 +175,37 @@ export class IDBLogger extends BaseLogger {
} }
} }
class IDBLogExport { class IDBLogExport implements ILogExport {
constructor(items, logger, platform) { private readonly _items: QueuedItem[];
private readonly _logger: IDBLogger;
private readonly _platform: Platform;
constructor(items: QueuedItem[], logger: IDBLogger, platform: Platform) {
this._items = items; this._items = items;
this._logger = logger; this._logger = logger;
this._platform = platform; this._platform = platform;
} }
get count() { get count(): number {
return this._items.length; return this._items.length;
} }
/** /**
* @return {Promise} * @return {Promise}
*/ */
removeFromStore() { removeFromStore(): Promise<void> {
return this._logger._removeItems(this._items); return this._logger._removeItems(this._items);
} }
asBlob() { asBlob(): BlobHandle {
const log = { const log = {
formatVersion: 1, formatVersion: 1,
appVersion: this._platform.updateService?.version, appVersion: this._platform.updateService?.version,
items: this._items.map(i => JSON.parse(i.json)) items: this._items.map(i => JSON.parse(i.json))
}; };
const json = JSON.stringify(log); const json = JSON.stringify(log);
const buffer = this._platform.encoding.utf8.encode(json); const buffer: Uint8Array = this._platform.encoding.utf8.encode(json);
const blob = this._platform.createBlob(buffer, "application/json"); const blob: BlobHandle = this._platform.createBlob(buffer, "application/json");
return blob; return blob;
} }
} }

View file

@ -14,31 +14,35 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
export const LogLevel = { import type {ILogItem, ISerializedItem} from "./types";
All: 1,
Debug: 2, export enum LogLevel {
Detail: 3, All = 1,
Info: 4, Debug,
Warn: 5, Detail,
Error: 6, Info,
Fatal: 7, Warn,
Off: 8, Error,
Fatal,
Off
} }
export class LogFilter { export class LogFilter {
constructor(parentFilter) { private _min?: LogLevel;
private _parentFilter?: LogFilter;
constructor(parentFilter?: LogFilter) {
this._parentFilter = parentFilter; this._parentFilter = parentFilter;
this._min = null;
} }
filter(item, children) { filter(item: ILogItem, children: ISerializedItem[] | null): boolean {
if (this._parentFilter) { if (this._parentFilter) {
if (!this._parentFilter.filter(item, children)) { if (!this._parentFilter.filter(item, children)) {
return false; return false;
} }
} }
// neither our children or us have a loglevel high enough, filter out. // neither our children or us have a loglevel high enough, filter out.
if (this._min !== null && !Array.isArray(children) && item.logLevel < this._min) { if (this._min !== undefined && !Array.isArray(children) && item.logLevel < this._min) {
return false; return false;
} else { } else {
return true; return true;
@ -46,7 +50,7 @@ export class LogFilter {
} }
/* methods to build the filter */ /* methods to build the filter */
minLevel(logLevel) { minLevel(logLevel: LogLevel): LogFilter {
this._min = logLevel; this._min = logLevel;
return this; return this;
} }

View file

@ -15,39 +15,47 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {LogLevel, LogFilter} from "./LogFilter.js"; import {LogLevel, LogFilter} from "./LogFilter";
import type {BaseLogger} from "./BaseLogger";
import type {ISerializedItem, ILogItem, LogItemValues, LabelOrValues, FilterCreator, LogCallback} from "./types";
export class LogItem { export class LogItem implements ILogItem {
constructor(labelOrValues, logLevel, filterCreator, logger) { public readonly start: number;
public logLevel: LogLevel;
public error?: Error;
public end?: number;
private _values: LogItemValues;
private _logger: BaseLogger;
private _filterCreator?: FilterCreator;
private _children?: Array<LogItem>;
constructor(labelOrValues: LabelOrValues, logLevel: LogLevel, logger: BaseLogger, filterCreator?: FilterCreator) {
this._logger = logger; this._logger = logger;
this._start = logger._now(); this.start = logger._now();
this._end = null;
// (l)abel // (l)abel
this._values = typeof labelOrValues === "string" ? {l: labelOrValues} : labelOrValues; this._values = typeof labelOrValues === "string" ? {l: labelOrValues} : labelOrValues;
this.error = null;
this.logLevel = logLevel; this.logLevel = logLevel;
this._children = null;
this._filterCreator = filterCreator; this._filterCreator = filterCreator;
} }
/** start a new root log item and run it detached mode, see BaseLogger.runDetached */ /** start a new root log item and run it detached mode, see BaseLogger.runDetached */
runDetached(labelOrValues, callback, logLevel, filterCreator) { runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem {
return this._logger.runDetached(labelOrValues, callback, logLevel, filterCreator); return this._logger.runDetached(labelOrValues, callback, logLevel, filterCreator);
} }
/** start a new detached root log item and log a reference to it from this item */ /** start a new detached root log item and log a reference to it from this item */
wrapDetached(labelOrValues, callback, logLevel, filterCreator) { wrapDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): void {
this.refDetached(this.runDetached(labelOrValues, callback, logLevel, filterCreator)); this.refDetached(this.runDetached(labelOrValues, callback, logLevel, filterCreator));
} }
/** logs a reference to a different log item, usually obtained from runDetached. /** logs a reference to a different log item, usually obtained from runDetached.
This is useful if the referenced operation can't be awaited. */ This is useful if the referenced operation can't be awaited. */
refDetached(logItem, logLevel = null) { refDetached(logItem: ILogItem, logLevel?: LogLevel): void {
logItem.ensureRefId(); logItem.ensureRefId();
return this.log({ref: logItem._values.refId}, logLevel); this.log({ref: logItem.values.refId}, logLevel);
} }
ensureRefId() { ensureRefId(): void {
if (!this._values.refId) { if (!this._values.refId) {
this.set("refId", this._logger._createRefId()); this.set("refId", this._logger._createRefId());
} }
@ -56,29 +64,33 @@ export class LogItem {
/** /**
* Creates a new child item and runs it in `callback`. * Creates a new child item and runs it in `callback`.
*/ */
wrap(labelOrValues, callback, logLevel = null, filterCreator = null) { wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T {
const item = this.child(labelOrValues, logLevel, filterCreator); const item = this.child(labelOrValues, logLevel, filterCreator);
return item.run(callback); return item.run(callback);
} }
get duration() { get duration(): number | undefined {
if (this._end) { if (this.end) {
return this._end - this._start; return this.end - this.start;
} else { } else {
return null; return undefined;
} }
} }
durationWithoutType(type) { durationWithoutType(type: string): number | undefined {
return this.duration - this.durationOfType(type); const durationOfType = this.durationOfType(type);
if (this.duration && durationOfType) {
return this.duration - durationOfType;
}
} }
durationOfType(type) { durationOfType(type: string): number | undefined {
if (this._values.t === type) { if (this._values.t === type) {
return this.duration; return this.duration;
} else if (this._children) { } else if (this._children) {
return this._children.reduce((sum, c) => { return this._children.reduce((sum, c) => {
return sum + c.durationOfType(type); const duration = c.durationOfType(type);
return sum + (duration ?? 0);
}, 0); }, 0);
} else { } else {
return 0; return 0;
@ -87,17 +99,16 @@ export class LogItem {
/** /**
* Creates a new child item that finishes immediately * Creates a new child item that finishes immediately
* and can hence not be modified anymore. * Finished items should not be modified anymore as they can be serialized
* * at any stage, but using `set` on the return value in a synchronous way should still be safe.
* Hence, the child item is not returned.
*/ */
log(labelOrValues, logLevel = null) { log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem {
const item = this.child(labelOrValues, logLevel, null); const item = this.child(labelOrValues, logLevel);
item._end = item._start; item.end = item.start;
return item; return item;
} }
set(key, value) { set(key: string | object, value?: unknown): void {
if(typeof key === "object") { if(typeof key === "object") {
const values = key; const values = key;
Object.assign(this._values, values); Object.assign(this._values, values);
@ -106,7 +117,7 @@ export class LogItem {
} }
} }
serialize(filter, parentStartTime = null, forced) { serialize(filter: LogFilter, parentStartTime: number | undefined, forced: boolean): ISerializedItem | undefined {
if (this._filterCreator) { if (this._filterCreator) {
try { try {
filter = this._filterCreator(new LogFilter(filter), this); filter = this._filterCreator(new LogFilter(filter), this);
@ -114,10 +125,10 @@ export class LogItem {
console.error("Error creating log filter", err); console.error("Error creating log filter", err);
} }
} }
let children; let children: Array<ISerializedItem> | null = null;
if (this._children !== null) { if (this._children) {
children = this._children.reduce((array, c) => { children = this._children.reduce((array: Array<ISerializedItem>, c) => {
const s = c.serialize(filter, this._start, false); const s = c.serialize(filter, this.start, false);
if (s) { if (s) {
if (array === null) { if (array === null) {
array = []; array = [];
@ -128,12 +139,12 @@ export class LogItem {
}, null); }, null);
} }
if (filter && !filter.filter(this, children)) { if (filter && !filter.filter(this, children)) {
return null; return;
} }
// in (v)alues, (l)abel and (t)ype are also reserved. // in (v)alues, (l)abel and (t)ype are also reserved.
const item = { const item: ISerializedItem = {
// (s)tart // (s)tart
s: parentStartTime === null ? this._start : this._start - parentStartTime, s: typeof parentStartTime === "number" ? this.start - parentStartTime : this.start,
// (d)uration // (d)uration
d: this.duration, d: this.duration,
// (v)alues // (v)alues
@ -172,20 +183,19 @@ export class LogItem {
* @param {Function} callback [description] * @param {Function} callback [description]
* @return {[type]} [description] * @return {[type]} [description]
*/ */
run(callback) { run<T>(callback: LogCallback<T>): T {
if (this._end !== null) { if (this.end !== undefined) {
console.trace("log item is finished, additional logs will likely not be recorded"); console.trace("log item is finished, additional logs will likely not be recorded");
} }
let result;
try { try {
result = callback(this); const result = callback(this);
if (result instanceof Promise) { if (result instanceof Promise) {
return result.then(promiseResult => { return result.then(promiseResult => {
this.finish(); this.finish();
return promiseResult; return promiseResult;
}, err => { }, err => {
throw this.catch(err); throw this.catch(err);
}); }) as unknown as T;
} else { } else {
this.finish(); this.finish();
return result; return result;
@ -199,45 +209,53 @@ export class LogItem {
* finished the item, recording the end time. After finishing, an item can't be modified anymore as it will be persisted. * finished the item, recording the end time. After finishing, an item can't be modified anymore as it will be persisted.
* @internal shouldn't typically be called by hand. allows to force finish if a promise is still running when closing the app * @internal shouldn't typically be called by hand. allows to force finish if a promise is still running when closing the app
*/ */
finish() { finish(): void {
if (this._end === null) { if (this.end === undefined) {
if (this._children !== null) { if (this._children) {
for(const c of this._children) { for(const c of this._children) {
c.finish(); c.finish();
} }
} }
this._end = this._logger._now(); this.end = this._logger._now();
} }
} }
// expose log level without needing import everywhere // expose log level without needing import everywhere
get level() { get level(): typeof LogLevel {
return LogLevel; return LogLevel;
} }
catch(err) { catch(err: Error): Error {
this.error = err; this.error = err;
this.logLevel = LogLevel.Error; this.logLevel = LogLevel.Error;
this.finish(); this.finish();
return err; return err;
} }
child(labelOrValues, logLevel, filterCreator) { child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): LogItem {
if (this._end !== null) { if (this.end) {
console.trace("log item is finished, additional logs will likely not be recorded"); console.trace("log item is finished, additional logs will likely not be recorded");
} }
if (!logLevel) { if (!logLevel) {
logLevel = this.logLevel || LogLevel.Info; logLevel = this.logLevel || LogLevel.Info;
} }
const item = new LogItem(labelOrValues, logLevel, filterCreator, this._logger); const item = new LogItem(labelOrValues, logLevel, this._logger, filterCreator);
if (this._children === null) { if (!this._children) {
this._children = []; this._children = [];
} }
this._children.push(item); this._children.push(item);
return item; return item;
} }
get logger() { get logger(): BaseLogger {
return this._logger; return this._logger;
} }
get values(): LogItemValues {
return this._values;
}
get children(): Array<LogItem> | undefined {
return this._children;
}
} }

View file

@ -1,99 +0,0 @@
/*
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {LogLevel} from "./LogFilter.js";
function noop () {}
export class NullLogger {
constructor() {
this.item = new NullLogItem(this);
}
log() {}
run(_, callback) {
return callback(this.item);
}
wrapOrRun(item, _, callback) {
if (item) {
return item.wrap(null, callback);
} else {
return this.run(null, callback);
}
}
runDetached(_, callback) {
new Promise(r => r(callback(this.item))).then(noop, noop);
}
async export() {
return null;
}
get level() {
return LogLevel;
}
}
export class NullLogItem {
constructor(logger) {
this.logger = logger;
}
wrap(_, callback) {
return callback(this);
}
log() {}
set() {}
runDetached(_, callback) {
new Promise(r => r(callback(this))).then(noop, noop);
}
wrapDetached(_, callback) {
return this.refDetached(null, callback);
}
run(callback) {
return callback(this);
}
refDetached() {}
ensureRefId() {}
get level() {
return LogLevel;
}
get duration() {
return 0;
}
catch(err) {
return err;
}
child() {
return this;
}
finish() {}
}
export const Instance = new NullLogger();

106
src/logging/NullLogger.ts Normal file
View file

@ -0,0 +1,106 @@
/*
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {LogLevel} from "./LogFilter";
import type {ILogger, ILogExport, ILogItem, LabelOrValues, LogCallback, LogItemValues} from "./types";
function noop (): void {}
export class NullLogger implements ILogger {
public readonly item: ILogItem = new NullLogItem(this);
log(): void {}
run<T>(_, callback: LogCallback<T>): T {
return callback(this.item);
}
wrapOrRun<T>(item: ILogItem | undefined, _, callback: LogCallback<T>): T {
if (item) {
return item.wrap(_, callback);
} else {
return this.run(_, callback);
}
}
runDetached(_, callback): ILogItem {
new Promise(r => r(callback(this.item))).then(noop, noop);
return this.item;
}
async export(): Promise<ILogExport | undefined> {
return undefined;
}
get level(): typeof LogLevel {
return LogLevel;
}
}
export class NullLogItem implements ILogItem {
public readonly logger: NullLogger;
public readonly logLevel: LogLevel;
public children?: Array<ILogItem>;
public values: LogItemValues;
public error?: Error;
constructor(logger: NullLogger) {
this.logger = logger;
}
wrap<T>(_: LabelOrValues, callback: LogCallback<T>): T {
return callback(this);
}
log(): void {}
set(): void {}
runDetached(_: LabelOrValues, callback: LogCallback<unknown>): ILogItem {
new Promise(r => r(callback(this))).then(noop, noop);
return this;
}
wrapDetached(_: LabelOrValues, _callback: LogCallback<unknown>): void {
return this.refDetached();
}
refDetached(): void {}
ensureRefId(): void {}
get level(): typeof LogLevel {
return LogLevel;
}
get duration(): 0 {
return 0;
}
catch(err: Error): Error {
return err;
}
child(): ILogItem {
return this;
}
finish(): void {}
serialize(): undefined {
return undefined;
}
}
export const Instance = new NullLogger();

82
src/logging/types.ts Normal file
View file

@ -0,0 +1,82 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {LogLevel, LogFilter} from "./LogFilter";
import type {BaseLogger} from "./BaseLogger";
import type {BlobHandle} from "../platform/web/dom/BlobHandle.js";
export interface ISerializedItem {
s: number;
d?: number;
v: LogItemValues;
l: LogLevel;
e?: {
stack?: string;
name: string;
message: string;
};
f?: boolean;
c?: Array<ISerializedItem>;
};
export interface ILogItem {
logLevel: LogLevel;
error?: Error;
readonly logger: ILogger;
readonly level: typeof LogLevel;
readonly end?: number;
readonly start?: number;
readonly values: LogItemValues;
wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): void;
set(key: string | object, value: unknown): void;
runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
wrapDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): void;
refDetached(logItem: ILogItem, logLevel?: LogLevel): void;
ensureRefId(): void;
catch(err: Error): Error;
serialize(filter: LogFilter, parentStartTime: number | undefined, forced: boolean): ISerializedItem | undefined;
}
export interface ILogger {
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): void;
wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
runDetached<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
run<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
export(): Promise<ILogExport | undefined>;
get level(): typeof LogLevel;
}
export interface ILogExport {
get count(): number;
removeFromStore(): Promise<void>;
asBlob(): BlobHandle;
}
export type LogItemValues = {
l?: string;
t?: string;
id?: unknown;
status?: string | number;
refId?: number;
ref?: number;
[key: string]: any
}
export type LabelOrValues = string | LogItemValues;
export type FilterCreator = ((filter: LogFilter, item: ILogItem) => LogFilter);
export type LogCallback<T> = (item: ILogItem) => T;

View file

@ -1,16 +0,0 @@
// these are helper functions if you can't assume you always have a log item (e.g. some code paths call with one set, others don't)
// if you know you always have a log item, better to use the methods on the log item than these utility functions.
import {Instance as NullLoggerInstance} from "./NullLogger.js";
export function wrapOrRunNullLogger(logItem, labelOrValues, callback, logLevel = null, filterCreator = null) {
if (logItem) {
return logItem.wrap(logItem, labelOrValues, callback, logLevel, filterCreator);
} else {
return NullLoggerInstance.run(null, callback);
}
}
export function ensureLogItem(logItem) {
return logItem || NullLoggerInstance.item;
}

18
src/logging/utils.ts Normal file
View file

@ -0,0 +1,18 @@
// these are helper functions if you can't assume you always have a log item (e.g. some code paths call with one set, others don't)
// if you know you always have a log item, better to use the methods on the log item than these utility functions.
import {Instance as NullLoggerInstance} from "./NullLogger";
import type {FilterCreator, ILogItem, LabelOrValues, LogCallback} from "./types";
import {LogLevel} from "./LogFilter";
export function wrapOrRunNullLogger<T>(logItem: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T | Promise<T> {
if (logItem) {
return logItem.wrap(labelOrValues, callback, logLevel, filterCreator);
} else {
return NullLoggerInstance.run(null, callback);
}
}
export function ensureLogItem(logItem: ILogItem): ILogItem {
return logItem || NullLoggerInstance.item;
}

View file

@ -19,7 +19,7 @@ import {Room} from "./room/Room.js";
import {ArchivedRoom} from "./room/ArchivedRoom.js"; import {ArchivedRoom} from "./room/ArchivedRoom.js";
import {RoomStatus} from "./room/RoomStatus.js"; import {RoomStatus} from "./room/RoomStatus.js";
import {Invite} from "./room/Invite.js"; import {Invite} from "./room/Invite.js";
import {Pusher} from "./push/Pusher.js"; import {Pusher} from "./push/Pusher";
import { ObservableMap } from "../observable/index.js"; import { ObservableMap } from "../observable/index.js";
import {User} from "./User.js"; import {User} from "./User.js";
import {DeviceMessageHandler} from "./DeviceMessageHandler.js"; import {DeviceMessageHandler} from "./DeviceMessageHandler.js";
@ -34,7 +34,7 @@ import {Encryption as MegOlmEncryption} from "./e2ee/megolm/Encryption.js";
import {MEGOLM_ALGORITHM} from "./e2ee/common.js"; import {MEGOLM_ALGORITHM} from "./e2ee/common.js";
import {RoomEncryption} from "./e2ee/RoomEncryption.js"; import {RoomEncryption} from "./e2ee/RoomEncryption.js";
import {DeviceTracker} from "./e2ee/DeviceTracker.js"; import {DeviceTracker} from "./e2ee/DeviceTracker.js";
import {LockMap} from "../utils/LockMap.js"; import {LockMap} from "../utils/LockMap";
import {groupBy} from "../utils/groupBy"; import {groupBy} from "../utils/groupBy";
import { import {
keyFromCredential as ssssKeyFromCredential, keyFromCredential as ssssKeyFromCredential,

View file

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {createEnum} from "../utils/enum.js"; import {createEnum} from "../utils/enum";
import {lookupHomeserver} from "./well-known.js"; import {lookupHomeserver} from "./well-known.js";
import {AbortableOperation} from "../utils/AbortableOperation"; import {AbortableOperation} from "../utils/AbortableOperation";
import {ObservableValue} from "../observable/ObservableValue"; import {ObservableValue} from "../observable/ObservableValue";
@ -26,9 +26,9 @@ import {MediaRepository} from "./net/MediaRepository.js";
import {RequestScheduler} from "./net/RequestScheduler.js"; import {RequestScheduler} from "./net/RequestScheduler.js";
import {Sync, SyncStatus} from "./Sync.js"; import {Sync, SyncStatus} from "./Sync.js";
import {Session} from "./Session.js"; import {Session} from "./Session.js";
import {PasswordLoginMethod} from "./login/PasswordLoginMethod.js"; import {PasswordLoginMethod} from "./login/PasswordLoginMethod";
import {TokenLoginMethod} from "./login/TokenLoginMethod.js"; import {TokenLoginMethod} from "./login/TokenLoginMethod";
import {SSOLoginHelper} from "./login/SSOLoginHelper.js"; import {SSOLoginHelper} from "./login/SSOLoginHelper";
import {getDehydratedDevice} from "./e2ee/Dehydration.js"; import {getDehydratedDevice} from "./e2ee/Dehydration.js";
export const LoadStatus = createEnum( export const LoadStatus = createEnum(
@ -329,7 +329,16 @@ export class SessionContainer {
request: this._platform.request, request: this._platform.request,
}); });
const olm = await this._olmPromise; const olm = await this._olmPromise;
const encryptedDehydratedDevice = await getDehydratedDevice(hsApi, olm, this._platform, log); let encryptedDehydratedDevice;
try {
encryptedDehydratedDevice = await getDehydratedDevice(hsApi, olm, this._platform, log);
} catch (err) {
if (err.name === "HomeServerError") {
log.set("not_supported", true);
} else {
throw err;
}
}
if (encryptedDehydratedDevice) { if (encryptedDehydratedDevice) {
let resolveStageFinish; let resolveStageFinish;
const promiseStageFinish = new Promise(r => resolveStageFinish = r); const promiseStageFinish = new Promise(r => resolveStageFinish = r);

View file

@ -16,7 +16,7 @@ limitations under the License.
*/ */
import {ObservableValue} from "../observable/ObservableValue"; import {ObservableValue} from "../observable/ObservableValue";
import {createEnum} from "../utils/enum.js"; import {createEnum} from "../utils/enum";
const INCREMENTAL_TIMEOUT = 30000; const INCREMENTAL_TIMEOUT = 30000;

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import anotherjson from "../../../lib/another-json/index.js"; import anotherjson from "another-json";
import {SESSION_E2EE_KEY_PREFIX, OLM_ALGORITHM, MEGOLM_ALGORITHM} from "./common.js"; import {SESSION_E2EE_KEY_PREFIX, OLM_ALGORITHM, MEGOLM_ALGORITHM} from "./common.js";
// use common prefix so it's easy to clear properties that are not e2ee related during session clear // use common prefix so it's easy to clear properties that are not e2ee related during session clear

View file

@ -16,7 +16,7 @@ limitations under the License.
import {MEGOLM_ALGORITHM, DecryptionSource} from "./common.js"; import {MEGOLM_ALGORITHM, DecryptionSource} from "./common.js";
import {groupEventsBySession} from "./megolm/decryption/utils"; import {groupEventsBySession} from "./megolm/decryption/utils";
import {mergeMap} from "../../utils/mergeMap.js"; import {mergeMap} from "../../utils/mergeMap";
import {groupBy} from "../../utils/groupBy"; import {groupBy} from "../../utils/groupBy";
import {makeTxnId} from "../common.js"; import {makeTxnId} from "../common.js";

View file

@ -14,8 +14,8 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import anotherjson from "../../../lib/another-json/index.js"; import anotherjson from "another-json";
import {createEnum} from "../../utils/enum.js"; import {createEnum} from "../../utils/enum";
export const DecryptionSource = createEnum("Sync", "Timeline", "Retry"); export const DecryptionSource = createEnum("Sync", "Timeline", "Retry");

View file

@ -26,7 +26,7 @@ import type {OlmWorker} from "../OlmWorker";
import type {Transaction} from "../../storage/idb/Transaction"; import type {Transaction} from "../../storage/idb/Transaction";
import type {TimelineEvent} from "../../storage/types"; import type {TimelineEvent} from "../../storage/types";
import type {DecryptionResult} from "../DecryptionResult"; import type {DecryptionResult} from "../DecryptionResult";
import type {LogItem} from "../../../logging/LogItem"; import type {ILogItem} from "../../../logging/types";
export class Decryption { export class Decryption {
private keyLoader: KeyLoader; private keyLoader: KeyLoader;
@ -136,7 +136,7 @@ export class Decryption {
* Extracts room keys from decrypted device messages. * Extracts room keys from decrypted device messages.
* The key won't be persisted yet, you need to call RoomKey.write for that. * The key won't be persisted yet, you need to call RoomKey.write for that.
*/ */
roomKeysFromDeviceMessages(decryptionResults: DecryptionResult[], log: LogItem): IncomingRoomKey[] { roomKeysFromDeviceMessages(decryptionResults: DecryptionResult[], log: ILogItem): IncomingRoomKey[] {
const keys: IncomingRoomKey[] = []; const keys: IncomingRoomKey[] = [];
for (const dr of decryptionResults) { for (const dr of decryptionResults) {
if (dr.event?.type !== "m.room_key" || dr.event.content?.algorithm !== MEGOLM_ALGORITHM) { if (dr.event?.type !== "m.room_key" || dr.event.content?.algorithm !== MEGOLM_ALGORITHM) {

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {DecryptionChanges} from "./DecryptionChanges.js"; import {DecryptionChanges} from "./DecryptionChanges.js";
import {mergeMap} from "../../../../utils/mergeMap.js"; import {mergeMap} from "../../../../utils/mergeMap";
/** /**
* Class that contains all the state loaded from storage to decrypt the given events * Class that contains all the state loaded from storage to decrypt the given events

View file

@ -16,7 +16,7 @@ limitations under the License.
import {DecryptionError} from "../common.js"; import {DecryptionError} from "../common.js";
import {groupBy} from "../../../utils/groupBy"; import {groupBy} from "../../../utils/groupBy";
import {MultiLock} from "../../../utils/Lock.js"; import {MultiLock} from "../../../utils/Lock";
import {Session} from "./Session.js"; import {Session} from "./Session.js";
import {DecryptionResult} from "../DecryptionResult.js"; import {DecryptionResult} from "../DecryptionResult.js";

View file

@ -38,7 +38,7 @@ export class HomeServerError extends Error {
} }
} }
export {AbortError} from "../utils/error.js"; export {AbortError} from "../utils/error";
export class ConnectionError extends Error { export class ConnectionError extends Error {
constructor(message, isTimeout) { constructor(message, isTimeout) {

View file

@ -14,17 +14,10 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
export class LoginMethod { import type {ILogItem} from "../../logging/types";
constructor({homeserver}) { import type {HomeServerApi} from "../net/HomeServerApi.js";
this.homeserver = homeserver;
}
// eslint-disable-next-line no-unused-vars export interface ILoginMethod {
async login(hsApi, deviceName, log) { homeserver: string;
/* login(hsApi: HomeServerApi, deviceName: string, log: ILogItem): Promise<Record<string, any>>;
Regardless of the login method, SessionContainer.startWithLogin()
can do SomeLoginMethod.login()
*/
throw("Not Implemented");
}
} }

View file

@ -1,29 +0,0 @@
/*
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {LoginMethod} from "./LoginMethod.js";
export class PasswordLoginMethod extends LoginMethod {
constructor(options) {
super(options);
this.username = options.username;
this.password = options.password;
}
async login(hsApi, deviceName, log) {
return await hsApi.passwordLogin(this.username, this.password, deviceName, {log}).response();
}
}

View file

@ -0,0 +1,35 @@
/*
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ILogItem} from "../../logging/types";
import {ILoginMethod} from "./LoginMethod";
import {HomeServerApi} from "../net/HomeServerApi.js";
export class PasswordLoginMethod implements ILoginMethod {
private readonly _username: string;
private readonly _password: string;
public readonly homeserver: string;
constructor({username, password, homeserver}: {username: string, password: string, homeserver: string}) {
this._username = username;
this._password = password;
this.homeserver = homeserver;
}
async login(hsApi: HomeServerApi, deviceName: string, log: ILogItem): Promise<Record<string, any>> {
return await hsApi.passwordLogin(this._username, this._password, deviceName, {log}).response();
}
}

View file

@ -15,13 +15,15 @@ limitations under the License.
*/ */
export class SSOLoginHelper{ export class SSOLoginHelper{
constructor(homeserver) { private _homeserver: string;
constructor(homeserver: string) {
this._homeserver = homeserver; this._homeserver = homeserver;
} }
get homeserver() { return this._homeserver; } get homeserver(): string { return this._homeserver; }
createSSORedirectURL(returnURL) { createSSORedirectURL(returnURL: string): string {
return `${this._homeserver}/_matrix/client/r0/login/sso/redirect?redirectUrl=${returnURL}`; return `${this._homeserver}/_matrix/client/r0/login/sso/redirect?redirectUrl=${returnURL}`;
} }
} }

View file

@ -14,16 +14,21 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {LoginMethod} from "./LoginMethod.js";
import {makeTxnId} from "../common.js"; import {makeTxnId} from "../common.js";
import {ILogItem} from "../../logging/types";
import {ILoginMethod} from "./LoginMethod";
import {HomeServerApi} from "../net/HomeServerApi.js";
export class TokenLoginMethod extends LoginMethod { export class TokenLoginMethod implements ILoginMethod {
constructor(options) { private readonly _loginToken: string;
super(options); public readonly homeserver: string;
this._loginToken = options.loginToken;
constructor({ homeserver, loginToken }: { homeserver: string, loginToken: string}) {
this.homeserver = homeserver;
this._loginToken = loginToken;
} }
async login(hsApi, deviceName, log) { async login(hsApi: HomeServerApi, deviceName: string, log: ILogItem): Promise<Record<string, any>> {
return await hsApi.tokenLogin(this._loginToken, makeTxnId(), deviceName, {log}).response(); return await hsApi.tokenLogin(this._loginToken, makeTxnId(), deviceName, {log}).response();
} }
} }

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {AbortError} from "../../utils/error.js"; import {AbortError} from "../../utils/error";
export class ExponentialRetryDelay { export class ExponentialRetryDelay {
constructor(createTimeout) { constructor(createTimeout) {

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {createEnum} from "../../utils/enum.js"; import {createEnum} from "../../utils/enum";
import {ObservableValue} from "../../observable/ObservableValue"; import {ObservableValue} from "../../observable/ObservableValue";
export const ConnectionStatus = createEnum( export const ConnectionStatus = createEnum(

View file

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {AbortError} from "../../utils/error.js"; import {AbortError} from "../../utils/error";
import {HomeServerError} from "../error.js"; import {HomeServerError} from "../error.js";
import {HomeServerApi} from "./HomeServerApi.js"; import {HomeServerApi} from "./HomeServerApi.js";
import {ExponentialRetryDelay} from "./ExponentialRetryDelay.js"; import {ExponentialRetryDelay} from "./ExponentialRetryDelay.js";

View file

@ -1,50 +0,0 @@
export class Pusher {
constructor(description) {
this._description = description;
}
static httpPusher(host, appId, pushkey, data) {
return new Pusher({
kind: "http",
append: true, // as pushkeys are shared between multiple users on one origin
data: Object.assign({}, data, {url: host + "/_matrix/push/v1/notify"}),
pushkey,
app_id: appId,
app_display_name: "Hydrogen",
device_display_name: "Hydrogen",
lang: "en"
});
}
static createDefaultPayload(sessionId) {
return {session_id: sessionId};
}
async enable(hsApi, log) {
try {
log.set("endpoint", new URL(this._description.data.endpoint).host);
} catch {
log.set("endpoint", null);
}
await hsApi.setPusher(this._description, {log}).response();
}
async disable(hsApi, log) {
const deleteDescription = Object.assign({}, this._description, {kind: null});
await hsApi.setPusher(deleteDescription, {log}).response();
}
serialize() {
return this._description;
}
equals(pusher) {
if (this._description.app_id !== pusher._description.app_id) {
return false;
}
if (this._description.pushkey !== pusher._description.pushkey) {
return false;
}
return JSON.stringify(this._description.data) === JSON.stringify(pusher._description.data);
}
}

90
src/matrix/push/Pusher.ts Normal file
View file

@ -0,0 +1,90 @@
/*
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {HomeServerApi} from "../net/HomeServerApi.js";
import type {ILogItem} from "../../logging/types";
export interface IPusherDescription {
kind: "http" | "email" | "null";
lang: string;
device_display_name: string;
app_display_name: string;
app_id: string;
pushkey: string;
data: IPusherData;
append?: boolean;
profile_tag?: string;
}
interface IPusherData {
format?: string;
url?: string;
endpoint?: PushSubscriptionJSON["endpoint"];
keys?: PushSubscriptionJSON["keys"];
}
export class Pusher {
private readonly _description: IPusherDescription;
constructor(description: IPusherDescription) {
this._description = description;
}
static httpPusher(host: string, appId: string, pushkey: string, data: IPusherData): Pusher {
return new Pusher({
kind: "http",
append: true, // as pushkeys are shared between multiple users on one origin
data: Object.assign({}, data, {url: host + "/_matrix/push/v1/notify"}),
pushkey,
app_id: appId,
app_display_name: "Hydrogen",
device_display_name: "Hydrogen",
lang: "en"
});
}
static createDefaultPayload(sessionId: string): {session_id: string} {
return {session_id: sessionId};
}
async enable(hsApi: HomeServerApi, log: ILogItem): Promise<void> {
try {
log.set("endpoint", new URL(this._description.data.endpoint!).host);
} catch {
log.set("endpoint", null);
}
await hsApi.setPusher(this._description, {log}).response();
}
async disable(hsApi: HomeServerApi, log: ILogItem): Promise<void> {
const deleteDescription = Object.assign({}, this._description, {kind: null});
await hsApi.setPusher(deleteDescription, {log}).response();
}
serialize(): IPusherDescription {
return this._description;
}
equals(pusher): boolean {
if (this._description.app_id !== pusher._description.app_id) {
return false;
}
if (this._description.pushkey !== pusher._description.pushkey) {
return false;
}
return JSON.stringify(this._description.data) === JSON.stringify(pusher._description.data);
}
}

View file

@ -27,7 +27,7 @@ import {Heroes} from "./members/Heroes.js";
import {EventEntry} from "./timeline/entries/EventEntry.js"; import {EventEntry} from "./timeline/entries/EventEntry.js";
import {ObservedEventMap} from "./ObservedEventMap.js"; import {ObservedEventMap} from "./ObservedEventMap.js";
import {DecryptionSource} from "../e2ee/common.js"; import {DecryptionSource} from "../e2ee/common.js";
import {ensureLogItem} from "../../logging/utils.js"; import {ensureLogItem} from "../../logging/utils";
import {PowerLevels} from "./PowerLevels.js"; import {PowerLevels} from "./PowerLevels.js";
import {RetainedObservableValue} from "../../observable/ObservableValue"; import {RetainedObservableValue} from "../../observable/ObservableValue";

View file

@ -244,7 +244,7 @@ export class Invite extends EventEmitter {
} }
} }
import {NullLogItem} from "../../logging/NullLogger.js"; import {NullLogItem} from "../../logging/NullLogger";
import {Clock as MockClock} from "../../mocks/Clock.js"; import {Clock as MockClock} from "../../mocks/Clock.js";
import {default as roomInviteFixture} from "../../fixtures/matrix/invites/room.js"; import {default as roomInviteFixture} from "../../fixtures/matrix/invites/room.js";
import {default as dmInviteFixture} from "../../fixtures/matrix/invites/dm.js"; import {default as dmInviteFixture} from "../../fixtures/matrix/invites/dm.js";

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {ObservableMap} from "../../../observable/map/ObservableMap.js"; import {ObservableMap} from "../../../observable/map/ObservableMap.js";
import {RetainedValue} from "../../../utils/RetainedValue.js"; import {RetainedValue} from "../../../utils/RetainedValue";
export class MemberList extends RetainedValue { export class MemberList extends RetainedValue {
constructor({members, closeCallback}) { constructor({members, closeCallback}) {

View file

@ -13,8 +13,8 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {createEnum} from "../../../utils/enum.js"; import {createEnum} from "../../../utils/enum";
import {AbortError} from "../../../utils/error.js"; import {AbortError} from "../../../utils/error";
import {REDACTION_TYPE} from "../common.js"; import {REDACTION_TYPE} from "../common.js";
import {getRelationFromContent, getRelationTarget, setRelationTarget} from "../timeline/relations.js"; import {getRelationFromContent, getRelationTarget, setRelationTarget} from "../timeline/relations.js";

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {SortedArray} from "../../../observable/list/SortedArray.js"; import {SortedArray} from "../../../observable/list/SortedArray";
import {ConnectionError} from "../../error.js"; import {ConnectionError} from "../../error.js";
import {PendingEvent, SendStatus} from "./PendingEvent.js"; import {PendingEvent, SendStatus} from "./PendingEvent.js";
import {makeTxnId, isTxnId} from "../../common.js"; import {makeTxnId, isTxnId} from "../../common.js";
@ -353,7 +353,7 @@ export class SendQueue {
import {HomeServer as MockHomeServer} from "../../../mocks/HomeServer.js"; import {HomeServer as MockHomeServer} from "../../../mocks/HomeServer.js";
import {createMockStorage} from "../../../mocks/Storage"; import {createMockStorage} from "../../../mocks/Storage";
import {ListObserver} from "../../../mocks/ListObserver.js"; import {ListObserver} from "../../../mocks/ListObserver.js";
import {NullLogger, NullLogItem} from "../../../logging/NullLogger.js"; import {NullLogger, NullLogItem} from "../../../logging/NullLogger";
import {createEvent, withTextBody, withTxnId} from "../../../mocks/event.js"; import {createEvent, withTextBody, withTxnId} from "../../../mocks/event.js";
import {poll} from "../../../mocks/poll.js"; import {poll} from "../../../mocks/poll.js";
import {createAnnotation} from "../timeline/relations.js"; import {createAnnotation} from "../timeline/relations.js";

View file

@ -16,7 +16,7 @@ limitations under the License.
*/ */
import {SortedArray, AsyncMappedList, ConcatList, ObservableArray} from "../../../observable/index.js"; import {SortedArray, AsyncMappedList, ConcatList, ObservableArray} from "../../../observable/index.js";
import {Disposables} from "../../../utils/Disposables.js"; import {Disposables} from "../../../utils/Disposables";
import {Direction} from "./Direction"; import {Direction} from "./Direction";
import {TimelineReader} from "./persistence/TimelineReader.js"; import {TimelineReader} from "./persistence/TimelineReader.js";
import {PendingEventEntry} from "./entries/PendingEventEntry.js"; import {PendingEventEntry} from "./entries/PendingEventEntry.js";
@ -346,7 +346,7 @@ import {Clock as MockClock} from "../../../mocks/Clock.js";
import {createMockStorage} from "../../../mocks/Storage"; import {createMockStorage} from "../../../mocks/Storage";
import {ListObserver} from "../../../mocks/ListObserver.js"; import {ListObserver} from "../../../mocks/ListObserver.js";
import {createEvent, withTextBody, withContent, withSender} from "../../../mocks/event.js"; import {createEvent, withTextBody, withContent, withSender} from "../../../mocks/event.js";
import {NullLogItem} from "../../../logging/NullLogger.js"; import {NullLogItem} from "../../../logging/NullLogger";
import {EventEntry} from "./entries/EventEntry.js"; import {EventEntry} from "./entries/EventEntry.js";
import {User} from "../../User.js"; import {User} from "../../User.js";
import {PendingEvent} from "../sending/PendingEvent.js"; import {PendingEvent} from "../sending/PendingEvent.js";

View file

@ -205,7 +205,7 @@ import {FragmentIdComparer} from "../FragmentIdComparer.js";
import {RelationWriter} from "./RelationWriter.js"; import {RelationWriter} from "./RelationWriter.js";
import {createMockStorage} from "../../../../mocks/Storage"; import {createMockStorage} from "../../../../mocks/Storage";
import {FragmentBoundaryEntry} from "../entries/FragmentBoundaryEntry.js"; import {FragmentBoundaryEntry} from "../entries/FragmentBoundaryEntry.js";
import {NullLogItem} from "../../../../logging/NullLogger.js"; import {NullLogItem} from "../../../../logging/NullLogger";
import {TimelineMock, eventIds, eventId} from "../../../../mocks/TimelineMock.ts"; import {TimelineMock, eventIds, eventId} from "../../../../mocks/TimelineMock.ts";
import {SyncWriter} from "./SyncWriter.js"; import {SyncWriter} from "./SyncWriter.js";
import {MemberWriter} from "./MemberWriter.js"; import {MemberWriter} from "./MemberWriter.js";

View file

@ -257,7 +257,7 @@ import {createMockStorage} from "../../../../mocks/Storage";
import {createEvent, withTextBody, withRedacts, withContent} from "../../../../mocks/event.js"; import {createEvent, withTextBody, withRedacts, withContent} from "../../../../mocks/event.js";
import {createAnnotation} from "../relations.js"; import {createAnnotation} from "../relations.js";
import {FragmentIdComparer} from "../FragmentIdComparer.js"; import {FragmentIdComparer} from "../FragmentIdComparer.js";
import {NullLogItem} from "../../../../logging/NullLogger.js"; import {NullLogItem} from "../../../../logging/NullLogger";
export function tests() { export function tests() {
const fragmentIdComparer = new FragmentIdComparer([]); const fragmentIdComparer = new FragmentIdComparer([]);

View file

@ -258,7 +258,7 @@ export class SyncWriter {
import {createMockStorage} from "../../../../mocks/Storage"; import {createMockStorage} from "../../../../mocks/Storage";
import {createEvent, withTextBody} from "../../../../mocks/event.js"; import {createEvent, withTextBody} from "../../../../mocks/event.js";
import {Instance as nullLogger} from "../../../../logging/NullLogger.js"; import {Instance as nullLogger} from "../../../../logging/NullLogger";
export function tests() { export function tests() {
const roomId = "!abc:hs.tld"; const roomId = "!abc:hs.tld";
return { return {

View file

@ -14,12 +14,33 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
export class SessionInfoStorage { interface ISessionInfo {
constructor(name) { id: string;
deviceId: string;
userId: string;
homeserver: string;
homeServer: string; // deprecate this over time
accessToken: string;
lastUsed: number;
}
// todo: this should probably be in platform/types?
interface ISessionInfoStorage {
getAll(): Promise<ISessionInfo[]>;
updateLastUsed(id: string, timestamp: number): Promise<void>;
get(id: string): Promise<ISessionInfo | undefined>;
add(sessionInfo: ISessionInfo): Promise<void>;
delete(sessionId: string): Promise<void>;
}
export class SessionInfoStorage implements ISessionInfoStorage {
private readonly _name: string;
constructor(name: string) {
this._name = name; this._name = name;
} }
getAll() { getAll(): Promise<ISessionInfo[]> {
const sessionsJson = localStorage.getItem(this._name); const sessionsJson = localStorage.getItem(this._name);
if (sessionsJson) { if (sessionsJson) {
const sessions = JSON.parse(sessionsJson); const sessions = JSON.parse(sessionsJson);
@ -30,7 +51,7 @@ export class SessionInfoStorage {
return Promise.resolve([]); return Promise.resolve([]);
} }
async updateLastUsed(id, timestamp) { async updateLastUsed(id: string, timestamp: number): Promise<void> {
const sessions = await this.getAll(); const sessions = await this.getAll();
if (sessions) { if (sessions) {
const session = sessions.find(session => session.id === id); const session = sessions.find(session => session.id === id);
@ -41,20 +62,20 @@ export class SessionInfoStorage {
} }
} }
async get(id) { async get(id: string): Promise<ISessionInfo | undefined> {
const sessions = await this.getAll(); const sessions = await this.getAll();
if (sessions) { if (sessions) {
return sessions.find(session => session.id === id); return sessions.find(session => session.id === id);
} }
} }
async add(sessionInfo) { async add(sessionInfo: ISessionInfo): Promise<void> {
const sessions = await this.getAll(); const sessions = await this.getAll();
sessions.push(sessionInfo); sessions.push(sessionInfo);
localStorage.setItem(this._name, JSON.stringify(sessions)); localStorage.setItem(this._name, JSON.stringify(sessions));
} }
async delete(sessionId) { async delete(sessionId: string): Promise<void> {
let sessions = await this.getAll(); let sessions = await this.getAll();
sessions = sessions.filter(s => s.id !== sessionId); sessions = sessions.filter(s => s.id !== sessionId);
localStorage.setItem(this._name, JSON.stringify(sessions)); localStorage.setItem(this._name, JSON.stringify(sessions));

View file

@ -18,7 +18,7 @@ import {KeyDescription, Key} from "./common.js";
import {keyFromPassphrase} from "./passphrase.js"; import {keyFromPassphrase} from "./passphrase.js";
import {keyFromRecoveryKey} from "./recoveryKey.js"; import {keyFromRecoveryKey} from "./recoveryKey.js";
import {SESSION_E2EE_KEY_PREFIX} from "../e2ee/common.js"; import {SESSION_E2EE_KEY_PREFIX} from "../e2ee/common.js";
import {createEnum} from "../../utils/enum.js"; import {createEnum} from "../../utils/enum";
const SSSS_KEY = `${SESSION_E2EE_KEY_PREFIX}ssssKey`; const SSSS_KEY = `${SESSION_E2EE_KEY_PREFIX}ssssKey`;

View file

@ -16,7 +16,7 @@ limitations under the License.
import {iterateCursor, DONE, NOT_DONE, reqAsPromise} from "./utils"; import {iterateCursor, DONE, NOT_DONE, reqAsPromise} from "./utils";
import {StorageError} from "../common"; import {StorageError} from "../common";
import {LogItem} from "../../../logging/LogItem.js"; import {ILogItem} from "../../../logging/types";
import {IDBKey} from "./Transaction"; import {IDBKey} from "./Transaction";
// this is the part of the Transaction class API that is used here and in the Store subclass, // this is the part of the Transaction class API that is used here and in the Store subclass,
@ -25,7 +25,7 @@ export interface ITransaction {
idbFactory: IDBFactory; idbFactory: IDBFactory;
IDBKeyRange: typeof IDBKeyRange; IDBKeyRange: typeof IDBKeyRange;
databaseName: string; databaseName: string;
addWriteError(error: StorageError, refItem: LogItem | undefined, operationName: string, keys: IDBKey[] | undefined); addWriteError(error: StorageError, refItem: ILogItem | undefined, operationName: string, keys: IDBKey[] | undefined);
} }
type Reducer<A,B> = (acc: B, val: A) => B type Reducer<A,B> = (acc: B, val: A) => B
@ -269,15 +269,20 @@ export class QueryTarget<T> {
} }
} }
import {createMockDatabase, MockIDBImpl} from "../../../mocks/Storage"; import {createMockDatabase, createMockIDBFactory, getMockIDBKeyRange} from "../../../mocks/Storage";
import {txnAsPromise} from "./utils"; import {txnAsPromise} from "./utils";
import {QueryTargetWrapper, Store} from "./Store"; import {QueryTargetWrapper, Store} from "./Store";
export function tests() { export async function tests() {
class MockTransaction extends MockIDBImpl { class MockTransaction {
constructor(public readonly idbFactory: IDBFactory, readonly idbKeyRangeType: typeof IDBKeyRange) {}
get IDBKeyRange(): typeof IDBKeyRange {
return this.idbKeyRangeType;
}
get databaseName(): string { return "mockdb"; } get databaseName(): string { return "mockdb"; }
addWriteError(error: StorageError, refItem: LogItem | undefined, operationName: string, keys: IDBKey[] | undefined) {} addWriteError(error: StorageError, refItem: ILogItem | undefined, operationName: string, keys: IDBKey[] | undefined) {}
} }
interface TestEntry { interface TestEntry {
@ -285,10 +290,12 @@ export function tests() {
} }
async function createTestStore(): Promise<Store<TestEntry>> { async function createTestStore(): Promise<Store<TestEntry>> {
const mockImpl = new MockTransaction(); const idbFactory = await createMockIDBFactory();
const idbKeyRangeType = await getMockIDBKeyRange();
const mockImpl = new MockTransaction(idbFactory, idbKeyRangeType);
const db = await createMockDatabase("findExistingKeys", (db: IDBDatabase) => { const db = await createMockDatabase("findExistingKeys", (db: IDBDatabase) => {
db.createObjectStore("test", {keyPath: "key"}); db.createObjectStore("test", {keyPath: "key"});
}, mockImpl); }, idbFactory);
const txn = db.transaction(["test"], "readwrite"); const txn = db.transaction(["test"], "readwrite");
return new Store<TestEntry>(txn.objectStore("test"), mockImpl); return new Store<TestEntry>(txn.objectStore("test"), mockImpl);
} }

View file

@ -18,7 +18,7 @@ import {IDOMStorage} from "./types";
import {Transaction} from "./Transaction"; import {Transaction} from "./Transaction";
import { STORE_NAMES, StoreNames, StorageError } from "../common"; import { STORE_NAMES, StoreNames, StorageError } from "../common";
import { reqAsPromise } from "./utils"; import { reqAsPromise } from "./utils";
import { BaseLogger } from "../../../logging/BaseLogger.js"; import { ILogger } from "../../../logging/types";
const WEBKITEARLYCLOSETXNBUG_BOGUS_KEY = "782rh281re38-boguskey"; const WEBKITEARLYCLOSETXNBUG_BOGUS_KEY = "782rh281re38-boguskey";
@ -26,13 +26,13 @@ export class Storage {
private _db: IDBDatabase; private _db: IDBDatabase;
private _hasWebkitEarlyCloseTxnBug: boolean; private _hasWebkitEarlyCloseTxnBug: boolean;
readonly logger: BaseLogger; readonly logger: ILogger;
readonly idbFactory: IDBFactory readonly idbFactory: IDBFactory
readonly IDBKeyRange: typeof IDBKeyRange; readonly IDBKeyRange: typeof IDBKeyRange;
readonly storeNames: typeof StoreNames; readonly storeNames: typeof StoreNames;
readonly localStorage: IDOMStorage; readonly localStorage: IDOMStorage;
constructor(idbDatabase: IDBDatabase, idbFactory: IDBFactory, _IDBKeyRange: typeof IDBKeyRange, hasWebkitEarlyCloseTxnBug: boolean, localStorage: IDOMStorage, logger: BaseLogger) { constructor(idbDatabase: IDBDatabase, idbFactory: IDBFactory, _IDBKeyRange: typeof IDBKeyRange, hasWebkitEarlyCloseTxnBug: boolean, localStorage: IDOMStorage, logger: ILogger) {
this._db = idbDatabase; this._db = idbDatabase;
this.idbFactory = idbFactory; this.idbFactory = idbFactory;
this.IDBKeyRange = _IDBKeyRange; this.IDBKeyRange = _IDBKeyRange;

View file

@ -20,11 +20,10 @@ import { openDatabase, reqAsPromise } from "./utils";
import { exportSession, importSession, Export } from "./export"; import { exportSession, importSession, Export } from "./export";
import { schema } from "./schema"; import { schema } from "./schema";
import { detectWebkitEarlyCloseTxnBug } from "./quirks"; import { detectWebkitEarlyCloseTxnBug } from "./quirks";
import { BaseLogger } from "../../../logging/BaseLogger.js"; import { ILogItem } from "../../../logging/types";
import { LogItem } from "../../../logging/LogItem.js";
const sessionName = (sessionId: string) => `hydrogen_session_${sessionId}`; const sessionName = (sessionId: string) => `hydrogen_session_${sessionId}`;
const openDatabaseWithSessionId = function(sessionId: string, idbFactory: IDBFactory, localStorage: IDOMStorage, log: LogItem) { const openDatabaseWithSessionId = function(sessionId: string, idbFactory: IDBFactory, localStorage: IDOMStorage, log: ILogItem) {
const create = (db, txn, oldVersion, version) => createStores(db, txn, oldVersion, version, localStorage, log); const create = (db, txn, oldVersion, version) => createStores(db, txn, oldVersion, version, localStorage, log);
return openDatabase(sessionName(sessionId), create, schema.length, idbFactory); return openDatabase(sessionName(sessionId), create, schema.length, idbFactory);
} }
@ -63,7 +62,7 @@ export class StorageFactory {
this._localStorage = localStorage; this._localStorage = localStorage;
} }
async create(sessionId: string, log: LogItem): Promise<Storage> { async create(sessionId: string, log: ILogItem): Promise<Storage> {
await this._serviceWorkerHandler?.preventConcurrentSessionAccess(sessionId); await this._serviceWorkerHandler?.preventConcurrentSessionAccess(sessionId);
requestPersistedStorage().then(persisted => { requestPersistedStorage().then(persisted => {
// Firefox lies here though, and returns true even if the user denied the request // Firefox lies here though, and returns true even if the user denied the request
@ -83,23 +82,25 @@ export class StorageFactory {
return reqAsPromise(req); return reqAsPromise(req);
} }
async export(sessionId: string, log: LogItem): Promise<Export> { async export(sessionId: string, log: ILogItem): Promise<Export> {
const db = await openDatabaseWithSessionId(sessionId, this._idbFactory, this._localStorage, log); const db = await openDatabaseWithSessionId(sessionId, this._idbFactory, this._localStorage, log);
return await exportSession(db); return await exportSession(db);
} }
async import(sessionId: string, data: Export, log: LogItem): Promise<void> { async import(sessionId: string, data: Export, log: ILogItem): Promise<void> {
const db = await openDatabaseWithSessionId(sessionId, this._idbFactory, this._localStorage, log); const db = await openDatabaseWithSessionId(sessionId, this._idbFactory, this._localStorage, log);
return await importSession(db, data); return await importSession(db, data);
} }
} }
async function createStores(db: IDBDatabase, txn: IDBTransaction, oldVersion: number | null, version: number, localStorage: IDOMStorage, log: LogItem): Promise<void> { async function createStores(db: IDBDatabase, txn: IDBTransaction, oldVersion: number | null, version: number, localStorage: IDOMStorage, log: ILogItem): Promise<void> {
const startIdx = oldVersion || 0; const startIdx = oldVersion || 0;
return log.wrap({l: "storage migration", oldVersion, version}, async log => { return log.wrap(
for(let i = startIdx; i < version; ++i) { { l: "storage migration", oldVersion, version },
const migrationFunc = schema[i]; async (log) => {
await log.wrap(`v${i + 1}`, log => migrationFunc(db, txn, localStorage, log)); for (let i = startIdx; i < version; ++i) {
} const migrationFunc = schema[i];
}); await log.wrap(`v${i + 1}`, (log) => migrationFunc(db, txn, localStorage, log));
}
});
} }

View file

@ -18,7 +18,7 @@ import {QueryTarget, IDBQuery, ITransaction} from "./QueryTarget";
import {IDBRequestError, IDBRequestAttemptError} from "./error"; import {IDBRequestError, IDBRequestAttemptError} from "./error";
import {reqAsPromise} from "./utils"; import {reqAsPromise} from "./utils";
import {Transaction, IDBKey} from "./Transaction"; import {Transaction, IDBKey} from "./Transaction";
import {LogItem} from "../../../logging/LogItem.js"; import {ILogItem} from "../../../logging/types";
const LOG_REQUESTS = false; const LOG_REQUESTS = false;
@ -145,7 +145,7 @@ export class Store<T> extends QueryTarget<T> {
return new QueryTarget<T>(new QueryTargetWrapper<T>(this._idbStore.index(indexName)), this._transaction); return new QueryTarget<T>(new QueryTargetWrapper<T>(this._idbStore.index(indexName)), this._transaction);
} }
put(value: T, log?: LogItem): void { put(value: T, log?: ILogItem): void {
// If this request fails, the error will bubble up to the transaction and abort it, // If this request fails, the error will bubble up to the transaction and abort it,
// which is the behaviour we want. Therefore, it is ok to not create a promise for this // which is the behaviour we want. Therefore, it is ok to not create a promise for this
// request and await it. // request and await it.
@ -160,13 +160,13 @@ export class Store<T> extends QueryTarget<T> {
this._prepareErrorLog(request, log, "put", undefined, value); this._prepareErrorLog(request, log, "put", undefined, value);
} }
add(value: T, log?: LogItem): void { add(value: T, log?: ILogItem): void {
// ok to not monitor result of request, see comment in `put`. // ok to not monitor result of request, see comment in `put`.
const request = this._idbStore.add(value); const request = this._idbStore.add(value);
this._prepareErrorLog(request, log, "add", undefined, value); this._prepareErrorLog(request, log, "add", undefined, value);
} }
async tryAdd(value: T, log: LogItem): Promise<boolean> { async tryAdd(value: T, log: ILogItem): Promise<boolean> {
try { try {
await reqAsPromise(this._idbStore.add(value)); await reqAsPromise(this._idbStore.add(value));
return true; return true;
@ -181,13 +181,13 @@ export class Store<T> extends QueryTarget<T> {
} }
} }
delete(keyOrKeyRange: IDBValidKey | IDBKeyRange, log?: LogItem): void { delete(keyOrKeyRange: IDBValidKey | IDBKeyRange, log?: ILogItem): void {
// ok to not monitor result of request, see comment in `put`. // ok to not monitor result of request, see comment in `put`.
const request = this._idbStore.delete(keyOrKeyRange); const request = this._idbStore.delete(keyOrKeyRange);
this._prepareErrorLog(request, log, "delete", keyOrKeyRange, undefined); this._prepareErrorLog(request, log, "delete", keyOrKeyRange, undefined);
} }
private _prepareErrorLog(request: IDBRequest, log: LogItem | undefined, operationName: string, key: IDBKey | undefined, value: T | undefined) { private _prepareErrorLog(request: IDBRequest, log: ILogItem | undefined, operationName: string, key: IDBKey | undefined, value: T | undefined) {
if (log) { if (log) {
log.ensureRefId(); log.ensureRefId();
} }

View file

@ -36,15 +36,14 @@ import {OutboundGroupSessionStore} from "./stores/OutboundGroupSessionStore";
import {GroupSessionDecryptionStore} from "./stores/GroupSessionDecryptionStore"; import {GroupSessionDecryptionStore} from "./stores/GroupSessionDecryptionStore";
import {OperationStore} from "./stores/OperationStore"; import {OperationStore} from "./stores/OperationStore";
import {AccountDataStore} from "./stores/AccountDataStore"; import {AccountDataStore} from "./stores/AccountDataStore";
import {LogItem} from "../../../logging/LogItem.js"; import type {ILogger, ILogItem} from "../../../logging/types";
import {BaseLogger} from "../../../logging/BaseLogger.js";
export type IDBKey = IDBValidKey | IDBKeyRange; export type IDBKey = IDBValidKey | IDBKeyRange;
class WriteErrorInfo { class WriteErrorInfo {
constructor( constructor(
public readonly error: StorageError, public readonly error: StorageError,
public readonly refItem: LogItem | undefined, public readonly refItem: ILogItem | undefined,
public readonly operationName: string, public readonly operationName: string,
public readonly keys: IDBKey[] | undefined, public readonly keys: IDBKey[] | undefined,
) {} ) {}
@ -77,7 +76,7 @@ export class Transaction {
return this._storage.databaseName; return this._storage.databaseName;
} }
get logger(): BaseLogger { get logger(): ILogger {
return this._storage.logger; return this._storage.logger;
} }
@ -169,7 +168,7 @@ export class Transaction {
return this._store(StoreNames.accountData, idbStore => new AccountDataStore(idbStore)); return this._store(StoreNames.accountData, idbStore => new AccountDataStore(idbStore));
} }
async complete(log?: LogItem): Promise<void> { async complete(log?: ILogItem): Promise<void> {
try { try {
await txnAsPromise(this._txn); await txnAsPromise(this._txn);
} catch (err) { } catch (err) {
@ -190,7 +189,7 @@ export class Transaction {
return error; return error;
} }
abort(log?: LogItem): void { abort(log?: ILogItem): void {
// TODO: should we wrap the exception in a StorageError? // TODO: should we wrap the exception in a StorageError?
try { try {
this._txn.abort(); this._txn.abort();
@ -202,14 +201,14 @@ export class Transaction {
} }
} }
addWriteError(error: StorageError, refItem: LogItem | undefined, operationName: string, keys: IDBKey[] | undefined) { addWriteError(error: StorageError, refItem: ILogItem | undefined, operationName: string, keys: IDBKey[] | undefined) {
// don't log subsequent `AbortError`s // don't log subsequent `AbortError`s
if (error.errcode !== "AbortError" || this._writeErrors.length === 0) { if (error.errcode !== "AbortError" || this._writeErrors.length === 0) {
this._writeErrors.push(new WriteErrorInfo(error, refItem, operationName, keys)); this._writeErrors.push(new WriteErrorInfo(error, refItem, operationName, keys));
} }
} }
private _logWriteErrors(parentItem: LogItem | undefined) { private _logWriteErrors(parentItem: ILogItem | undefined) {
const callback = errorGroupItem => { const callback = errorGroupItem => {
// we don't have context when there is no parentItem, so at least log stores // we don't have context when there is no parentItem, so at least log stores
if (!parentItem) { if (!parentItem) {

View file

@ -11,10 +11,10 @@ import {SessionStore} from "./stores/SessionStore";
import {Store} from "./Store"; import {Store} from "./Store";
import {encodeScopeTypeKey} from "./stores/OperationStore"; import {encodeScopeTypeKey} from "./stores/OperationStore";
import {MAX_UNICODE} from "./stores/common"; import {MAX_UNICODE} from "./stores/common";
import {LogItem} from "../../../logging/LogItem.js"; import {ILogItem} from "../../../logging/types";
export type MigrationFunc = (db: IDBDatabase, txn: IDBTransaction, localStorage: IDOMStorage, log: LogItem) => Promise<void> | void; export type MigrationFunc = (db: IDBDatabase, txn: IDBTransaction, localStorage: IDOMStorage, log: ILogItem) => Promise<void> | void;
// FUNCTIONS SHOULD ONLY BE APPENDED!! // FUNCTIONS SHOULD ONLY BE APPENDED!!
// the index in the array is the database version // the index in the array is the database version
export const schema: MigrationFunc[] = [ export const schema: MigrationFunc[] = [
@ -166,7 +166,7 @@ function createTimelineRelationsStore(db: IDBDatabase) : void {
} }
//v11 doesn't change the schema, but ensures all userIdentities have all the roomIds they should (see #470) //v11 doesn't change the schema, but ensures all userIdentities have all the roomIds they should (see #470)
async function fixMissingRoomsInUserIdentities(db: IDBDatabase, txn: IDBTransaction, localStorage: IDOMStorage, log: LogItem) { async function fixMissingRoomsInUserIdentities(db: IDBDatabase, txn: IDBTransaction, localStorage: IDOMStorage, log: ILogItem) {
const roomSummaryStore = txn.objectStore("roomSummary"); const roomSummaryStore = txn.objectStore("roomSummary");
const trackedRoomIds: string[] = []; const trackedRoomIds: string[] = [];
await iterateCursor<SummaryData>(roomSummaryStore.openCursor(), roomSummary => { await iterateCursor<SummaryData>(roomSummaryStore.openCursor(), roomSummary => {
@ -220,7 +220,7 @@ async function changeSSSSKeyPrefix(db: IDBDatabase, txn: IDBTransaction) {
} }
} }
// v13 // v13
async function backupAndRestoreE2EEAccountToLocalStorage(db: IDBDatabase, txn: IDBTransaction, localStorage: IDOMStorage, log: LogItem) { async function backupAndRestoreE2EEAccountToLocalStorage(db: IDBDatabase, txn: IDBTransaction, localStorage: IDOMStorage, log: ILogItem) {
const session = txn.objectStore("session"); const session = txn.objectStore("session");
// the Store object gets passed in several things through the Transaction class (a wrapper around IDBTransaction), // the Store object gets passed in several things through the Transaction class (a wrapper around IDBTransaction),
// the only thing we should need here is the databaseName though, so we mock it out. // the only thing we should need here is the databaseName though, so we mock it out.

View file

@ -16,8 +16,8 @@ limitations under the License.
import {Store} from "../Store"; import {Store} from "../Store";
import {IDOMStorage} from "../types"; import {IDOMStorage} from "../types";
import {SESSION_E2EE_KEY_PREFIX} from "../../../e2ee/common.js"; import {SESSION_E2EE_KEY_PREFIX} from "../../../e2ee/common.js";
import {LogItem} from "../../../../logging/LogItem.js";
import {parse, stringify} from "../../../../utils/typedJSON"; import {parse, stringify} from "../../../../utils/typedJSON";
import type {ILogItem} from "../../../../logging/types";
export interface SessionEntry { export interface SessionEntry {
key: string; key: string;
@ -64,7 +64,7 @@ export class SessionStore {
}); });
} }
async tryRestoreE2EEIdentityFromLocalStorage(log: LogItem): Promise<boolean> { async tryRestoreE2EEIdentityFromLocalStorage(log: ILogItem): Promise<boolean> {
let success = false; let success = false;
const lsPrefix = this._localStorageKeyPrefix; const lsPrefix = this._localStorageKeyPrefix;
const prefix = lsPrefix + SESSION_E2EE_KEY_PREFIX; const prefix = lsPrefix + SESSION_E2EE_KEY_PREFIX;

View file

@ -20,7 +20,7 @@ import { encodeUint32, decodeUint32 } from "../utils";
import {KeyLimits} from "../../common"; import {KeyLimits} from "../../common";
import {Store} from "../Store"; import {Store} from "../Store";
import {TimelineEvent, StateEvent} from "../../types"; import {TimelineEvent, StateEvent} from "../../types";
import {LogItem} from "../../../../logging/LogItem.js"; import {ILogItem} from "../../../../logging/types";
interface Annotation { interface Annotation {
count: number; count: number;
@ -286,7 +286,7 @@ export class TimelineEventStore {
* *
* Returns if the event was not yet known and the entry was written. * Returns if the event was not yet known and the entry was written.
*/ */
tryInsert(entry: TimelineEventEntry, log: LogItem): Promise<boolean> { tryInsert(entry: TimelineEventEntry, log: ILogItem): Promise<boolean> {
(entry as TimelineEventStorageEntry).key = encodeKey(entry.roomId, entry.fragmentId, entry.eventIndex); (entry as TimelineEventStorageEntry).key = encodeKey(entry.roomId, entry.fragmentId, entry.eventIndex);
(entry as TimelineEventStorageEntry).eventIdKey = encodeEventIdKey(entry.roomId, entry.event.event_id); (entry as TimelineEventStorageEntry).eventIdKey = encodeEventIdKey(entry.roomId, entry.event.event_id);
return this._timelineStore.tryAdd(entry as TimelineEventStorageEntry, log); return this._timelineStore.tryAdd(entry as TimelineEventStorageEntry, log);
@ -320,7 +320,7 @@ export class TimelineEventStore {
import {createMockStorage} from "../../../../mocks/Storage"; import {createMockStorage} from "../../../../mocks/Storage";
import {createEvent, withTextBody} from "../../../../mocks/event.js"; import {createEvent, withTextBody} from "../../../../mocks/event.js";
import {createEventEntry} from "../../../room/timeline/persistence/common.js"; import {createEventEntry} from "../../../room/timeline/persistence/common.js";
import {Instance as logItem} from "../../../../logging/NullLogger.js"; import {Instance as nullLogger} from "../../../../logging/NullLogger";
export function tests() { export function tests() {
@ -368,7 +368,7 @@ export function tests() {
let eventKey = EventKey.defaultFragmentKey(109); let eventKey = EventKey.defaultFragmentKey(109);
for (const insertedId of insertedIds) { for (const insertedId of insertedIds) {
const entry = createEventEntry(eventKey.nextKey(), roomId, createEventWithId(insertedId)); const entry = createEventEntry(eventKey.nextKey(), roomId, createEventWithId(insertedId));
assert(await txn.timelineEvents.tryInsert(entry, logItem)); assert(await txn.timelineEvents.tryInsert(entry, nullLogger.item));
eventKey = eventKey.nextKey(); eventKey = eventKey.nextKey();
} }
const eventKeyMap = await txn.timelineEvents.getEventKeysForIds(roomId, checkedIds); const eventKeyMap = await txn.timelineEvents.getEventKeysForIds(roomId, checkedIds);

View file

@ -17,7 +17,7 @@ limitations under the License.
import { IDBRequestError } from "./error"; import { IDBRequestError } from "./error";
import { StorageError } from "../common"; import { StorageError } from "../common";
import { AbortError } from "../../../utils/error.js"; import { AbortError } from "../../../utils/error";
let needsSyncPromise = false; let needsSyncPromise = false;

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {AbortError} from "../utils/error.js"; import {AbortError} from "../utils/error";
export class BaseRequest { export class BaseRequest {
constructor() { constructor() {

View file

@ -14,31 +14,35 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {FDBFactory, FDBKeyRange} from "../../lib/fake-indexeddb/index.js";
import {StorageFactory} from "../matrix/storage/idb/StorageFactory"; import {StorageFactory} from "../matrix/storage/idb/StorageFactory";
import {IDOMStorage} from "../matrix/storage/idb/types"; import {IDOMStorage} from "../matrix/storage/idb/types";
import {Storage} from "../matrix/storage/idb/Storage"; import {Storage} from "../matrix/storage/idb/Storage";
import {Instance as nullLogger} from "../logging/NullLogger.js"; import {Instance as nullLogger} from "../logging/NullLogger";
import {openDatabase, CreateObjectStore} from "../matrix/storage/idb/utils"; import {openDatabase, CreateObjectStore} from "../matrix/storage/idb/utils";
export function createMockStorage(): Promise<Storage> { export async function createMockStorage(): Promise<Storage> {
return new StorageFactory(null as any, new FDBFactory(), FDBKeyRange, new MockLocalStorage()).create("1", nullLogger.item); const idbFactory = await createMockIDBFactory();
const FDBKeyRange = await getMockIDBKeyRange();
return new StorageFactory(null as any, idbFactory, FDBKeyRange, new MockLocalStorage()).create("1", nullLogger.item);
} }
export function createMockDatabase(name: string, createObjectStore: CreateObjectStore, impl: MockIDBImpl): Promise<IDBDatabase> { // don't import fake-indexeddb until it's safe to assume we're actually in a unit test,
return openDatabase(name, createObjectStore, 1, impl.idbFactory); // as this is a devDependency
export async function createMockIDBFactory(): Promise<IDBFactory> {
// @ts-ignore
const FDBFactory = (await import("fake-indexeddb/lib/FDBFactory.js")).default;
return new FDBFactory();
} }
export class MockIDBImpl { // don't import fake-indexeddb until it's safe to assume we're actually in a unit test,
idbFactory: FDBFactory; // as this is a devDependency
export async function getMockIDBKeyRange(): Promise<typeof IDBKeyRange> {
// @ts-ignore
return (await import("fake-indexeddb/lib/FDBKeyRange.js")).default;
}
constructor() { export function createMockDatabase(name: string, createObjectStore: CreateObjectStore, idbFactory: IDBFactory): Promise<IDBDatabase> {
this.idbFactory = new FDBFactory(); return openDatabase(name, createObjectStore, 1, idbFactory);
}
get IDBKeyRange(): typeof IDBKeyRange {
return FDBKeyRange;
}
} }
class MockLocalStorage implements IDOMStorage { class MockLocalStorage implements IDOMStorage {

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {AbortError} from "../utils/error.js"; import {AbortError} from "../utils/error";
import {BaseObservable} from "./BaseObservable"; import {BaseObservable} from "./BaseObservable";
// like an EventEmitter, but doesn't have an event type // like an EventEmitter, but doesn't have an event type

View file

@ -20,11 +20,11 @@ import {MappedMap} from "./map/MappedMap.js";
import {JoinedMap} from "./map/JoinedMap.js"; import {JoinedMap} from "./map/JoinedMap.js";
import {BaseObservableMap} from "./map/BaseObservableMap.js"; import {BaseObservableMap} from "./map/BaseObservableMap.js";
// re-export "root" (of chain) collections // re-export "root" (of chain) collections
export { ObservableArray } from "./list/ObservableArray.js"; export { ObservableArray } from "./list/ObservableArray";
export { SortedArray } from "./list/SortedArray.js"; export { SortedArray } from "./list/SortedArray";
export { MappedList } from "./list/MappedList.js"; export { MappedList } from "./list/MappedList";
export { AsyncMappedList } from "./list/AsyncMappedList.js"; export { AsyncMappedList } from "./list/AsyncMappedList";
export { ConcatList } from "./list/ConcatList.js"; export { ConcatList } from "./list/ConcatList";
export { ObservableMap } from "./map/ObservableMap.js"; export { ObservableMap } from "./map/ObservableMap.js";
// avoid circular dependency between these classes // avoid circular dependency between these classes

View file

@ -15,15 +15,14 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseMappedList, runAdd, runUpdate, runRemove, runMove, runReset} from "./BaseMappedList"; import {IListObserver} from "./BaseObservableList";
import {BaseMappedList, Mapper, Updater, runAdd, runUpdate, runRemove, runMove, runReset} from "./BaseMappedList";
export class AsyncMappedList extends BaseMappedList { export class AsyncMappedList<F,T> extends BaseMappedList<F,T,Promise<T>> implements IListObserver<F> {
constructor(sourceList, mapper, updater, removeCallback) { private _eventQueue: AsyncEvent<F>[] | null = null;
super(sourceList, mapper, updater, removeCallback); private _flushing: boolean = false;
this._eventQueue = null;
}
onSubscribeFirst() { onSubscribeFirst(): void {
this._sourceUnsubscribe = this._sourceList.subscribe(this); this._sourceUnsubscribe = this._sourceList.subscribe(this);
this._eventQueue = []; this._eventQueue = [];
this._mappedValues = []; this._mappedValues = [];
@ -35,122 +34,112 @@ export class AsyncMappedList extends BaseMappedList {
this._flush(); this._flush();
} }
async _flush() { async _flush(): Promise<void> {
if (this._flushing) { if (this._flushing) {
return; return;
} }
this._flushing = true; this._flushing = true;
try { try {
while (this._eventQueue.length) { while (this._eventQueue!.length) {
const event = this._eventQueue.shift(); const event = this._eventQueue!.shift();
await event.run(this); await event!.run(this);
} }
} finally { } finally {
this._flushing = false; this._flushing = false;
} }
} }
onReset() { onReset(): void {
if (this._eventQueue) { if (this._eventQueue) {
this._eventQueue.push(new ResetEvent()); this._eventQueue.push(new ResetEvent());
this._flush(); this._flush();
} }
} }
onAdd(index, value) { onAdd(index: number, value: F): void {
if (this._eventQueue) { if (this._eventQueue) {
this._eventQueue.push(new AddEvent(index, value)); this._eventQueue.push(new AddEvent(index, value));
this._flush(); this._flush();
} }
} }
onUpdate(index, value, params) { onUpdate(index: number, value: F, params: any): void {
if (this._eventQueue) { if (this._eventQueue) {
this._eventQueue.push(new UpdateEvent(index, value, params)); this._eventQueue.push(new UpdateEvent(index, value, params));
this._flush(); this._flush();
} }
} }
onRemove(index) { onRemove(index: number): void {
if (this._eventQueue) { if (this._eventQueue) {
this._eventQueue.push(new RemoveEvent(index)); this._eventQueue.push(new RemoveEvent(index));
this._flush(); this._flush();
} }
} }
onMove(fromIdx, toIdx) { onMove(fromIdx: number, toIdx: number): void {
if (this._eventQueue) { if (this._eventQueue) {
this._eventQueue.push(new MoveEvent(fromIdx, toIdx)); this._eventQueue.push(new MoveEvent(fromIdx, toIdx));
this._flush(); this._flush();
} }
} }
onUnsubscribeLast() { onUnsubscribeLast(): void {
this._sourceUnsubscribe(); this._sourceUnsubscribe!();
this._eventQueue = null; this._eventQueue = null;
this._mappedValues = null; this._mappedValues = null;
} }
} }
class AddEvent { type AsyncEvent<F> = AddEvent<F> | UpdateEvent<F> | RemoveEvent<F> | MoveEvent<F> | ResetEvent<F>
constructor(index, value) {
this.index = index;
this.value = value;
}
async run(list) { class AddEvent<F> {
constructor(public index: number, public value: F) {}
async run<T>(list: AsyncMappedList<F,T>): Promise<void> {
const mappedValue = await list._mapper(this.value); const mappedValue = await list._mapper(this.value);
runAdd(list, this.index, mappedValue); runAdd(list, this.index, mappedValue);
} }
} }
class UpdateEvent { class UpdateEvent<F> {
constructor(index, value, params) { constructor(public index: number, public value: F, public params: any) {}
this.index = index;
this.value = value;
this.params = params;
}
async run(list) { async run<T>(list: AsyncMappedList<F,T>): Promise<void> {
runUpdate(list, this.index, this.value, this.params); runUpdate(list, this.index, this.value, this.params);
} }
} }
class RemoveEvent { class RemoveEvent<F> {
constructor(index) { constructor(public index: number) {}
this.index = index;
}
async run(list) { async run<T>(list: AsyncMappedList<F,T>): Promise<void> {
runRemove(list, this.index); runRemove(list, this.index);
} }
} }
class MoveEvent { class MoveEvent<F> {
constructor(fromIdx, toIdx) { constructor(public fromIdx: number, public toIdx: number) {}
this.fromIdx = fromIdx;
this.toIdx = toIdx;
}
async run(list) { async run<T>(list: AsyncMappedList<F,T>): Promise<void> {
runMove(list, this.fromIdx, this.toIdx); runMove(list, this.fromIdx, this.toIdx);
} }
} }
class ResetEvent { class ResetEvent<F> {
async run(list) { async run<T>(list: AsyncMappedList<F,T>): Promise<void> {
runReset(list); runReset(list);
} }
} }
import {ObservableArray} from "./ObservableArray.js"; import {ObservableArray} from "./ObservableArray";
import {ListObserver} from "../../mocks/ListObserver.js"; import {ListObserver} from "../../mocks/ListObserver.js";
export function tests() { export function tests() {
return { return {
"events are emitted in order": async assert => { "events are emitted in order": async assert => {
const double = n => n * n; const double = n => n * n;
const source = new ObservableArray(); const source = new ObservableArray<number>();
const mapper = new AsyncMappedList(source, async n => { const mapper = new AsyncMappedList(source, async n => {
await new Promise(r => setTimeout(r, n)); await new Promise(r => setTimeout(r, n));
return {n: double(n)}; return {n: double(n)};

View file

@ -21,15 +21,15 @@ import {findAndUpdateInArray} from "./common";
export type Mapper<F,T> = (value: F) => T export type Mapper<F,T> = (value: F) => T
export type Updater<F,T> = (mappedValue: T, params: any, value: F) => void; export type Updater<F,T> = (mappedValue: T, params: any, value: F) => void;
export class BaseMappedList<F,T> extends BaseObservableList<T> { export class BaseMappedList<F,T,R = T> extends BaseObservableList<T> {
protected _sourceList: BaseObservableList<F>; protected _sourceList: BaseObservableList<F>;
protected _sourceUnsubscribe: (() => void) | null = null; protected _sourceUnsubscribe: (() => void) | null = null;
_mapper: Mapper<F,T>; _mapper: Mapper<F,R>;
_updater: Updater<F,T>; _updater?: Updater<F,T>;
_removeCallback?: (value: T) => void; _removeCallback?: (value: T) => void;
_mappedValues: T[] | null = null; _mappedValues: T[] | null = null;
constructor(sourceList: BaseObservableList<F>, mapper: Mapper<F,T>, updater: Updater<F,T>, removeCallback?: (value: T) => void) { constructor(sourceList: BaseObservableList<F>, mapper: Mapper<F,R>, updater?: Updater<F,T>, removeCallback?: (value: T) => void) {
super(); super();
this._sourceList = sourceList; this._sourceList = sourceList;
this._mapper = mapper; this._mapper = mapper;
@ -50,12 +50,12 @@ export class BaseMappedList<F,T> extends BaseObservableList<T> {
} }
} }
export function runAdd<F,T>(list: BaseMappedList<F,T>, index: number, mappedValue: T): void { export function runAdd<F,T,R>(list: BaseMappedList<F,T,R>, index: number, mappedValue: T): void {
list._mappedValues!.splice(index, 0, mappedValue); list._mappedValues!.splice(index, 0, mappedValue);
list.emitAdd(index, mappedValue); list.emitAdd(index, mappedValue);
} }
export function runUpdate<F,T>(list: BaseMappedList<F,T>, index: number, value: F, params: any): void { export function runUpdate<F,T,R>(list: BaseMappedList<F,T,R>, index: number, value: F, params: any): void {
const mappedValue = list._mappedValues![index]; const mappedValue = list._mappedValues![index];
if (list._updater) { if (list._updater) {
list._updater(mappedValue, params, value); list._updater(mappedValue, params, value);
@ -63,7 +63,7 @@ export function runUpdate<F,T>(list: BaseMappedList<F,T>, index: number, value:
list.emitUpdate(index, mappedValue, params); list.emitUpdate(index, mappedValue, params);
} }
export function runRemove<F,T>(list: BaseMappedList<F,T>, index: number): void { export function runRemove<F,T,R>(list: BaseMappedList<F,T,R>, index: number): void {
const mappedValue = list._mappedValues![index]; const mappedValue = list._mappedValues![index];
list._mappedValues!.splice(index, 1); list._mappedValues!.splice(index, 1);
if (list._removeCallback) { if (list._removeCallback) {
@ -72,14 +72,14 @@ export function runRemove<F,T>(list: BaseMappedList<F,T>, index: number): void {
list.emitRemove(index, mappedValue); list.emitRemove(index, mappedValue);
} }
export function runMove<F,T>(list: BaseMappedList<F,T>, fromIdx: number, toIdx: number): void { export function runMove<F,T,R>(list: BaseMappedList<F,T,R>, fromIdx: number, toIdx: number): void {
const mappedValue = list._mappedValues![fromIdx]; const mappedValue = list._mappedValues![fromIdx];
list._mappedValues!.splice(fromIdx, 1); list._mappedValues!.splice(fromIdx, 1);
list._mappedValues!.splice(toIdx, 0, mappedValue); list._mappedValues!.splice(toIdx, 0, mappedValue);
list.emitMove(fromIdx, toIdx, mappedValue); list.emitMove(fromIdx, toIdx, mappedValue);
} }
export function runReset<F,T>(list: BaseMappedList<F,T>): void { export function runReset<F,T,R>(list: BaseMappedList<F,T,R>): void {
list._mappedValues = []; list._mappedValues = [];
list.emitReset(); list.emitReset();
} }

View file

@ -24,7 +24,18 @@ export interface IListObserver<T> {
onMove(from: number, to: number, value: T, list: BaseObservableList<T>): void onMove(from: number, to: number, value: T, list: BaseObservableList<T>): void
} }
export abstract class BaseObservableList<T> extends BaseObservable<IListObserver<T>> { export function defaultObserverWith<T>(overrides: { [key in keyof IListObserver<T>]?: IListObserver<T>[key] }): IListObserver<T> {
const defaults = {
onReset(){},
onAdd(){},
onUpdate(){},
onRemove(){},
onMove(){},
}
return Object.assign(defaults, overrides);
}
export abstract class BaseObservableList<T> extends BaseObservable<IListObserver<T>> implements Iterable<T> {
emitReset() { emitReset() {
for(let h of this._handlers) { for(let h of this._handlers) {
h.onReset(this); h.onReset(this);
@ -38,7 +49,7 @@ export abstract class BaseObservableList<T> extends BaseObservable<IListObserver
} }
} }
emitUpdate(index: number, value: T, params: any): void { emitUpdate(index: number, value: T, params?: any): void {
for(let h of this._handlers) { for(let h of this._handlers) {
h.onUpdate(index, value, params, this); h.onUpdate(index, value, params, this);
} }
@ -58,6 +69,6 @@ export abstract class BaseObservableList<T> extends BaseObservable<IListObserver
} }
} }
abstract [Symbol.iterator](): IterableIterator<T>; abstract [Symbol.iterator](): Iterator<T>;
abstract get length(): number; abstract get length(): number;
} }

View file

@ -14,16 +14,18 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {BaseObservableList} from "./BaseObservableList"; import {BaseObservableList, IListObserver} from "./BaseObservableList";
export class ConcatList extends BaseObservableList { export class ConcatList<T> extends BaseObservableList<T> implements IListObserver<T> {
constructor(...sourceLists) { protected _sourceLists: BaseObservableList<T>[];
protected _sourceUnsubscribes: (() => void)[] | null = null;
constructor(...sourceLists: BaseObservableList<T>[]) {
super(); super();
this._sourceLists = sourceLists; this._sourceLists = sourceLists;
this._sourceUnsubscribes = null;
} }
_offsetForSource(sourceList) { _offsetForSource(sourceList: BaseObservableList<T>): number {
const listIdx = this._sourceLists.indexOf(sourceList); const listIdx = this._sourceLists.indexOf(sourceList);
let offset = 0; let offset = 0;
for (let i = 0; i < listIdx; ++i) { for (let i = 0; i < listIdx; ++i) {
@ -32,17 +34,17 @@ export class ConcatList extends BaseObservableList {
return offset; return offset;
} }
onSubscribeFirst() { onSubscribeFirst(): void {
this._sourceUnsubscribes = this._sourceLists.map(sourceList => sourceList.subscribe(this)); this._sourceUnsubscribes = this._sourceLists.map(sourceList => sourceList.subscribe(this));
} }
onUnsubscribeLast() { onUnsubscribeLast(): void {
for (const sourceUnsubscribe of this._sourceUnsubscribes) { for (const sourceUnsubscribe of this._sourceUnsubscribes!) {
sourceUnsubscribe(); sourceUnsubscribe();
} }
} }
onReset() { onReset(): void {
// TODO: not ideal if other source lists are large // TODO: not ideal if other source lists are large
// but working impl for now // but working impl for now
// reset, and // reset, and
@ -54,11 +56,11 @@ export class ConcatList extends BaseObservableList {
} }
} }
onAdd(index, value, sourceList) { onAdd(index: number, value: T, sourceList: BaseObservableList<T>): void {
this.emitAdd(this._offsetForSource(sourceList) + index, value); this.emitAdd(this._offsetForSource(sourceList) + index, value);
} }
onUpdate(index, value, params, sourceList) { onUpdate(index: number, value: T, params: any, sourceList: BaseObservableList<T>): void {
// if an update is emitted while calling source.subscribe() from onSubscribeFirst, ignore it // if an update is emitted while calling source.subscribe() from onSubscribeFirst, ignore it
// as we are not supposed to call `length` on any uninitialized list // as we are not supposed to call `length` on any uninitialized list
if (!this._sourceUnsubscribes) { if (!this._sourceUnsubscribes) {
@ -67,16 +69,16 @@ export class ConcatList extends BaseObservableList {
this.emitUpdate(this._offsetForSource(sourceList) + index, value, params); this.emitUpdate(this._offsetForSource(sourceList) + index, value, params);
} }
onRemove(index, value, sourceList) { onRemove(index: number, value: T, sourceList: BaseObservableList<T>): void {
this.emitRemove(this._offsetForSource(sourceList) + index, value); this.emitRemove(this._offsetForSource(sourceList) + index, value);
} }
onMove(fromIdx, toIdx, value, sourceList) { onMove(fromIdx: number, toIdx: number, value: T, sourceList: BaseObservableList<T>): void {
const offset = this._offsetForSource(sourceList); const offset = this._offsetForSource(sourceList);
this.emitMove(offset + fromIdx, offset + toIdx, value); this.emitMove(offset + fromIdx, offset + toIdx, value);
} }
get length() { get length(): number {
let len = 0; let len = 0;
for (let i = 0; i < this._sourceLists.length; ++i) { for (let i = 0; i < this._sourceLists.length; ++i) {
len += this._sourceLists[i].length; len += this._sourceLists[i].length;
@ -104,7 +106,8 @@ export class ConcatList extends BaseObservableList {
} }
} }
import {ObservableArray} from "./ObservableArray.js"; import {ObservableArray} from "./ObservableArray";
import {defaultObserverWith} from "./BaseObservableList";
export async function tests() { export async function tests() {
return { return {
test_length(assert) { test_length(assert) {
@ -133,13 +136,13 @@ export async function tests() {
const list2 = new ObservableArray([11, 12, 13]); const list2 = new ObservableArray([11, 12, 13]);
const all = new ConcatList(list1, list2); const all = new ConcatList(list1, list2);
let fired = false; let fired = false;
all.subscribe({ all.subscribe(defaultObserverWith({
onAdd(index, value) { onAdd(index, value) {
fired = true; fired = true;
assert.equal(index, 4); assert.equal(index, 4);
assert.equal(value, 11.5); assert.equal(value, 11.5);
} }
}); }));
list2.insert(1, 11.5); list2.insert(1, 11.5);
assert(fired); assert(fired);
}, },
@ -148,13 +151,13 @@ export async function tests() {
const list2 = new ObservableArray([11, 12, 13]); const list2 = new ObservableArray([11, 12, 13]);
const all = new ConcatList(list1, list2); const all = new ConcatList(list1, list2);
let fired = false; let fired = false;
all.subscribe({ all.subscribe(defaultObserverWith({
onUpdate(index, value) { onUpdate(index, value) {
fired = true; fired = true;
assert.equal(index, 4); assert.equal(index, 4);
assert.equal(value, 10); assert.equal(value, 10);
} }
}); }));
list2.emitUpdate(1, 10); list2.emitUpdate(1, 10);
assert(fired); assert(fired);
}, },

View file

@ -15,9 +15,10 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {IListObserver} from "./BaseObservableList";
import {BaseMappedList, runAdd, runUpdate, runRemove, runMove, runReset} from "./BaseMappedList"; import {BaseMappedList, runAdd, runUpdate, runRemove, runMove, runReset} from "./BaseMappedList";
export class MappedList extends BaseMappedList { export class MappedList<F,T> extends BaseMappedList<F,T> implements IListObserver<F> {
onSubscribeFirst() { onSubscribeFirst() {
this._sourceUnsubscribe = this._sourceList.subscribe(this); this._sourceUnsubscribe = this._sourceList.subscribe(this);
this._mappedValues = []; this._mappedValues = [];
@ -26,16 +27,16 @@ export class MappedList extends BaseMappedList {
} }
} }
onReset() { onReset(): void {
runReset(this); runReset(this);
} }
onAdd(index, value) { onAdd(index: number, value: F): void {
const mappedValue = this._mapper(value); const mappedValue = this._mapper(value);
runAdd(this, index, mappedValue); runAdd(this, index, mappedValue);
} }
onUpdate(index, value, params) { onUpdate(index: number, value: F, params: any): void {
// if an update is emitted while calling source.subscribe() from onSubscribeFirst, ignore it // if an update is emitted while calling source.subscribe() from onSubscribeFirst, ignore it
if (!this._mappedValues) { if (!this._mappedValues) {
return; return;
@ -43,24 +44,25 @@ export class MappedList extends BaseMappedList {
runUpdate(this, index, value, params); runUpdate(this, index, value, params);
} }
onRemove(index) { onRemove(index: number): void {
runRemove(this, index); runRemove(this, index);
} }
onMove(fromIdx, toIdx) { onMove(fromIdx: number, toIdx: number): void {
runMove(this, fromIdx, toIdx); runMove(this, fromIdx, toIdx);
} }
onUnsubscribeLast() { onUnsubscribeLast(): void {
this._sourceUnsubscribe(); this._sourceUnsubscribe!();
} }
} }
import {ObservableArray} from "./ObservableArray.js"; import {ObservableArray} from "./ObservableArray";
import {BaseObservableList} from "./BaseObservableList"; import {BaseObservableList} from "./BaseObservableList";
import {defaultObserverWith} from "./BaseObservableList";
export async function tests() { export async function tests() {
class MockList extends BaseObservableList { class MockList extends BaseObservableList<number> {
get length() { get length() {
return 0; return 0;
} }
@ -74,26 +76,26 @@ export async function tests() {
const source = new MockList(); const source = new MockList();
const mapped = new MappedList(source, n => {return {n: n*n};}); const mapped = new MappedList(source, n => {return {n: n*n};});
let fired = false; let fired = false;
const unsubscribe = mapped.subscribe({ const unsubscribe = mapped.subscribe(defaultObserverWith({
onAdd(idx, value) { onAdd(idx, value) {
fired = true; fired = true;
assert.equal(idx, 0); assert.equal(idx, 0);
assert.equal(value.n, 36); assert.equal(value.n, 36);
} }
}); }));
source.emitAdd(0, 6); source.emitAdd(0, 6);
assert(fired); assert(fired);
unsubscribe(); unsubscribe();
}, },
test_update(assert) { test_update(assert) {
const source = new MockList(); const source = new MockList();
const mapped = new MappedList( const mapped = new MappedList<number, { n: number, m?: number }>(
source, source,
n => {return {n: n*n};}, n => {return {n: n*n};},
(o, p, n) => o.m = n*n (o, p, n) => o.m = n*n
); );
let fired = false; let fired = false;
const unsubscribe = mapped.subscribe({ const unsubscribe = mapped.subscribe(defaultObserverWith({
onAdd() {}, onAdd() {},
onUpdate(idx, value) { onUpdate(idx, value) {
fired = true; fired = true;
@ -101,7 +103,7 @@ export async function tests() {
assert.equal(value.n, 36); assert.equal(value.n, 36);
assert.equal(value.m, 49); assert.equal(value.m, 49);
} }
}); }));
source.emitAdd(0, 6); source.emitAdd(0, 6);
source.emitUpdate(0, 7); source.emitUpdate(0, 7);
assert(fired); assert(fired);
@ -113,9 +115,9 @@ export async function tests() {
source, source,
n => {return n*n;} n => {return n*n;}
); );
mapped.subscribe({ mapped.subscribe(defaultObserverWith({
onUpdate() { assert.fail(); } onUpdate() { assert.fail(); }
}); }));
assert.equal(mapped.findAndUpdate( assert.equal(mapped.findAndUpdate(
n => n === 100, n => n === 100,
() => assert.fail() () => assert.fail()
@ -127,9 +129,9 @@ export async function tests() {
source, source,
n => {return n*n;} n => {return n*n;}
); );
mapped.subscribe({ mapped.subscribe(defaultObserverWith({
onUpdate() { assert.fail(); } onUpdate() { assert.fail(); }
}); }));
let fired = false; let fired = false;
assert.equal(mapped.findAndUpdate( assert.equal(mapped.findAndUpdate(
n => n === 9, n => n === 9,
@ -148,14 +150,14 @@ export async function tests() {
n => {return n*n;} n => {return n*n;}
); );
let fired = false; let fired = false;
mapped.subscribe({ mapped.subscribe(defaultObserverWith({
onUpdate(idx, n, params) { onUpdate(idx, n, params) {
assert.equal(idx, 1); assert.equal(idx, 1);
assert.equal(n, 9); assert.equal(n, 9);
assert.equal(params, "param"); assert.equal(params, "param");
fired = true; fired = true;
} }
}); }));
assert.equal(mapped.findAndUpdate(n => n === 9, () => "param"), true); assert.equal(mapped.findAndUpdate(n => n === 9, () => "param"), true);
assert.equal(fired, true); assert.equal(fired, true);
}, },

View file

@ -16,52 +16,62 @@ limitations under the License.
import {BaseObservableList} from "./BaseObservableList"; import {BaseObservableList} from "./BaseObservableList";
export class ObservableArray extends BaseObservableList { export class ObservableArray<T> extends BaseObservableList<T> {
constructor(initialValues = []) { private _items: T[];
constructor(initialValues: T[] = []) {
super(); super();
this._items = initialValues; this._items = initialValues;
} }
append(item) { append(item: T): void {
this._items.push(item); this._items.push(item);
this.emitAdd(this._items.length - 1, item); this.emitAdd(this._items.length - 1, item);
} }
remove(idx) { remove(idx: number): void {
const [item] = this._items.splice(idx, 1); const [item] = this._items.splice(idx, 1);
this.emitRemove(idx, item); this.emitRemove(idx, item);
} }
insertMany(idx, items) { insertMany(idx: number, items: T[]): void {
for(let item of items) { for(let item of items) {
this.insert(idx, item); this.insert(idx, item);
idx += 1; idx += 1;
} }
} }
insert(idx, item) { insert(idx: number, item: T): void {
this._items.splice(idx, 0, item); this._items.splice(idx, 0, item);
this.emitAdd(idx, item); this.emitAdd(idx, item);
} }
update(idx, item, params = null) { move(fromIdx: number, toIdx: number): void {
if (fromIdx < this._items.length && toIdx < this._items.length) {
const [item] = this._items.splice(fromIdx, 1);
this._items.splice(toIdx, 0, item);
this.emitMove(fromIdx, toIdx, item);
}
}
update(idx: number, item: T, params: any = null): void {
if (idx < this._items.length) { if (idx < this._items.length) {
this._items[idx] = item; this._items[idx] = item;
this.emitUpdate(idx, item, params); this.emitUpdate(idx, item, params);
} }
} }
get array() { get array(): Readonly<T[]> {
return this._items; return this._items;
} }
at(idx) { at(idx: number): T | undefined {
if (this._items && idx >= 0 && idx < this._items.length) { if (this._items && idx >= 0 && idx < this._items.length) {
return this._items[idx]; return this._items[idx];
} }
} }
get length() { get length(): number {
return this._items.length; return this._items.length;
} }

View file

@ -15,21 +15,23 @@ limitations under the License.
*/ */
import {BaseObservableList} from "./BaseObservableList"; import {BaseObservableList} from "./BaseObservableList";
import {sortedIndex} from "../../utils/sortedIndex.js"; import {sortedIndex} from "../../utils/sortedIndex";
import {findAndUpdateInArray} from "./common"; import {findAndUpdateInArray} from "./common";
export class SortedArray extends BaseObservableList { export class SortedArray<T> extends BaseObservableList<T> {
constructor(comparator) { private _comparator: (left: T, right: T) => number;
private _items: T[] = [];
constructor(comparator: (left: T, right: T) => number) {
super(); super();
this._comparator = comparator; this._comparator = comparator;
this._items = [];
} }
setManyUnsorted(items) { setManyUnsorted(items: T[]): void {
this.setManySorted(items); this.setManySorted(items);
} }
setManySorted(items) { setManySorted(items: T[]): void {
// TODO: we can make this way faster by only looking up the first and last key, // TODO: we can make this way faster by only looking up the first and last key,
// and merging whatever is inbetween with items // and merging whatever is inbetween with items
// if items is not sorted, 💩🌀 will follow! // if items is not sorted, 💩🌀 will follow!
@ -42,11 +44,11 @@ export class SortedArray extends BaseObservableList {
} }
} }
findAndUpdate(predicate, updater) { findAndUpdate(predicate: (value: T) => boolean, updater: (value: T) => any | false): boolean {
return findAndUpdateInArray(predicate, this._items, this, updater); return findAndUpdateInArray(predicate, this._items, this, updater);
} }
getAndUpdate(item, updater, updateParams = null) { getAndUpdate(item: T, updater: (existing: T, item: T) => any, updateParams: any = null): void {
const idx = this.indexOf(item); const idx = this.indexOf(item);
if (idx !== -1) { if (idx !== -1) {
const existingItem = this._items[idx]; const existingItem = this._items[idx];
@ -56,7 +58,7 @@ export class SortedArray extends BaseObservableList {
} }
} }
update(item, updateParams = null) { update(item: T, updateParams: any = null): void {
const idx = this.indexOf(item); const idx = this.indexOf(item);
if (idx !== -1) { if (idx !== -1) {
this._items[idx] = item; this._items[idx] = item;
@ -64,7 +66,7 @@ export class SortedArray extends BaseObservableList {
} }
} }
indexOf(item) { indexOf(item: T): number {
const idx = sortedIndex(this._items, item, this._comparator); const idx = sortedIndex(this._items, item, this._comparator);
if (idx < this._items.length && this._comparator(this._items[idx], item) === 0) { if (idx < this._items.length && this._comparator(this._items[idx], item) === 0) {
return idx; return idx;
@ -73,7 +75,7 @@ export class SortedArray extends BaseObservableList {
} }
} }
_getNext(item) { _getNext(item: T): T | undefined {
let idx = sortedIndex(this._items, item, this._comparator); let idx = sortedIndex(this._items, item, this._comparator);
while(idx < this._items.length && this._comparator(this._items[idx], item) <= 0) { while(idx < this._items.length && this._comparator(this._items[idx], item) <= 0) {
idx += 1; idx += 1;
@ -81,7 +83,7 @@ export class SortedArray extends BaseObservableList {
return this.get(idx); return this.get(idx);
} }
set(item, updateParams = null) { set(item: T, updateParams: any = null): void {
const idx = sortedIndex(this._items, item, this._comparator); const idx = sortedIndex(this._items, item, this._comparator);
if (idx >= this._items.length || this._comparator(this._items[idx], item) !== 0) { if (idx >= this._items.length || this._comparator(this._items[idx], item) !== 0) {
this._items.splice(idx, 0, item); this._items.splice(idx, 0, item);
@ -92,21 +94,21 @@ export class SortedArray extends BaseObservableList {
} }
} }
get(idx) { get(idx: number): T | undefined {
return this._items[idx]; return this._items[idx];
} }
remove(idx) { remove(idx: number): void {
const item = this._items[idx]; const item = this._items[idx];
this._items.splice(idx, 1); this._items.splice(idx, 1);
this.emitRemove(idx, item); this.emitRemove(idx, item);
} }
get array() { get array(): T[] {
return this._items; return this._items;
} }
get length() { get length(): number {
return this._items.length; return this._items.length;
} }
@ -116,8 +118,11 @@ export class SortedArray extends BaseObservableList {
} }
// iterator that works even if the current value is removed while iterating // iterator that works even if the current value is removed while iterating
class Iterator { class Iterator<T> {
constructor(sortedArray) { private _sortedArray: SortedArray<T> | null
private _current: T | null | undefined
constructor(sortedArray: SortedArray<T>) {
this._sortedArray = sortedArray; this._sortedArray = sortedArray;
this._current = null; this._current = null;
} }
@ -145,7 +150,7 @@ class Iterator {
export function tests() { export function tests() {
return { return {
"setManyUnsorted": assert => { "setManyUnsorted": assert => {
const sa = new SortedArray((a, b) => a.localeCompare(b)); const sa = new SortedArray<string>((a, b) => a.localeCompare(b));
sa.setManyUnsorted(["b", "a", "c"]); sa.setManyUnsorted(["b", "a", "c"]);
assert.equal(sa.length, 3); assert.equal(sa.length, 3);
assert.equal(sa.get(0), "a"); assert.equal(sa.get(0), "a");
@ -153,7 +158,7 @@ export function tests() {
assert.equal(sa.get(2), "c"); assert.equal(sa.get(2), "c");
}, },
"_getNext": assert => { "_getNext": assert => {
const sa = new SortedArray((a, b) => a.localeCompare(b)); const sa = new SortedArray<string>((a, b) => a.localeCompare(b));
sa.setManyUnsorted(["b", "a", "f"]); sa.setManyUnsorted(["b", "a", "f"]);
assert.equal(sa._getNext("a"), "b"); assert.equal(sa._getNext("a"), "b");
assert.equal(sa._getNext("b"), "f"); assert.equal(sa._getNext("b"), "f");
@ -162,7 +167,7 @@ export function tests() {
assert.equal(sa._getNext("f"), undefined); assert.equal(sa._getNext("f"), undefined);
}, },
"iterator with removals": assert => { "iterator with removals": assert => {
const queue = new SortedArray((a, b) => a.idx - b.idx); const queue = new SortedArray<{idx: number}>((a, b) => a.idx - b.idx);
queue.setManyUnsorted([{idx: 5}, {idx: 3}, {idx: 1}, {idx: 4}, {idx: 2}]); queue.setManyUnsorted([{idx: 5}, {idx: 3}, {idx: 1}, {idx: 4}, {idx: 2}]);
const it = queue[Symbol.iterator](); const it = queue[Symbol.iterator]();
assert.equal(it.next().value.idx, 1); assert.equal(it.next().value.idx, 1);

View file

@ -15,7 +15,7 @@ limitations under the License.
*/ */
import {BaseObservableList} from "./BaseObservableList"; import {BaseObservableList} from "./BaseObservableList";
import {sortedIndex} from "../../utils/sortedIndex.js"; import {sortedIndex} from "../../utils/sortedIndex";
/* /*

View file

@ -14,8 +14,9 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import aesjs from "../../../lib/aes-js/index.js"; import aesjs from "aes-js";
import {hkdf} from "../../utils/crypto/hkdf.js"; import {hkdf} from "../../utils/crypto/hkdf";
import {Platform as ModernPlatform} from "./Platform.js"; import {Platform as ModernPlatform} from "./Platform.js";
export function Platform(container, paths) { export function Platform(container, paths) {

View file

@ -17,12 +17,12 @@ limitations under the License.
import {createFetchRequest} from "./dom/request/fetch.js"; import {createFetchRequest} from "./dom/request/fetch.js";
import {xhrRequest} from "./dom/request/xhr.js"; import {xhrRequest} from "./dom/request/xhr.js";
import {StorageFactory} from "../../matrix/storage/idb/StorageFactory"; import {StorageFactory} from "../../matrix/storage/idb/StorageFactory";
import {SessionInfoStorage} from "../../matrix/sessioninfo/localstorage/SessionInfoStorage.js"; import {SessionInfoStorage} from "../../matrix/sessioninfo/localstorage/SessionInfoStorage";
import {SettingsStorage} from "./dom/SettingsStorage.js"; import {SettingsStorage} from "./dom/SettingsStorage.js";
import {Encoding} from "./utils/Encoding.js"; import {Encoding} from "./utils/Encoding.js";
import {OlmWorker} from "../../matrix/e2ee/OlmWorker.js"; import {OlmWorker} from "../../matrix/e2ee/OlmWorker.js";
import {IDBLogger} from "../../logging/IDBLogger.js"; import {IDBLogger} from "../../logging/IDBLogger";
import {ConsoleLogger} from "../../logging/ConsoleLogger.js"; import {ConsoleLogger} from "../../logging/ConsoleLogger";
import {RootView} from "./ui/RootView.js"; import {RootView} from "./ui/RootView.js";
import {Clock} from "./dom/Clock.js"; import {Clock} from "./dom/Clock.js";
import {ServiceWorkerHandler} from "./dom/ServiceWorkerHandler.js"; import {ServiceWorkerHandler} from "./dom/ServiceWorkerHandler.js";
@ -35,7 +35,7 @@ import {WorkerPool} from "./dom/WorkerPool.js";
import {BlobHandle} from "./dom/BlobHandle.js"; import {BlobHandle} from "./dom/BlobHandle.js";
import {hasReadPixelPermission, ImageHandle, VideoHandle} from "./dom/ImageHandle.js"; import {hasReadPixelPermission, ImageHandle, VideoHandle} from "./dom/ImageHandle.js";
import {downloadInIframe} from "./dom/download.js"; import {downloadInIframe} from "./dom/download.js";
import {Disposables} from "../../utils/Disposables.js"; import {Disposables} from "../../utils/Disposables";
import {parseHTML} from "./parsehtml.js"; import {parseHTML} from "./parsehtml.js";
import {handleAvatarError} from "./ui/avatar.js"; import {handleAvatarError} from "./ui/avatar.js";
@ -67,21 +67,22 @@ async function loadOlm(olmPaths) {
} }
return null; return null;
} }
// turn asset path to absolute path if it isn't already
// make path relative to basePath, // so it can be loaded independent of base
// assuming it and basePath are relative to document function assetAbsPath(assetPath) {
function relPath(path, basePath) { if (!assetPath.startsWith("/")) {
const idx = basePath.lastIndexOf("/"); return new URL(assetPath, document.location.href).pathname;
const dir = idx === -1 ? "" : basePath.slice(0, idx); }
const dirCount = dir.length ? dir.split("/").length : 0; return assetPath;
return "../".repeat(dirCount) + path;
} }
async function loadOlmWorker(config) { async function loadOlmWorker(config) {
const workerPool = new WorkerPool(config.worker, 4); const workerPool = new WorkerPool(config.worker, 4);
await workerPool.init(); await workerPool.init();
const path = relPath(config.olm.legacyBundle, config.worker); await workerPool.sendAll({
await workerPool.sendAll({type: "load_olm", path}); type: "load_olm",
path: assetAbsPath(config.olm.legacyBundle)
});
const olmWorker = new OlmWorker(workerPool); const olmWorker = new OlmWorker(workerPool);
return olmWorker; return olmWorker;
} }
@ -132,11 +133,7 @@ export class Platform {
this.clock = new Clock(); this.clock = new Clock();
this.encoding = new Encoding(); this.encoding = new Encoding();
this.random = Math.random; this.random = Math.random;
if (options?.development) { this._createLogger(options?.development);
this.logger = new ConsoleLogger({platform: this});
} else {
this.logger = new IDBLogger({name: "hydrogen_logs", platform: this});
}
this.history = new History(); this.history = new History();
this.onlineStatus = new OnlineStatus(); this.onlineStatus = new OnlineStatus();
this._serviceWorkerHandler = null; this._serviceWorkerHandler = null;
@ -162,6 +159,21 @@ export class Platform {
this._disposables = new Disposables(); this._disposables = new Disposables();
} }
_createLogger(isDevelopment) {
// Make sure that loginToken does not end up in the logs
const transformer = (item) => {
if (item.e?.stack) {
item.e.stack = item.e.stack.replace(/\/\?loginToken=(.+)/, "?loginToken=<snip>");
}
return item;
};
if (isDevelopment) {
this.logger = new ConsoleLogger({platform: this});
} else {
this.logger = new IDBLogger({name: "hydrogen_logs", platform: this, serializedTransformer: transformer});
}
}
get updateService() { get updateService() {
return this._serviceWorkerHandler; return this._serviceWorkerHandler;
} }
@ -265,10 +277,37 @@ export class Platform {
} }
get version() { get version() {
return window.HYDROGEN_VERSION; return DEFINE_VERSION;
} }
dispose() { dispose() {
this._disposables.dispose(); this._disposables.dispose();
} }
} }
import {LogItem} from "../../logging/LogItem";
export function tests() {
return {
"loginToken should not be in logs": (assert) => {
const transformer = (item) => {
if (item.e?.stack) {
item.e.stack = item.e.stack.replace(/(?<=\/\?loginToken=).+/, "<snip>");
}
return item;
};
const logger = {
_queuedItems: [],
_serializedTransformer: transformer,
_now: () => {}
};
logger.persist = IDBLogger.prototype._persistItem.bind(logger);
const logItem = new LogItem("test", 1, logger);
logItem.error = new Error();
logItem.error.stack = "main http://localhost:3000/src/main.js:55\n<anonymous> http://localhost:3000/?loginToken=secret:26"
logger.persist(logItem, null, false);
const item = logger._queuedItems.pop();
console.log(item);
assert.strictEqual(item.json.search("secret"), -1);
}
};
}

View file

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

View file

Before

Width:  |  Height:  |  Size: 1 KiB

After

Width:  |  Height:  |  Size: 1 KiB

View file

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

View file

Before

Width:  |  Height:  |  Size: 968 B

After

Width:  |  Height:  |  Size: 968 B

View file

@ -3,10 +3,10 @@
"short_name": "Hydrogen", "short_name": "Hydrogen",
"display": "standalone", "display": "standalone",
"description": "Lightweight matrix client with legacy and mobile browser support", "description": "Lightweight matrix client with legacy and mobile browser support",
"start_url": "index.html", "start_url": "../index.html",
"icons": [ "icons": [
{"src": "assets/icon.png", "sizes": "384x384", "type": "image/png"}, {"src": "icon.png", "sizes": "384x384", "type": "image/png"},
{"src": "assets/icon-maskable.png", "sizes": "384x384", "type": "image/png", "purpose": "maskable"} {"src": "icon-maskable.png", "sizes": "384x384", "type": "image/png", "purpose": "maskable"}
], ],
"theme_color": "#0DBD8B" "theme_color": "#0DBD8B"
} }

View file

@ -1,43 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Hydrogen Chat</title>
<meta name="viewport" content="width=device-width, user-scalable=no">
<meta name="application-name" content="Hydrogen Chat"/>
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<meta name="apple-mobile-web-app-title" content="Hydrogen Chat">
<meta name="description" content="A matrix chat application">
<link rel="apple-touch-icon" href="assets/icon-maskable.png">
<link rel="icon" type="image/png" href="assets/icon-maskable.png">
<link rel="stylesheet" type="text/css" href="src/platform/web/ui/css/main.css">
<link rel="stylesheet" type="text/css" href="src/platform/web/ui/css/themes/element/theme.css" title="Element Theme">
<link rel="alternate stylesheet" type="text/css" href="src/platform/web/ui/css/themes/bubbles/theme.css" title="Bubbles Theme">
</head>
<body class="hydrogen">
<script id="version" type="disabled">
window.HYDROGEN_VERSION = "%%VERSION%%";
window.HYDROGEN_GLOBAL_HASH = "%%GLOBAL_HASH%%";
</script>
<script id="main" type="module">
import {main} from "./src/main.js";
import {Platform} from "./src/platform/web/Platform.js";
main(new Platform(document.body, {
worker: "src/worker.js",
downloadSandbox: "assets/download-sandbox.html",
defaultHomeServer: "matrix.org",
// NOTE: uncomment this if you want the service worker for local development
// serviceWorker: "sw.js",
// NOTE: provide push config if you want push notifs for local development
// see assets/config.json for what the config looks like
// push: {...},
olm: {
wasm: "lib/olm/olm.wasm",
legacyBundle: "lib/olm/olm_legacy.js",
wasmBundle: "lib/olm/olm.js",
}
}, null, {development: true}));
</script>
</body>
</html>

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {AbortError} from "../../../utils/error.js"; import {AbortError} from "../../../utils/error";
class Timeout { class Timeout {
constructor(ms) { constructor(ms) {

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import base64 from "../../../../lib/base64-arraybuffer/index.js"; import base64 from "base64-arraybuffer";
// turn IE11 result into promise // turn IE11 result into promise
function subtleCryptoResult(promiseOrOp, method) { function subtleCryptoResult(promiseOrOp, method) {

View file

@ -181,11 +181,11 @@ export class ServiceWorkerHandler {
} }
get version() { get version() {
return window.HYDROGEN_VERSION; return DEFINE_VERSION;
} }
get buildHash() { get buildHash() {
return window.HYDROGEN_GLOBAL_HASH; return DEFINE_GLOBAL_HASH;
} }
async preventConcurrentSessionAccess(sessionId) { async preventConcurrentSessionAccess(sessionId) {

View file

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
*/ */
import {AbortError} from "../../../utils/error.js"; import {AbortError} from "../../../utils/error";
class WorkerState { class WorkerState {
constructor(worker) { constructor(worker) {

Some files were not shown because too many files have changed in this diff Show more