Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(web): move common/web/utils β†’ web/src/engine/common/utils/ πŸ—οΈ #12034

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion common/models/templates/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ builder_describe "Builds the predictive-text model template implementation modul
"@/common/web/keyman-version" \
"@/common/web/es-bundling" \
"@/common/models/wordbreakers" \
"@/common/web/utils" \
"@/web/src/engine/common/utils" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this is only going to be okay if we are also moving common/models/templates over, otherwise we are introducing a deep dep from common -> web.

Copy link
Contributor

@jahorton jahorton Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also of note: some of my current predictive-text work is likely to see developer/src/kmc-model import from common/models/templates.

Copy link
Contributor

@jahorton jahorton Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From #11994 (currently, WIP):

https://github.com/keymanapp/keyman/blob/5011d7fed68f79e0368f42471d2c74bc7742556d/common/models/templates/src/trie-builder.ts pulls in the primary Trie-construction code currently found in kmc-model. We'll likely want it for user-dictionary support.

Accordingly...

import { ModelCompilerError, ModelCompilerMessageContext, ModelCompilerMessages } from "./model-compiler-messages.js";
import { callbacks } from "./compiler-callbacks.js";
import { TrieBuilder } from '@keymanapp/models-templates';

Copy link
Contributor

@jahorton jahorton Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a similar vein, #11088 will want to affect the same section of code (i.e, trie building), adding better Trie compression strategies. The compression part could be kept kmc-model only in theory, but the decompression part will certainly be needed within the live model - and personally, it makes the most sense if the two parts can be tested together in parallel. My current plan is to place the definition for both within common/models/templates.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that's a bad dependency chain we are setting up for ourselves there -- developer/src/kmc-model -> common/models/templates -> web/src/engine/common/utils.

I think we need to discuss this more before we continue moving things around, because we are treading on each others' toes a bit, and it's liable to become (more of) a mess if we are not careful!

"clean" \
"configure" \
"build" \
Expand Down
1 change: 0 additions & 1 deletion common/web/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ THIS_SCRIPT="$(readlink -f "${BASH_SOURCE[0]}")"
builder_describe "Keyman common web modules" \
:keyman-version \
:types \
:utils \
clean \
configure \
build \
Expand Down
4 changes: 2 additions & 2 deletions common/web/gesture-recognizer/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ BUNDLE_CMD="node $KEYMAN_ROOT/common/web/es-bundling/build/common-bundle.mjs"

builder_describe "Builds the gesture-recognition model for Web-based on-screen keyboards" \
"@/common/web/es-bundling build" \
"@/common/web/utils build" \
"@/web/src/engine/common/utils build" \
"clean" \
"configure" \
"build" \
Expand Down Expand Up @@ -75,4 +75,4 @@ fi

if builder_has_action test:tools && ! builder_has_action test:module; then
echo "The $(builder_term test:tools) action is currently a no-op."
fi
fi
2 changes: 1 addition & 1 deletion common/web/keyboard-processor/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ builder_describe \
"@/common/web/keyman-version" \
"@/common/web/es-bundling" \
"@/common/web/types" \
"@/common/web/utils" \
"@/web/src/engine/common/utils" \
configure \
clean \
build \
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion resources/gosh/gosh.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"foo": "gosh ./foo.sh ..."
}
For example, in common/web/utils/package.json, npm run build is defined as:
For example, in web/src/engine/common/utils/package.json, npm run build is defined as:
"build": "gosh ./build.sh -skip-package-install"
Expand Down
2 changes: 1 addition & 1 deletion tsconfig.base.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"@keymanapp/models-types": ["./common/models/types"],
"@keymanapp/models-templates": ["./common/models/templates"],
"@keymanapp/models-wordbreakers": ["./common/models/wordbreakers"],
"@keymanapp/web-utils": ["./common/web/utils"],
"@keymanapp/web-utils": ["./web/src/engine/common/utils"],
"@keymanapp/lm-message-types": ["./common/web/lm-message-types"],
"@keymanapp/keyman-version": ["./common/web/keyman-version"],
"@keymanapp/ldml-keyboard-constants": [ "./core/include/ldml" ],
Expand Down
4 changes: 2 additions & 2 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
{ "path": "./common/web/recorder/tsconfig.json" },
{ "path": "./common/web/sentry-manager/src/tsconfig.json" },
{ "path": "./common/web/types/" },
{ "path": "./common/web/utils/tsconfig.json" },

{ "path": "./developer/src/common/web/test-helpers/tsconfig.json" },
{ "path": "./developer/src/common/web/utils/tsconfig.json" },
Expand All @@ -45,7 +44,8 @@
{ "path": "./resources/build/version/" },

{ "path": "./web/src/tsconfig.all.json" },
{ "path": "./web/src/engine/common/utils/tsconfig.json" },
// { "path": "./web/tools/recorder/tsconfig.json" },
// { "path": "./web/tools/sourcemap-root/tsconfig.json" },
]
}
}
103 changes: 62 additions & 41 deletions web/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,33 +9,38 @@ configure your build environment.

The following folders contain the distribution for Keyman Engine for Web:

src Source code
build/app/resources OSK + UI resources for inclusion in all build types;
keymanweb-osk.ttf is maintained at https://github.com/silnrsi/font-keymanweb-osk

build/app/browser/release Fully-compiled KeymanWeb modules for release
build/app/webview/release Fully-compiled KMEA/KMEI modules for inclusion in mobile app builds
build/app/browser/debug Fully-compiled but non-minified KeymanWeb modules
build/app/webview/debug Fully-compiled but non-minified KMEA/KMEI modules

src/samples Sample pages demonstrating ways to link with KeymanWeb
src/test/manual Test-case web-pages for various aspects of KeymanWeb functionality
src/test/auto A Node-driven test suite for automated testing of KeymanWeb
```text
src Source code
build/app/resources OSK + UI resources for inclusion in all build types;
keymanweb-osk.ttf is maintained at https://github.com/silnrsi/font-keymanweb-osk

build/app/browser/release Fully-compiled KeymanWeb modules for release
build/app/webview/release Fully-compiled KMEA/KMEI modules for inclusion in
mobile app builds
build/app/browser/debug Fully-compiled but non-minified KeymanWeb modules
build/app/webview/debug Fully-compiled but non-minified KMEA/KMEI modules

src/samples Sample pages demonstrating ways to link with KeymanWeb
src/test/manual Test-case web-pages for various aspects of KeymanWeb functionality
src/test/auto A Node-driven test suite for automated testing of KeymanWeb
```

**********************************************************************

## Usage

Open **index.html** or **samples/index.html** in your browser. Be sure to
compile Keyman Engine for Web before viewing the pages.

Refer to the samples for usage details.

To view pages using compiled Keyman Engine for Web,

1. cd to **keyman/web/**
2. Run `./build.sh`
- Use `./build.sh --help` for the script's documentation.

### Unit Testing ###
### Unit Testing

Before running unit tests on Keyman Engine for Web, first run `./build.sh`
according to the instructions above.
Expand All @@ -45,13 +50,14 @@ on your local machine in-browser. Alternatively, see `test.sh`, which
the former command executes.

### Debugging Unit Tests

1. During development, to run a specific unit test, change the `it` to
`it.only`. You can also run all tests under a specific group with
`describe.only`.
2. From this directory, run `./test.sh --debug`. Alternatively, from
`web/` or any `web/` subdirectory,

```
```bash
npm run test -- --debug
```

Expand All @@ -68,69 +74,84 @@ the former command executes.
### Approximate Overall Design

```mermaid
---
title: Dependency Graph
---
%% For rendering, use e.g. https://mermaid.live
%%{init: {"flowchart": {"htmlLabels": false}} }%%
graph TD;
OSK[web/src/engine/osk];
KP["common/web/keyboard-processor"];
IP["common/web/input-processor"];
OSK["/web/src/engine/osk"];
KP["@keymanapp/keyboard-processor<br>(/common/web/keyboard-processor)"];
IP["@keymanapp/input-processor<br>(/common/web/input-processor)"];
OSK-->KP;
IP-->KP;
Utils["common/web/utils"];
KP---->Utils;
Wordbreakers["common/models/wordbreakers"];
Models["common/models/templates"];
Models-->Utils;
LMWorker["common/web/lm-worker"];
WebUtils["@keymanapp/web-utils<br>(/web/src/engine/common/utils)"];
KP---->WebUtils;
Wordbreakers["@keymanapp/models-wordbreakers<br>(/common/models/wordbreakers)"];
Models["@keymanapp/models-templates<br>(/common/models/templates)"];
Models-->WebUtils;
LMWorker["@keymanapp/lm-worker<br>(/common/web/lm-worker)"];
LMWorker-->Models;
LMWorker-->Wordbreakers;
LMLayer["common/predictive-text"];
LMLayer["@keymanapp/lexical-model-layer<br>(/common/predictive-text)"];
LMLayer-->LMWorker;
IP-->LMLayer;
Gestures["@keymanapp/gesture-recognizer<br>(/common/web/gesture-recognizer)"];
Gestures-->WebUtils;

subgraph PredText["WebWorker + its interface"]
subgraph PredText["PredText: WebWorker + its interface"]
LMLayer;
LMWorker;
Models;
Wordbreakers;
end

subgraph Headless["Fully headless components"]
subgraph Headless["`**Headless**
Fully headless components`"]
direction LR
KP;
IP;
Utils;
WebUtils;
PredText;
Gestures;
end

subgraph ClassicWeb["Previously unmodularized components"]
Device[web/src/engine/device-detect];
Device----->Utils;
Elements[web/src/engine/element-wrappers];
subgraph ClassicWeb["`**ClassicWeb**
Previously unmodularized components`"]
Device["/web/src/engine/device-detect"];
Device----->WebUtils;
Elements["/web/src/engine/element-wrappers"];
Elements-->KP;
KeyboardCache[web/src/engine/package-cache];
KeyboardCache["/web/src/engine/package-cache"];
KeyboardCache-->IP;
DomUtils[web/src/engine/dom-utils];
DomUtils-->Utils;
KeyboardCache-->Configuration;
DomUtils["/web/src/engine/dom-utils"];
DomUtils-->WebUtils;
DomUtils-->KP;
OSK-->DomUtils;
OSK---->IP;
Configuration[web/src/engine/paths];
OSK-->Gestures;
Configuration["/web/src/engine/paths"];
Configuration-->OSK;
CommonEngine[web/src/engine/main];
CommonEngine["/web/src/engine/main"];
CommonEngine-->Configuration;
CommonEngine-->Device;
CommonEngine-->KeyboardCache;
CommonEngine-->OSK;
Attachment[web/src/engine/attachment];
CommonEngine-->IP;
Attachment["/web/src/engine/attachment"];
Attachment-->DomUtils;
Attachment-->Elements;
end

subgraph WebEngine["Keyman Engine for Web (top-level libraries)"]
Browser[web/src/app/browser];
WebView[web/src/app/webview];
subgraph WebEngine["`**WebEngine**
Keyman Engine for Web (top-level libraries)`"]
Browser["/web/src/app/browser"];
WebView["/web/src/app/webview"];

WebView--->CommonEngine;

Browser--->CommonEngine;
Browser-->Attachment;
end
```
```
2 changes: 2 additions & 0 deletions web/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ builder_describe "Builds engine modules for Keyman Engine for Web (KMW)." \
":app/webview A puppetable version of KMW designed for use in a host app's WebView" \
":app/ui Builds KMW's desktop form-factor keyboard-selection UI modules" \
":engine/attachment Subset used for detecting valid page contexts for use in text editing " \
":engine/common/utils Web-oriented utility function module. " \
":engine/device-detect Subset used for device-detection " \
":engine/dom-utils A common subset of function used for DOM calculations, layout, etc" \
":engine/events Specialized classes utilized to support KMW API events" \
Expand Down Expand Up @@ -53,6 +54,7 @@ builder_describe_outputs \
build:app/webview "/web/build/app/webview/${config}/keymanweb-webview.js" \
build:app/ui "/web/build/app/ui/${config}/kmwuitoggle.js" \
build:engine/attachment "/web/build/engine/attachment/lib/index.mjs" \
build:engine/common/utils "/web/build/engine/common/utils/lib/index.mjs" \
build:engine/device-detect "/web/build/engine/device-detect/lib/index.mjs" \
build:engine/dom-utils "/web/build/engine/dom-utils/obj/index.js" \
build:engine/events "/web/build/engine/events/lib/index.mjs" \
Expand Down
6 changes: 5 additions & 1 deletion web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@
"types": "./build/engine/attachment/obj/index.d.ts",
"import": "./build/engine/attachment/obj/index.js"
},
"./engine/common/utils": {
"es6-bundling": "./src/engine/common/utils/src/index.ts",
"types": "./build/engine/common/utils/obj/index.d.ts",
"import": "./build/engine/common/utils/obj/index.js"
},
"./engine/paths": {
"es6-bundling": "./src/engine/paths/src/index.ts",
"types": "./build/engine/paths/obj/index.d.ts",
Expand Down Expand Up @@ -105,7 +110,6 @@
"@keymanapp/lexical-model-layer": "*",
"@keymanapp/models-types": "*",
"@keymanapp/recorder-core": "*",
"@keymanapp/web-utils": "*",
"core-js": "^3.34.0",
"eventemitter3": "^5.0.0",
"tslib": "^2.5.2"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
],
"exclude-after-remap": true,
"reporter": ["text", "text-summary"],
"reports-dir": "build/coverage",
"reports-dir": "../../../../build/coverage",
"src": [
"src/"
]
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,17 @@
## START STANDARD BUILD SCRIPT INCLUDE
# adjust relative paths as necessary
THIS_SCRIPT="$(readlink -f "${BASH_SOURCE[0]}")"
. "${THIS_SCRIPT%/*}/../../../resources/build/builder.inc.sh"
. "${THIS_SCRIPT%/*}/../../../../../resources/build/builder.inc.sh"
## END STANDARD BUILD SCRIPT INCLUDE

. "$KEYMAN_ROOT/resources/shellHelperFunctions.sh"
# shellcheck disable=SC2154
. "${KEYMAN_ROOT}/resources/shellHelperFunctions.sh"

BUNDLE_CMD="node $KEYMAN_ROOT/common/web/es-bundling/build/common-bundle.mjs"
BUNDLE_CMD="node ${KEYMAN_ROOT}/common/web/es-bundling/build/common-bundle.mjs"

################################ Main script ################################

# shellcheck disable=SC2154
builder_describe \
"Compiles the web-oriented utility function module." \
"@/common/web/keyman-version" \
Expand All @@ -23,21 +25,22 @@ builder_describe \

builder_describe_outputs \
configure "/node_modules" \
build "/common/web/utils/build/obj/index.js"
build "/web/src/engine/common/utils/build/lib/index.mjs"

builder_parse "$@"

function do_build() {
tsc --build "$THIS_SCRIPT_PATH/tsconfig.json"
# shellcheck disable=SC2154
tsc --build "${THIS_SCRIPT_PATH}/tsconfig.json"

# May be useful one day, for building a mass .d.ts for KMW as a whole.
# So... tsc does declaration-bundling on its own pretty well, at least for local development.
tsc --emitDeclarationOnly --outFile ./build/lib/index.d.ts
tsc --emitDeclarationOnly --outFile "${KEYMAN_ROOT}/web/src/engine/common/utils/build/lib/index.d.ts"

# One of the functions (timedPromise) is quite helpful for automated testing, even in the DOM.
# So, to make sure it's easily-accessible for the DOM-based tests...
$BUNDLE_CMD "${KEYMAN_ROOT}/common/web/utils/build/obj/index.js" \
--out "${KEYMAN_ROOT}/common/web/utils/build/lib/index.mjs" \
${BUNDLE_CMD} "${KEYMAN_ROOT}/web/src/engine/common/utils/build/obj/index.js" \
--out "${KEYMAN_ROOT}/web/src/engine/common/utils/build/lib/index.mjs" \
--format esm
}

Expand All @@ -47,10 +50,11 @@ function do_test() {
local FLAGS=
if builder_has_option --ci; then
echo "Replacing user-friendly test reports with CI-friendly versions."
FLAGS="$FLAGS --reporter mocha-teamcity-reporter"
FLAGS="${FLAGS} --reporter mocha-teamcity-reporter"
fi

c8 mocha --recursive $FLAGS ./src/test/
# shellcheck disable=SC2086
c8 mocha --recursive ${FLAGS} ./src/test/
}

builder_run_action configure verify_npm_setup
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"extends": "../tsconfig.kmw-main-base.json",
"extends": "../../../../tsconfig.base.json",
"compilerOptions": {
"tsBuildInfoFile": "./build/obj/tsconfig.tsbuildinfo",
"baseUrl": "./",
Expand Down
Loading
Loading