SSS: Improve types for validation (#2002)

## Summary:

This PR begins working out the types for Scoring, Validation, and how they relate to our full widget options types. There are currently three "trees" of types that are shaped as a map of `widgetId` to something. They are: 

  * Full widget options (starting from `PerseusRenderer`)
  * Scoring data - used to score the learner's guess (user input)
  * Validation data - a shared subset (of Render and Scoring data) used to do empty widget checking (aka validation). This helps the frontend to know if the question is scorable yet. 

Finally, there is also a widget map known as User Input. This map is a map of widget ids from the item to the user input the learner has entered so far. 

Issue: LEMS-2561

## Test plan:

`yarn typecheck` (especially, the new `validation.typetest.ts` file!)
`yarn test` (just to be sure)

Author: jeremywiebe

Reviewers: jeremywiebe, Myranae, handeyeco

Required Reviewers:

Approved By: Myranae

Checks: ✅ Publish npm snapshot (ubuntu-latest, 20.x), ✅ Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Cypress (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ Publish Storybook to Chromatic (ubuntu-latest, 20.x), ✅ Check builds for changes in size (ubuntu-latest, 20.x)

Pull Request URL: #2002

.changeset/few-rings-cover.md

@@ -0,0 +1,5 @@
+---
+"@khanacademy/perseus": minor
+---
+
+Add and improve types for scoring and validation

docs/architecture.md

@@ -11,10 +11,10 @@ base Markdown syntax:
 
   1. Widgets - Perseus can render custom widgets (in the form of React
      components) which conform to a special API that enables the user to
-     interact with the widget and for the widget to check taht input for
-     correctness against a rubric. Widgets are denoted using the following
-     Markdown syntax: `[[☃️ widget-id ]]` (where `widget-id` represents a
-     generated ID that is unique within the Perseus instance.
+     interact with the widget and for the widget to check that input for
+     correctness against a set of scoring data. Widgets are denoted using the
+     following Markdown syntax: `[[☃️ widget-id ]]` (where `widget-id`
+     represents a generated ID that is unique within the Perseus instance.
   1. Math - Perseus can also render beautiful math using MathJax. Math is
      denoted using an opening and close dollar sign (eg. `$y = mx + b$`).
 
@@ -181,15 +181,15 @@ the widgets options type (ie. the type `T` wrapped in `WidgetOptions<T>` from
 In a few rare cases, this type is defined as the sum of RenderProps wrapped in
 `WidgetOptions`.
 
-### `Rubric`
+### `Scoring Data`
 
 This type defines the data that the scoring function needs in order to score
 the learner's guess (aka user input).
 
 ### `Props`
 
 Finally, `Props` form the entire set of props that widget's component supports.
-Typically it is defined as `type Props = WidgetProps<RenderProps, Rubric>`. In
+Typically it is defined as `type Props = WidgetProps<RenderProps, ScoringData>`. In
 cases where there are `RenderProps` that are optional that are provided via
 `DefaultProps`, this `Props` type "redefines" these props as `myProp:
 NonNullable<ExternalProps["myProps"]>;`.

packages/perseus/src/__tests__/validation.typetest.ts

@@ -0,0 +1,25 @@
+/**
+ * This file contains TypeScript type "tests" which ensure that types needed
+ * for scoring and validation stay in sync with other types in the system.
+ *
+ * If you make a change and `Extends<>` starts to complain, that will usually
+ * mean you've made a change that will cause runtime breakages in scoring or
+ * validation. ie. The types that should be compatible are no longer
+ * compatible. Read the TypeScript error message closely and it should point
+ * you in the right direction.
+ */
+import type {PerseusRenderer} from "../perseus-types";
+import type {ScoringDataMap, ValidationDataMap} from "../validation.types";
+
+/**
+ * An utility type that verifies that the given type `E` extends the type `T`.
+ * This is useful for asserting that one type remains a compatible subset of
+ * the other.
+ */
+type Extends<T, E extends T> = (T) => E;
+
+// We can use a 'widgets' map from a PerseusRenderer as a ValidationDataMap
+type _ = Extends<ValidationDataMap, PerseusRenderer["widgets"]>;
+
+// We can use a ScoringDataMap as a ValidationDataMap
+type __ = Extends<ValidationDataMap, ScoringDataMap>;

packages/perseus/src/perseus-types.ts

@@ -37,6 +37,50 @@ export type Size = [width: number, height: number];
 export type CollinearTuple = [Vector2, Vector2];
 export type ShowSolutions = "all" | "selected" | "none";
 
+/**
+ * A utility type that constructs a widget map from a "registry interface".
+ * The keys of the registry should be the widget type (aka, "categorizer" or
+ * "radio", etc) and the value should be the option type stored in the value
+ * of the map.
+ *
+ * You can think of this as a type that generates another type. We use
+ * "registry interfaces" as a way to keep a set of widget types to their data
+ * type in several places in Perseus. This type then allows us to generate a
+ * map type that maps a widget id to its data type and keep strong typing by
+ * widget id.
+ *
+ * For example, given a fictitious registry such as this:
+ *
+ * ```
+ * interface DummyRegistry {
+ *     categorizer: { categories: ReadonlyArray<string> };
+ *     dropdown: { choices: ReadonlyArray<string> }:
+ * }
+ * ```
+ *
+ * If we create a DummyMap using this helper:
+ *
+ * ```
+ * type DummyMap = MakeWidgetMap<DummyRegistry>;
+ * ```
+ *
+ * We'll get a map that looks like this:
+ *
+ * ```
+ * type DummyMap = {
+ *     `categorizer ${number}`: { categories: ReadonlyArray<string> };
+ *     `dropdown ${number}`: { choices: ReadonlyArray<string> };
+ * }
+ * ```
+ *
+ * We use interfaces for the registries so that they can be extended in cases
+ * where the consuming app brings along their own widgets. Interfaces in
+ * TypeScript are always open (ie. you can extend them) whereas types aren't.
+ */
+export type MakeWidgetMap<TRegistry> = {
+    [Property in keyof TRegistry as `${Property & string} ${number}`]: TRegistry[Property];
+};
+
 /**
  * Our core set of Perseus widgets.
  *
@@ -131,9 +175,7 @@ export interface PerseusWidgetTypes {
  * @see {@link PerseusWidgetTypes} additional widgets can be added to this map type
  * by augmenting the PerseusWidgetTypes with new widget types!
  */
-export type PerseusWidgetsMap = {
-    [Property in keyof PerseusWidgetTypes as `${Property} ${number}`]: PerseusWidgetTypes[Property];
-};
+export type PerseusWidgetsMap = MakeWidgetMap<PerseusWidgetTypes>;
 
 /**
  * A "PerseusItem" is a classic Perseus item. It is rendered by the

packages/perseus/src/validation.types.ts

@@ -5,19 +5,20 @@
  *
  * These types are:
  *
- * `Perseus<Widget>UserInput`: the data returned by the widget that the user
- * entered. This is referred to as the 'guess' in some older parts of Perseus.
+ *   * `Perseus<Widget>UserInput`: the data from the widget that represents the
+ *     data the user entered. This is referred to as the 'guess' in some older
+ *     parts of Perseus.
  *
- * `Perseus<Widget>ValidationData`: the data needed to do validation of the
- * user input. Validation refers to the different checks that we can do both on
- * the client-side (before submitting user input for scoring) and on the
- * server-side (when we score it). As such, it cannot contain any of the
- * sensitive scoring data that would reveal the answer.
+ *   * `Perseus<Widget>ValidationData`: the data needed to do validation of the
+ *     user input. Validation refers to the different checks that we can do
+ *     both on the client-side (before submitting user input for scoring) and
+ *     on the server-side (when we score it). As such, it cannot contain any of
+ *     the sensitive scoring data that would reveal the answer.
  *
- * `Perseus<Widget>ScoringData` (nee `Perseus<Widget>Rubric`): the data needed
- * to score the user input. By convention, this type is defined as the set of
- * sensitive answer data and then intersected with
- * `Perseus<Widget>ValidationData`.
+ *   * `Perseus<Widget>ScoringData` (nee `Perseus<Widget>Rubric`): the data
+ *     needed to score the user input. By convention, this type is defined as
+ *     the set of sensitive answer data and then intersected with
+ *     `Perseus<Widget>ValidationData`.
  *
  * For example:
  * ```
@@ -41,6 +42,7 @@ import type {
     PerseusOrdererWidgetOptions,
     PerseusRadioChoice,
     PerseusGraphCorrectType,
+    MakeWidgetMap,
 } from "./perseus-types";
 import type {Relationship} from "./widgets/number-line/number-line";
 
@@ -237,93 +239,95 @@ export type PerseusTableScoringData = {
 
 export type PerseusTableUserInput = ReadonlyArray<ReadonlyArray<string>>;
 
-export type ScoringData =
-    | PerseusCategorizerScoringData
-    | PerseusDropdownScoringData
-    | PerseusExpressionScoringData
-    | PerseusGroupScoringData
-    | PerseusGradedGroupScoringData
-    | PerseusGradedGroupSetScoringData
-    | PerseusGrapherScoringData
-    | PerseusInputNumberScoringData
-    | PerseusInteractiveGraphScoringData
-    | PerseusLabelImageScoringData
-    | PerseusMatcherScoringData
-    | PerseusMatrixScoringData
-    | PerseusNumberLineScoringData
-    | PerseusNumericInputScoringData
-    | PerseusOrdererScoringData
-    | PerseusPlotterScoringData
-    | PerseusRadioScoringData
-    | PerseusSorterScoringData
-    | PerseusTableScoringData;
-
-export type UserInput =
-    | PerseusCategorizerUserInput
-    | PerseusCSProgramUserInput
-    | PerseusDropdownUserInput
-    | PerseusExpressionUserInput
-    | PerseusGrapherUserInput
-    | PerseusGroupUserInput
-    | PerseusIFrameUserInput
-    | PerseusInputNumberUserInput
-    | PerseusInteractiveGraphUserInput
-    | PerseusLabelImageUserInput
-    | PerseusMatcherUserInput
-    | PerseusMatrixUserInput
-    | PerseusNumberLineUserInput
-    | PerseusNumericInputUserInput
-    | PerseusOrdererUserInput
-    | PerseusPlotterUserInput
-    | PerseusRadioUserInput
-    | PerseusSorterUserInput
-    | PerseusTableUserInput;
-
-export type UserInputMap = {[widgetId: string]: UserInput};
+export interface ScoringDataRegistry {
+    categorizer: PerseusCategorizerScoringData;
+    dropdown: PerseusDropdownScoringData;
+    expression: PerseusExpressionScoringData;
+    grapher: PerseusGrapherScoringData;
+    "graded-group-set": PerseusGradedGroupSetScoringData;
+    "graded-group": PerseusGradedGroupScoringData;
+    group: PerseusGroupScoringData;
+    image: PerseusLabelImageScoringData;
+    "input-number": PerseusInputNumberScoringData;
+    "interactive-graph": PerseusInteractiveGraphScoringData;
+    "label-image": PerseusLabelImageScoringData;
+    matcher: PerseusMatcherScoringData;
+    matrix: PerseusMatrixScoringData;
+    "number-line": PerseusNumberLineScoringData;
+    "numeric-input": PerseusNumericInputScoringData;
+    orderer: PerseusOrdererScoringData;
+    plotter: PerseusPlotterScoringData;
+    radio: PerseusRadioScoringData;
+    sorter: PerseusSorterScoringData;
+    table: PerseusTableScoringData;
+}
+
+/**
+ * A map of scoring data (previously referred to as "rubric"), keyed by
+ * `widgetId`. This data is used to score a learner's guess for a PerseusItem.
+ *
+ * NOTE:  The value in this map is intentionally a subset of WidgetOptions<T>.
+ * By using the same shape (minus any unneeded render data), we are able to
+ * share functionality that understands how to traverse maps of `widget id` to
+ * `options`.
+ */
+export type ScoringDataMap = {
+    [Property in keyof ScoringDataRegistry as `${Property} ${number}`]: {
+        type: Property;
+        static?: boolean;
+        options: ScoringDataRegistry[Property];
+    };
+};
+
+export type ScoringData = ScoringDataRegistry[keyof ScoringDataRegistry];
+
+/**
+ * This is an interface so that it can be extended if a widget is created
+ * outside of this Perseus package. See `PerseusWidgetTypes` for a full
+ * explanation.
+ */
+interface UserInputRegistry {
+    categorizer: PerseusCategorizerUserInput;
+    "cs-program": PerseusCSProgramUserInput;
+    dropdown: PerseusDropdownUserInput;
+    expression: PerseusExpressionUserInput;
+    grapher: PerseusGrapherUserInput;
+    group: PerseusGroupUserInput;
+    iframe: PerseusIFrameUserInput;
+    "input-number": PerseusInputNumberUserInput;
+    "interactive-graph": PerseusInteractiveGraphUserInput;
+    "label-image": PerseusLabelImageUserInput;
+    matcher: PerseusMatcherUserInput;
+    matrix: PerseusMatrixUserInput;
+    "number-line": PerseusNumberLineUserInput;
+    "numeric-input": PerseusNumericInputUserInput;
+    orderer: PerseusOrdererUserInput;
+    plotter: PerseusPlotterUserInput;
+    radio: PerseusRadioUserInput;
+    sorter: PerseusSorterUserInput;
+    table: PerseusTableUserInput;
+}
+
+/** A union type of all the widget user input types */
+export type UserInput = UserInputRegistry[keyof UserInputRegistry];
+
+/**
+ * A map of widget IDs to user input types (strongly typed based on the format
+ * of the widget ID).
+ */
+export type UserInputMap = MakeWidgetMap<UserInputRegistry>;
 
 /**
  * deprecated prefer using UserInputMap
  */
 export type UserInputArray = ReadonlyArray<
     UserInputArray | UserInput | null | undefined
 >;
+
 export interface ValidationDataTypes {
     categorizer: PerseusCategorizerValidationData;
-    // "cs-program": PerseusCSProgramValidationData;
-    // definition: PerseusDefinitionValidationData;
-    // dropdown: PerseusDropdownValidationData;
-    // explanation: PerseusExplanationValidationData;
-    // expression: PerseusExpressionValidationData;
-    // grapher: PerseusGrapherValidationData;
-    // "graded-group-set": PerseusGradedGroupSetValidationData;
-    // "graded-group": PerseusGradedGroupValidationData;
     group: PerseusGroupValidationData;
-    // iframe: PerseusIFrameValidationData;
-    // image: PerseusImageValidationData;
-    // "input-number": PerseusInputNumberValidationData;
-    // interaction: PerseusInteractionValidationData;
-    // "interactive-graph": PerseusInteractiveGraphValidationData;
-    // "label-image": PerseusLabelImageValidationData;
-    // matcher: PerseusMatcherValidationData;
-    // matrix: PerseusMatrixValidationData;
-    // measurer: PerseusMeasurerValidationData;
-    // "molecule-renderer": PerseusMoleculeRendererValidationData;
-    // "number-line": PerseusNumberLineValidationData;
-    // "numeric-input": PerseusNumericInputValidationData;
-    // orderer: PerseusOrdererValidationData;
-    // "passage-ref-target": PerseusRefTargetValidationData;
-    // "passage-ref": PerseusPassageRefValidationData;
-    // passage: PerseusPassageValidationData;
-    // "phet-simulation": PerseusPhetSimulationValidationData;
-    // "python-program": PerseusPythonProgramValidationData;
     plotter: PerseusPlotterValidationData;
-    // radio: PerseusRadioValidationData;
-    // sorter: PerseusSorterValidationData;
-    // table: PerseusTableValidationData;
-    // video: PerseusVideoValidationData;
-
-    // Deprecated widgets
-    // sequence: PerseusAutoCorrectValidationData;
 }
 
 /**
@@ -332,7 +336,7 @@ export interface ValidationDataTypes {
  * data that's available in the client (widget options) and server (scoring
  * data) and is represented by a group of types known as "validation data".
  *
- * NOTE:  The value in this map is intentionally a subset of WidgetOptions<T>.
+ * NOTE: The value in this map is intentionally a subset of WidgetOptions<T>.
  * By using the same shape (minus any unneeded data), we are able to pass a
  * `PerseusWidgetsMap` or ` into any function that accepts a
  * `ValidationDataMap` without any mutation of data.

packages/perseus/src/widgets/group/group.tsx

@@ -23,6 +23,7 @@ import type {
 import type {
     PerseusGroupScoringData,
     UserInputArray,
+    UserInputMap,
 } from "../../validation.types";
 import type {GroupPromptJSON} from "../../widget-ai-utils/group/group-ai-utils";
 
@@ -59,7 +60,7 @@ class Group extends React.Component<Props> implements Widget {
         return Changeable.change.apply(this, args);
     };
 
-    getUserInputMap() {
+    getUserInputMap(): UserInputMap | undefined {
         return this.rendererRef?.getUserInputMap();
     }