From 089c7bbb242f2b548b88e3a7f99b2bf3b3f00091 Mon Sep 17 00:00:00 2001 From: "Steven R. Loomis" Date: Tue, 26 Sep 2023 19:20:04 -0500 Subject: [PATCH 1/5] CLDR-16943 kbd: Normalization - remove , add - dtd and spec update Co-authored-by: Marc Durdin --- docs/ldml/tr35-keyboards.md | 35 +++++++++++++++++++++------------ keyboards/dtd/ldmlKeyboard3.dtd | 7 ++++--- keyboards/dtd/ldmlKeyboard3.xsd | 6 +++--- 3 files changed, 29 insertions(+), 19 deletions(-) diff --git a/docs/ldml/tr35-keyboards.md b/docs/ldml/tr35-keyboards.md index 94ebc21180d..990252b887a 100644 --- a/docs/ldml/tr35-keyboards.md +++ b/docs/ldml/tr35-keyboards.md @@ -486,7 +486,6 @@ Element containing informative properties about the layout, for displaying in us ```xml ``` @@ -505,16 +504,6 @@ _Attribute:_ `author` > The `author` attribute contains the name of the author of the layout file. -_Attribute:_ `normalization` - -> The `normalization` attribute describes the intended normalization form of the keyboard layout output. The valid values are `NFC`, `NFD` or `other`. -> -> An example use case is aiding a user to choose among the two same layouts with one outputting characters in the normalization form C and one in the normalization form D. -> -> All keyboards in the CLDR repository will be in `NFC` or `NFD` forms. However, users and implementations may produce and consume other normalization forms or mixed output, use the `other` value to indicate this case. -> -> When using `NFC` or `NFD`, tooling should verify that all possible keystrokes, gestures, and transforms on the keyboard only produce the specified normalization form, producing warnings if not. - _Attribute:_ `layout` > The `layout` attribute describes the layout pattern, such as QWERTY, DVORAK, INSCRIPT, etc. typically used to distinguish various layouts for the same language. @@ -596,7 +585,7 @@ An element used to keep track of layout specific settings. This element may or m **Syntax** ```xml - + ``` > @@ -632,6 +621,26 @@ Indicates that: 2. If a transform is escaped, output the contents of the buffer. 3. During a transform, hide the contents of the buffer as the user is typing. +_Attribute:_ `normalization` + +> Normalization will not typically be the responsibility of the keyboard author, rather this will be managed by the implementation. +> Output from the keyboard, following application of all transform rules, will be normalized to implementation or application-requested form. +> +> However, it is recognized that there may be some keyboards which, for compatibility or legacy reasons, need to manage their own normalization. The implementation in that case will do no normalization at all. The keyboard author must make use of transforms in the keyboard to any required normalization. +> +> **Note**: while this attribute is allowed by the specification, its use is discouraged, and keyboards with `normalization="mixed"` would not be accepted into the ClDR repository. + + +**Example** + +```xml + + … + + … + +``` + * * * ### Element: keys @@ -957,7 +966,7 @@ split up their keyboard into multiple files, as the intent is for each single XM `` can be used as a child of a number of elements (see the _Parents_ section immediately below). Multiple `` elements may be used, however, `` elements must come before any other sibling elements. If two identical elements are defined, the later element will take precedence, that is, override. -**Note:** imported files do not have any indication of their normalization mode. For this reason, the keyboard author must verify that the imported file is of a compatible normalization mode. See the [`info` element](#Element_info) for further details. +**Note:** imported files do not have any indication of their normalization mode. For this reason, the keyboard author must verify that the imported file is of a compatible normalization mode. See the [`settings` element](#element-settings) for further details. **Syntax** ```xml diff --git a/keyboards/dtd/ldmlKeyboard3.dtd b/keyboards/dtd/ldmlKeyboard3.dtd index 25d03368f86..dc5f773320b 100644 --- a/keyboards/dtd/ldmlKeyboard3.dtd +++ b/keyboards/dtd/ldmlKeyboard3.dtd @@ -48,9 +48,6 @@ Please view the subcommittee page for the most recent information. - - - @@ -76,6 +73,10 @@ Please view the subcommittee page for the most recent information. + + + + diff --git a/keyboards/dtd/ldmlKeyboard3.xsd b/keyboards/dtd/ldmlKeyboard3.xsd index f3a7b8bed8e..b8ef6d00577 100644 --- a/keyboards/dtd/ldmlKeyboard3.xsd +++ b/keyboards/dtd/ldmlKeyboard3.xsd @@ -99,7 +99,6 @@ Note: DTD @-annotations are not currently converted to .xsd. For full CLDR file - @@ -111,8 +110,6 @@ Note: DTD @-annotations are not currently converted to .xsd. For full CLDR file - - @@ -143,11 +140,14 @@ Note: DTD @-annotations are not currently converted to .xsd. For full CLDR file + + + From f68c2a40e3048341fb35c84014b9ca9178c1458d Mon Sep 17 00:00:00 2001 From: "Steven R. Loomis" Date: Wed, 27 Sep 2023 10:56:03 -0500 Subject: [PATCH 2/5] CLDR-16943 kbd: Normalization - update xml --- keyboards/3.0/fr-t-k0-azerty.xml | 2 +- keyboards/3.0/mt-t-k0-47key.xml | 2 +- keyboards/3.0/mt.xml | 2 +- .../unicode/cldr/tool/KeyboardFlatten/broken-import-missing.xml | 2 +- .../cldr/tool/KeyboardFlatten/broken-import-unknownbase.xml | 2 +- .../cldr/tool/KeyboardFlatten/broken-import-unknownver.xml | 2 +- .../cldr/tool/KeyboardFlatten/broken-import-wrongparent.xml | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/keyboards/3.0/fr-t-k0-azerty.xml b/keyboards/3.0/fr-t-k0-azerty.xml index 4ccedc062f3..eb35a9c811a 100644 --- a/keyboards/3.0/fr-t-k0-azerty.xml +++ b/keyboards/3.0/fr-t-k0-azerty.xml @@ -16,7 +16,7 @@ keyboard formats --> - + diff --git a/keyboards/3.0/mt-t-k0-47key.xml b/keyboards/3.0/mt-t-k0-47key.xml index 02db33accc6..11ca8cda824 100644 --- a/keyboards/3.0/mt-t-k0-47key.xml +++ b/keyboards/3.0/mt-t-k0-47key.xml @@ -5,7 +5,7 @@ - + diff --git a/keyboards/3.0/mt.xml b/keyboards/3.0/mt.xml index a5a415d8745..4e5fbfca37e 100644 --- a/keyboards/3.0/mt.xml +++ b/keyboards/3.0/mt.xml @@ -11,7 +11,7 @@ - + diff --git a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-missing.xml b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-missing.xml index 803de54ae7a..f706ab3fcd1 100644 --- a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-missing.xml +++ b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-missing.xml @@ -8,7 +8,7 @@ - + diff --git a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownbase.xml b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownbase.xml index 334cceee9b2..65e0307e8d9 100644 --- a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownbase.xml +++ b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownbase.xml @@ -8,7 +8,7 @@ - + diff --git a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownver.xml b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownver.xml index 8907032203c..4b9bae247b2 100644 --- a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownver.xml +++ b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-unknownver.xml @@ -8,7 +8,7 @@ - + diff --git a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-wrongparent.xml b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-wrongparent.xml index c5a63cbae0c..730ff719c96 100644 --- a/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-wrongparent.xml +++ b/tools/cldr-code/src/test/resources/org/unicode/cldr/tool/KeyboardFlatten/broken-import-wrongparent.xml @@ -8,7 +8,7 @@ - + From 331b2c0d5b6a745493b5a3571a88408ab078ebdf Mon Sep 17 00:00:00 2001 From: "Steven R. Loomis" Date: Wed, 27 Sep 2023 10:58:37 -0500 Subject: [PATCH 3/5] CLDR-16943 kbd: Normalization - expand about matching --- docs/ldml/tr35-keyboards.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/ldml/tr35-keyboards.md b/docs/ldml/tr35-keyboards.md index 990252b887a..feae4056aa9 100644 --- a/docs/ldml/tr35-keyboards.md +++ b/docs/ldml/tr35-keyboards.md @@ -624,6 +624,7 @@ Indicates that: _Attribute:_ `normalization` > Normalization will not typically be the responsibility of the keyboard author, rather this will be managed by the implementation. +> The implementation will apply normalization as appropriate when matching transform rules and `` value matching. > Output from the keyboard, following application of all transform rules, will be normalized to implementation or application-requested form. > > However, it is recognized that there may be some keyboards which, for compatibility or legacy reasons, need to manage their own normalization. The implementation in that case will do no normalization at all. The keyboard author must make use of transforms in the keyboard to any required normalization. From 4b72d72658be2ee57da09c6ab433dca79b401711 Mon Sep 17 00:00:00 2001 From: "Steven R. Loomis" Date: Tue, 3 Oct 2023 10:51:13 -0500 Subject: [PATCH 4/5] CLDR-16943 kbd normalization=disabled - previously 'mixed' --- docs/ldml/tr35-keyboards.md | 8 +++++--- keyboards/dtd/ldmlKeyboard3.dtd | 3 +-- keyboards/dtd/ldmlKeyboard3.xsd | 8 +++++++- 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/ldml/tr35-keyboards.md b/docs/ldml/tr35-keyboards.md index 58b0c837127..04cca495cec 100644 --- a/docs/ldml/tr35-keyboards.md +++ b/docs/ldml/tr35-keyboards.md @@ -589,7 +589,7 @@ An element used to keep track of layout-specific settings by implementations. Th **Syntax** ```xml - + ``` > @@ -632,7 +632,9 @@ _Attribute:_ `normalization` > > However, it is recognized that there may be some keyboards which, for compatibility or legacy reasons, need to manage their own normalization. The implementation in that case will do no normalization at all. The keyboard author must make use of transforms in the keyboard to any required normalization. > -> **Note**: while this attribute is allowed by the specification, its use is discouraged, and keyboards with `normalization="mixed"` would not be accepted into the ClDR repository. +> In this case, the attribute `normalization="disabled"` can be used to specify that normalization is not done. +> +> **Note**: while this attribute is allowed by the specification, its use is discouraged, and keyboards with `normalization="disabled"` would not be accepted into the ClDR repository. **Example** @@ -640,7 +642,7 @@ _Attribute:_ `normalization` ```xml … - + … ``` diff --git a/keyboards/dtd/ldmlKeyboard3.dtd b/keyboards/dtd/ldmlKeyboard3.dtd index bbcb2fe4ad6..2cfe1513781 100644 --- a/keyboards/dtd/ldmlKeyboard3.dtd +++ b/keyboards/dtd/ldmlKeyboard3.dtd @@ -73,8 +73,7 @@ Please view the subcommittee page for the most recent information. - - + diff --git a/keyboards/dtd/ldmlKeyboard3.xsd b/keyboards/dtd/ldmlKeyboard3.xsd index 1de30c7e450..6c2adc6c784 100644 --- a/keyboards/dtd/ldmlKeyboard3.xsd +++ b/keyboards/dtd/ldmlKeyboard3.xsd @@ -139,7 +139,13 @@ Note: DTD @-annotations are not currently converted to .xsd. For full CLDR file - + + + + + + + From b6c5d2bfd5a1c3479cab3d52dc6061aec7a36695 Mon Sep 17 00:00:00 2001 From: "Steven R. Loomis" Date: Tue, 3 Oct 2023 16:36:00 -0500 Subject: [PATCH 5/5] CLDR-16943 kbd: normalization bis --- docs/ldml/tr35-keyboards.md | 6 ++---- keyboards/dtd/ldmlKeyboard3.xsd | 1 - 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/ldml/tr35-keyboards.md b/docs/ldml/tr35-keyboards.md index 04cca495cec..6cd93351509 100644 --- a/docs/ldml/tr35-keyboards.md +++ b/docs/ldml/tr35-keyboards.md @@ -606,7 +606,7 @@ _Attribute:_ `fallback="omit"` > The presence of this attribute means that when a modifier key combination goes unmatched, no output is produced. The default behavior (when this attribute is not present) is to fall back to the base map when the modifier key combination goes unmatched. -If this attribute is present, it must have a value of omit. +If this attribute is present, it must have a value of `omit`. **Example** @@ -630,9 +630,7 @@ _Attribute:_ `normalization` > The implementation will apply normalization as appropriate when matching transform rules and `` value matching. > Output from the keyboard, following application of all transform rules, will be normalized to implementation or application-requested form. > -> However, it is recognized that there may be some keyboards which, for compatibility or legacy reasons, need to manage their own normalization. The implementation in that case will do no normalization at all. The keyboard author must make use of transforms in the keyboard to any required normalization. -> -> In this case, the attribute `normalization="disabled"` can be used to specify that normalization is not done. +> However, it is recognized that there may be some keyboards which, for compatibility or legacy reasons, need to manage their own normalization. The implementation in that case will do no normalization at all. The keyboard author must make use of transforms in the keyboard to any required normalization. In this case, the attribute value `normalization="disabled"` is used to indicate that no automatic normalization happens. > > **Note**: while this attribute is allowed by the specification, its use is discouraged, and keyboards with `normalization="disabled"` would not be accepted into the ClDR repository. diff --git a/keyboards/dtd/ldmlKeyboard3.xsd b/keyboards/dtd/ldmlKeyboard3.xsd index 6c2adc6c784..1a772458658 100644 --- a/keyboards/dtd/ldmlKeyboard3.xsd +++ b/keyboards/dtd/ldmlKeyboard3.xsd @@ -152,7 +152,6 @@ Note: DTD @-annotations are not currently converted to .xsd. For full CLDR file -