controller/keyboard navigation support for <select> #566

Paril · 2024-01-09T12:16:00Z

This is a PR to address #565 as well as some other things I ran into while using <select>.

using spatial navigation/arrow keys will now ensure that the selections are scrolled into view; currently it is impossible to use keyboard/controllers on scrolling selectboxes
the selection is also scrolled into view when the box opens, except anchored to the top closest to the box
KI_ESCAPE can now be used to cancel a selectbox operation - this is currently done very naively, by simply storing the index of the current selection on open and restoring it on KI_ESCAPE if it is set (any change to the options list while it is open will remove the ability to cancel selection). This is a break from HTML standard behaviors, but I needed it for my project on all select boxes and it seemed like something that might be useful for others too (especially for something like a video resolution selector; in games people tend to expect a way to cancel a select box, esp on controller)

(The third point was removed in the PR, but keeping this block below for posterity)

The third 'fix' is messy, and I wonder if diverging from the way browsers handling it is worth looking into:

options do not get selected simply by scrolling the list; this also saves a ton of events on "Change" when the selectbox hasn't actually technically been committed yet
only selecting an item via click or enter should actually close the box and assign the selected element
the select "value" box should always show the current value, not the value focused in the list (to see what would be reverted to if you hit escape)
hitting escape to cancel just closes the box, no operation needs to be done since nothing has been committed yet

this is a bit more than I wanted to shove into a single PR and wanted to discuss it further before I commit to changing any behaviors like that though.

mikke89

Thanks for the pull request!

I think scrolling inside select boxes is one of those things that haven't been tested a whole lot, nice to see it getting some needed attention.

I don't believe it is justified to diverge from the HTML behavior in terms of change events and escape key. I think the current behavior here useful in many situations. So my suggest ion is to remove the value_last_selected logic.

When testing this PR, I am getting quite some quirky behavior: The selection box keeps closing on me. Sometimes just when pressing arrow keys or when selecting an option. I tested this on the demo sample.

Another behavior that I noticed is that the scrollbar keeps flashing on and off as I press arrow keys to select new options. However, I see that this occurs in master too, so it is unrelated to this PR. But maybe you could take a look at that while you're at it? :)

selectbox now scrolls when nav is being used

Can you elaborate on this?

mikke89 · 2024-01-23T22:49:00Z

Source/Core/Elements/WidgetDropDown.cpp

+				SetSelection(GetOption(value_last_selected), true);
+				value_last_selected = -1;
+			}
+			ShowSelectBox(false);


I think hiding the box is perfectly fine here. However, I don't think we should set the selection back. When we change the selected option (with arrow keys or otherwise), we commit it and send change events. I don't think reverting that change is the right thing for escape. And as you mention too, web browser also seems to keep the selected value.

I think this behavior you want is better implemented locally.

I'll remove this from the PR. I kinda wish the change itself was deferred, though, because scrolling through the list on controller emits a ton of change events (they have to "focus" every element to get the active one). This is how browsers handle it, too ("change" isn't emitted until you actually press enter or click on an option), so this is one way we're currently diverging from browsers (the escape thing I can live with, especially if this is implemented).

mikke89 · 2024-01-23T22:56:18Z

Source/Core/Elements/WidgetDropDown.cpp

+		if (selection != -1)
+		{
+			Rml::ScrollIntoViewOptions scrollOptions {
+				box_layout_dirty == DropDownBoxLayoutType::Open ? Rml::ScrollAlignment::Start : Rml::ScrollAlignment::Nearest


Can you elaborate on the box_layout_dirty condition here, why do we need that?

Can you explain the motivation behind using Start? I wonder if using Rml::ScrollAlignment::Nearest always is the more appropriate choice. Especially when scrolling up it feels weird that it keeps selection at the bottom.

I think it's a matter of preference thing, when I was using Nearest it felt awkward to have the selection start at the bottom of the list. Might also depend how you have the box styled or something. Looking at how browsers handle it, though, it seems they are using (the equivalent to) End on open (opening the box with a selection that has a scrolling area always puts it at the bottom of the list) rather than Nearest. I'll have to play more with this.

Nearest is used when scrolling normally, though, so the selection should 'float' like it does in browsers. At least that's the behavior I experienced when testing it locally (it won't scroll until the edges are reached): https://streamable.com/rwiyew

Source/Core/Elements/WidgetDropDown.h

Source/Core/Elements/ElementFormControlSelect.cpp

Paril · 2024-01-24T18:01:02Z

When testing this PR, I am getting quite some quirky behavior: The selection box keeps closing on me. Sometimes just when pressing arrow keys or when selecting an option. I tested this on the demo sample.

Another behavior that I noticed is that the scrollbar keeps flashing on and off as I press arrow keys to select new options. However, I see that this occurs in master too, so it is unrelated to this PR. But maybe you could take a look at that while you're at it? :)

I've not seen either of these locally, oddly enough. If I run into them I'll certainly fix them. I'll check the demo sample to see if I can repro it there.

selectbox now scrolls when nav is being used

Can you elaborate on this?

The actual scroll box moves now if you are using KI_UP/KI_DOWN. Before, the box would just stay where it was, meaning you could not read the option being selected.

Paril · 2024-02-06T22:58:44Z

Converting to draft for now as I won't have time for a while to poke these.

mikke89 · 2024-02-06T23:06:30Z

I understand. I haven't been able to follow up on PRs and issues as much as I'd like lately, so things might have been a bit slow from my side too.

There are some nice improvements here, so I hope you are able to resume at some point. But of course at you own discretion, cheers.

Paril · 2024-02-06T23:22:10Z

Yeah I'm hoping to come back to this in a few weeks - just a lot of projects going on. I shall return, though! (EDIT: also super excited about the Filters/Effects PR seemingly being ready for merging; I've had my eye on it for a while!)

Paril · 2024-07-30T19:24:54Z

Alright, I'm ready for this to get reviewed again. I removed the escape key functionality from this PR, but I still recommend we look into this; I'm using this functionality in our games and it's absolutely vital that escape/Back Button cancels the select box, because otherwise it switches menus which is not the behavior a user will expect from hitting escape/Back with a dropdown open. I can open it as a separate PR and we can figure out the details.

I addressed the feedback that was left unaddressed before. Let me know what you think!

mikke89 · 2024-07-30T19:46:57Z

Thanks for getting back at this one, very much appreciate that! I'll have to hold this one for a bit, right now I need to focus on the upcoming release, which hopefully is not too far out. After that I'll get back to this and the other PRs, cheers.

- selectbox now scrolls when nav is being used - selectbox ensures that the selected option is scrolled into view, anchored to the top - "escape" can be used to cancel selection

Use separate bool for layout_dirty vs box just opened, as these two states are not really mutually exclusive. Fixes a bug where the box was not considered just opened since the "open" state was overwritten by the "switch" state. Use center vertical alignment for a just opened box. This is admittedly a subjective matter. However, it looked a bit weird to me how it would scroll past the first few elements when you select, say, just the second or third element. With center, both the first few, and the last items, are shown when selecting something near the beginning or the end respectively. This also feels more symmetrical when the selection box ends up being placed above the select element.

mikke89 · 2024-10-30T20:03:12Z

Hey, I finally got around to taking a look at this one. I pushed a new commit with some suggested changes here:

Use separate bool for layout_dirty vs box just opened, as these two states are not really mutually exclusive. Fixes a bug where the box was not considered just opened since the "open" state was overwritten by the "switch" state.

Use center vertical alignment for a just opened box. This is admittedly a subjective matter. However, it looked a bit weird to me how it would scroll past the first few elements when you select, say, just the second or third element. With center, both the first few, and the last items, are shown when selecting something near the beginning or the end respectively. This also feels more symmetrical when the selection box ends up being placed above the select element.

Let me know if this sounds reasonable to you.

While working with this, I discovered several related issues that I've started looking at (not due to this PR). Including some weird scrollbar behavior. So I will push some follow-up commits after this one has been merged.

mikke89 · 2024-10-30T22:57:43Z

Working a bit more on these follow-ups, I am planning to make a related change: 514bd17. Just letting you know in case you have some input on that.

Paril · 2024-10-31T09:44:50Z

Yeah sounds reasonable to me - sorry I've been mostly unavailable for the past week or so & taking a minor vacation soon, but I'll be able to have a closer look when I start working on the UI for our next projects since I want to upgrade to 6.0.

I trust your judgment on the usability; if it feels good to you for keyboard nav, it should feel good for controller too. Thanks!

EDIT: reading back up too, I did want to stress again about the cancel button behavior; it also matches what web browsers do, as escape (or the matching 'cancel' button on console web browsers) closes select boxes too without bubbling the key up to the menu.

mikke89 · 2024-10-31T20:11:47Z

Sounds good, I merged this one for now, and also just pushed a lot of related fixups. We can follow up in the future if we want to tweak it more.

As for cancel button behavior. I played around with browsers a bit, see in particular this fiddle, I don't really see the behavior you describe. When I press escape, it commits whichever option is selected and changes the value accordingly, just as if pressing enter.

An interesting sidenote, I noticed a difference between Firefox and Chromium though. When selecting different options with the keyboard, in Firefox the value is changed in the background immediately, but the onchange/oninput event is only submitted when it closes. On Chrome, the value changes only when it closes (when the events are submitted).

In any case, I generally agree that it would be a nice functionality. How about: We add a new method to the select element called CancelSelection() or something like that, Then users are free to trigger that whenever the element receives an escape key. Further, we might want to do the same as web browsers do by default when seeing that same key (but of course ensuring that the behavior can be overridden).

This reverts the value to the one set when the selection box was opened. This is particularly helpful for use cases where e.g. the "Escape" key should cancel the selection. Added example usage to the demo sample.

mikke89 · 2024-10-31T20:54:11Z

Alright, I went ahead and added what I just suggested: 347773b

Of course, still open for feedback.

As for not submitting events until the selection box is closed, I think that would be a natural fit with the proposal in #702.

Paril · 2024-11-05T16:23:17Z

As for cancel button behavior. I played around with browsers a bit, see in particular this fiddle, I don't really see the behavior you describe. When I press escape, it commits whichever option is selected and changes the value accordingly, just as if pressing enter.

Yeah that's fine, but what was happening before was ESCAPE was backing out of the menu if you have the ESCAPE key set to return to previous menu - the key was bubbling up to the menu itself. I don't know if that's still how it is, but that's how it was in 5.0 prior to this branch. If you have a <select> open it should send the key to the selectbox to close the box, but not bubble it back up to the menu (because the selectbox consumed it).

I'm trying to think of any other control that might have similar behavior with keyboard navigation - maybe input type="text / textarea? If you hit ENTER on it, it activates the control letting you type into it, and escape should exit focus of the input but not send the key back to the menu since you don't want to return to the previous menu which is the natural use-case of the escape key.

Yeah 702 is exactly what this would go under, so that makes sense to me.

mikke89 · 2024-11-08T17:24:27Z

So I believe we don't do any special actions for the escape key at all. And since it is not handled, it bubbles up like every other key that is not handled. I suspect the escape behavior is something you added yourselves.

We could always discuss whether we should have some default actions for that. But regardless, if you make some custom actions to handle the key, the appropriate thing is to call event.StopPropagation() in the same handler, which should resolve the issue you're describing. This applies to the escape key, and also any other key being handled, normally. This is exactly what I did in the example in 347773b.

Paril · 2024-11-09T00:16:31Z

Yeah, it's something I added - I just mean that it seems logical for <select> and text inputs to natively handle escape. I have a hard time seeing a situation where you wouldn't want that behavior by default, & we'd just be adding it to every single select/input we use anyways so it seems like a waste/extra boilerplate that the underlying framework should be handling itself.

Paril force-pushed the paril/select-cancel branch from 80fdc9f to a34a52a Compare January 9, 2024 12:18

mikke89 added the enhancement New feature or request label Jan 22, 2024

mikke89 reviewed Jan 23, 2024

View reviewed changes

Paril marked this pull request as draft February 6, 2024 22:57

Paril marked this pull request as ready for review July 30, 2024 19:22

Paril changed the title ~~<select> improvements~~ controller/keyboard navigation support for <select> Jul 30, 2024

Paril and others added 6 commits October 30, 2024 20:56

fix a couple quirks with <select>:

64dc124

- selectbox now scrolls when nav is being used - selectbox ensures that the selected option is scrolled into view, anchored to the top - "escape" can be used to cancel selection

remove C++17 assignment

b86a673

remove the "cancel on escape" behavior

b97d4b4

adjust names to match ElementDocument conventions

19dbaea

document DropDownBoxLayoutType

bb8ab0e

mikke89 force-pushed the paril/select-cancel branch from 0c5ddd7 to 13e210c Compare October 30, 2024 19:57

mikke89 merged commit ad62159 into mikke89:master Oct 31, 2024
32 checks passed

mikke89 mentioned this pull request Dec 11, 2024

selectbox does not automatically scroll options into view when using spatial navigation #565

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

controller/keyboard navigation support for <select> #566

controller/keyboard navigation support for <select> #566

Paril commented Jan 9, 2024 •

edited

Loading

mikke89 left a comment •

edited

Loading

mikke89 Jan 23, 2024

Paril Jan 24, 2024

mikke89 Jan 23, 2024 •

edited

Loading

Paril Jan 24, 2024 •

edited

Loading

Paril commented Jan 24, 2024 •

edited

Loading

Paril commented Feb 6, 2024

mikke89 commented Feb 6, 2024

Paril commented Feb 6, 2024 •

edited

Loading

Paril commented Jul 30, 2024 •

edited

Loading

mikke89 commented Jul 30, 2024

mikke89 commented Oct 30, 2024

mikke89 commented Oct 30, 2024

Paril commented Oct 31, 2024 •

edited

Loading

mikke89 commented Oct 31, 2024

mikke89 commented Oct 31, 2024

Paril commented Nov 5, 2024 •

edited

Loading

mikke89 commented Nov 8, 2024

Paril commented Nov 9, 2024

controller/keyboard navigation support for <select> #566

controller/keyboard navigation support for <select> #566

Conversation

Paril commented Jan 9, 2024 • edited Loading

mikke89 left a comment • edited Loading

Choose a reason for hiding this comment

mikke89 Jan 23, 2024

Choose a reason for hiding this comment

Paril Jan 24, 2024

Choose a reason for hiding this comment

mikke89 Jan 23, 2024 • edited Loading

Choose a reason for hiding this comment

Paril Jan 24, 2024 • edited Loading

Choose a reason for hiding this comment

Paril commented Jan 24, 2024 • edited Loading

Paril commented Feb 6, 2024

mikke89 commented Feb 6, 2024

Paril commented Feb 6, 2024 • edited Loading

Paril commented Jul 30, 2024 • edited Loading

mikke89 commented Jul 30, 2024

mikke89 commented Oct 30, 2024

mikke89 commented Oct 30, 2024

Paril commented Oct 31, 2024 • edited Loading

mikke89 commented Oct 31, 2024

mikke89 commented Oct 31, 2024

Paril commented Nov 5, 2024 • edited Loading

mikke89 commented Nov 8, 2024

Paril commented Nov 9, 2024

Paril commented Jan 9, 2024 •

edited

Loading

mikke89 left a comment •

edited

Loading

mikke89 Jan 23, 2024 •

edited

Loading

Paril Jan 24, 2024 •

edited

Loading

Paril commented Jan 24, 2024 •

edited

Loading

Paril commented Feb 6, 2024 •

edited

Loading

Paril commented Jul 30, 2024 •

edited

Loading

Paril commented Oct 31, 2024 •

edited

Loading

Paril commented Nov 5, 2024 •

edited

Loading