NVDA Magnifier available for early testing and feedback

[seanbudd]

An initial version of NVDA Magnifier is now available in alpha builds.

Thank you @Boumtchack for all your hard work in #19228! Also thank you to @CyrilleB79 and @JulienCochuyt for supporting him in his native language.

Please read the "Magnifier" section of the user guide in the latest alpha build to learn more about this feature.

Please note, this is an initial version, with @Boumtchack intending to expand support of the Magnifier further.

We'd love feedback on this feature, and our plans to improve it further.

Roadmap

Phase 1 - complete

• Full screen magnifier that can track system focus
• Cover different cursor tracking modes: mouse tracking, browse mode / caret tracking, review & object nav cursor support
• Full screen mode focus tracking: border, fixed, relative
• Full screen spotlight
• Key commands/input gestures/shortcuts
  • TODO: add Magnifier implementation #19228 (comment)
• Color filters including inversion
• User guide docs
• Basic Settings: i.e. default behavior on start.

Phase 2

• Bug fixes from phase 1
• Added display modes:
  • Windowed (fixed)
  • Windowed (docked)
  • Windowed (lens tracking)
• Cycle display mode: fullscreen, fixed, docked, lens tracking
• Cycle docked position: left/right/top/bottom (unbound)
• Expand settings:
  • Settings for each windowed mode:
    • Size of window
      • Fixed: height and width % of screen
      • Docked: height/width % of screen
      • Lens tracking: height and width % of screen
    • Position of window
      • Fixed: x/y position
      • Docked: left/right/top/bottom
      • Lens tracking: none? Prefer left/right of cursor?
    • border width for window
  • Create cycle list filters like speech modes, which allow you to remove items from the cycle list for cycle commands
    • display modes cycle list
    • filters cycle list
    • Full Screen Mode focus tracking modes cycle list
    • docked position cycle list (e.g. left/right/top/bottom)

Phase 3

• Basic training docs
• Multi display support
• Bug fixes from phase 2
• Other feature suggestions/improvements

[seanbudd]

Some initial feedback:

• moving the frame is quite jerky, Windows Magnifier is much smoother. could we apply similar transitions steps as spotlight to help smooth it out? Particularly an issue with mouse movement.
• we noticed the magnifier getting stuck sometimes, requiring the user to restart it. a log and steps to reproduce is needed here
• we may want to add shortcuts to unbound commands particularly the filter switching command
• it only works with one display, the primary display. multiple displays are not supported

[Boumtchack]

I've open a PR for the smoothing of the mouse, and a fix on spotlight #19468

[Boumtchack]

I remember that I was trying to make work docked and lens modes, what do you mean by fixed window?

I started an other pr to start working on them #19473

[seanbudd]

I envision 3 windowed modes:

• Fixed window: a window of a set height/width and set position of the screen. e.g. I want a 100x100px window that floats roughly in the top right of my screen.
• Docked window: a window docked to the top/left/right/bottom of the screen, with a fixed width. Basically a fixed window pinned to the edge of the screen.
• Lens tracking: a window of a set height/width that follows the cursor.

[Boumtchack]

I feel like fixed and docked are pretty much the same, just size and position that are differents, wouldn't it be more interisting to have only one type that we can customise in size and position, pin to the edge or not.
I'm not a user of such tools so I don't know if having two different is really mandatory

[seanbudd]

Docked serves a different purpose to fixed. For example, having a small "looking glass" pinned just to the side of something you are focusing on. I think being able to quickly switch from a docked window (which is the most popular windowed format) and a smaller pinned window is important.
The amount of screen taken up by the magnifier tends to depend a lot on how much vision a person has. If you need something at high magnification to see anything at all, you tend use full screen magnification. If you just need a section of the screen magnifier for reading, you tend to use docked mode. Having a smaller fixed / tracking window is mostly useful for those that rarely need magnification.

[CyrilleB79]

I was really expecting thes.

Here are my feedback, my questioning and many suggestions (I hope you don't mind):

Display modes

1. What is "fixed" display mode?

2. Re the priorities, I am not very sure that all these display modes are really expected. Especially regarding lense mode, I do not know if it is actually used by visually impaired people (e.g. I do not remember zoomtext implementing such a mode).
   Though, I acknowledge that a part of the work has already been done for other display modes, so maybe you just intend to capitalize this work.

More keyboard commands

Even considering only the existing full screen display mode, I suggest the addition of the following commands:

• move the zoomed view (see NVDA Magnifier - Add gestures to pan the zoomed view #19471)
• move the view to center it on the mouse, on the the focus / the caret, on the navigator object / the review cursor.
• move the mouse to the center of the view
• save / restore the position of the zoomed view
• toggle mouse tracking, focus/caret tracking, navigator object/review cursor tracking.

Also for mouse users (I am not one of them), I guess that a command to zoom in/out with modifier+mouseWheel (e.g. NVDA+mouseWheel) would be appreciated.

Keyboard commands consistency

If all (or most of) the commands listed in the previous sections are implemented in the future, and maybe other ones, we should have a reflexion on a set of consistent and user-friendly set of keyboard gestures for these commands.

The existing assigned gestures, or those about to be integrated (#19464), all use NVDA+shift as modifiers keys, which is a good thing for consistency.

Regarding the main key, we have not been able to assign one single command its first letter as main key. This indicates that we already have difficulty to find suitable keys. If we assign mmany more commands as described in the section above, it will be difficult to have a user-friendly set of commands.

Alternatively, if layered gestures (#3687) was implemented, we could move all the commands to open settings panels to layered commands and free up a wide range of NVDA+control+letter gestures. But it does not seem that any contributor is about to implement #3687 for now.

Filters

Windows already provides color filters support and you can configure control+windows+C to toggle it. I wonderr, do we really need another color filtering in NVDA which would take one more key? Wouldn't it be better to encourage people to use Windows native feature?

More, it's worth noting that Windows color filtering is not incompatible with screen curtain, i.e. does not prevent it to start, while Windows Magnifier color inversion feature blocks screen curtain.

Default starting values

There are 3 parameters (default zoom level, default focus tracking mode and default filter) which just define the value at magnifier startup.

This is not the approach that is usually taken for NVDA parameters. For example for synthesizer speech rate, there is no default starting rate.
People just modify the speech rate and, depending on if "Save configuration when exiting NVDA" is checked or not, they restart with the last used rate, or the last rate manually saved in NVDA config.

I do not see any reason to make something different for magnifier parameter. Or is there a specific reason why this would differ?

Multi display support

What do you have in mind with this?

A fantastic feature would be to have full screen display mode on screen 1 and the full unzoomed desktop on screen 2.
The two important use cases for this feature are:

• working with (well) sighted people: the VI user works on screen 1 while the sighted colleague works on screen 2.
• For a VI user, being able to switch from zoomed view to full screen picture without needing to use the fullscreen spotlight command/feature.

Though, I'm afraid this cannot be easily accomplished with the currently used API and will need to involve more low-level development for this. Or am I wrong?

[CyrilleB79]

Adding these display modes is a small amount of work, and were requested features in the limited market research we performed.

OK.

Could you elaborate more on this one, and what the expected UX is:

• toggle mouse tracking, focus/caret tracking, navigator object/review cursor tracking.

Actually the Magnifier can reposition its zoomed view based on various system or NVDA events, that's called tracking. These system events can be: mouse cursor move, focus move, caret move. Regarding NVDA specific events, they can be browse mode cursor move, navigator object move, review cursor move. And I do not yet consider other special cases such as excel cells moving, navigation in equations, etc.

The idea is to be able to configure which tracking is enabled or not. Of course, using all these categories may be confusing so it would be nice to group them.

As an example for Windows Magnifier, you have 4 tracking types that are separately configurable with 4 checkboxes: follow mouse, follow focus, follow caret and follow Narrator cursor.

For NVDA, we should be able to enable or disable:

• mouse tracking
• focus tracking
• caret (text cursor) tracking; it would make sense to include browse mode cursor here.
• navigator object tracking
• review cursor tracking
  The 2 last categories may be grouped together because I do not know if there is a use case for someone to enable one and disable the other one.

Windows already provides color filters support and you can configure control+windows+C to toggle it. I wonder, do we really need another color filtering in NVDA which would take one more key? Wouldn't it be better to encourage people to use Windows native feature?

I think that command only toggles filters on/off, not allow cycling. Additionally, we have the potential to offer more/better filters. I think there is value in integrating most/all of Windows Magnifier features over, so we can have tighter integration with NVDA and can develop them further.

OK.

[Boumtchack]

I think we might have consistent behaviour here actually, and just the naming of the settings is misleading.
@Boumtchack can you confirm what the behaviour is here? does zooming in on the magnifier mean when you restart, the zoom level is kept?
What do you think about changing the behaviour to what Cyrille suggested here.

Maybe I didn't understand well but I think I removed it.
When we talked about this I think I understand that you wanted the option to start with what the user had entered in the parameters and that whatever changes he would do during the use would not be saved.
But I could put it back.

The ideas you raise sound interesting. I'm not sure if they are technically feasible, maybe @Boumtchack has a better idea here.
As multi-display support is a bit more complicated, we can flesh out the UX more once Phase 2 is closer to complete.

No idea on how it could be done. I'm not sure about the priority of it.

For NVDA, we should be able to enable or disable:
mouse tracking
focus tracking
caret (text cursor) tracking; it would make sense to include browse mode cursor here.
navigator object tracking
review cursor tracking
The 2 last categories may be grouped together because I do not know if there is a use case for someone to enable one and disable the other one.

For now I have only 2 type of focus: mouse and NVDA, I don't think it should be impossible to do but I'm afraid I might need to rethink some really early code I made and that could be a lot of work to addapt everything that is impacted

[seanbudd]

Thanks @CyrilleB79 for clarifying the tracking mode settings

When we talked about this I think I understand that you wanted the option to start with what the user had entered in the parameters and that whatever changes he would do during the use would not be saved.
But I could put it back.

I think lets keep this behaviour for now. I think adjusting the zoom and filters on the fly might be worth keeping separate to setting defaults. It is unusual UX for NVDA, but I think might make sense contextually here?

No idea on how it could be done. I'm not sure about the priority of it.

Definitely a low priority - we can investigate it much further down the line

[CyrilleB79]

When we talked about this I think I understand that you wanted the option to start with what the user had entered in the parameters and that whatever changes he would do during the use would not be saved.
But I could put it back.

I think lets keep this behaviour for now. I think adjusting the zoom and filters on the fly might be worth keeping separate to setting defaults. It is unusual UX for NVDA, but I think might make sense contextually here?

@seanbudd, you write "for now" and endyour comment with a question. It seem to me that you are not very sure of the approach to take here. Then would you mind discuss it more broadly with other NV Access people to have a more structured opinion on this question?
Personally and for now, I think that using a start value approach is not a good idea because it differs from the existing UX, thus creating a more complex UX. Though, if it is confirmed as the choice of NV Access, I'd expect some clarification on when (in which case) and why this approach should be used for Magnifier (and not for speech settings for example).

[seanbudd]

@CyrilleB79 - I tend to agree, we will resolve the decision in #19493

[CyrilleB79]

Two bug feedback:

Tracking issue in excel

When you move the active cell around the sheet with left/right/up/down arrows (without editing it), the Magnifier does not follow.

Focus tracking issue

If you disable "focus moves navigator object" (NVDA+7), the Magnifier does not follow the focus anymore.
Actually, it seems that the Magnifier tracks the navigator object, not the focus.
IMO, it should be able to track both, and both tracking possibilities should be configurable separately, as written yesterday.
I acknowledge that it may represent a significant amount of work; I have not had a look to it though.

---

Both bugs are related to the exact nature of what is actually tracked.
@Boumtchack have you present in mind the difference between focus and navigator object? Or system caret/cursor (and browse mode cursor) vs review cursor? We may discuss it privately in French if you wish.

[Boumtchack]

I tought my magnifier was following any of the NVDA's object, wich was the last one moving. I will look into it.
I hope it will resolve the excel problem.

difference between focus and navigator object

For me it's just a color difference (added from settings) and gesture to make them move, as I don't really use them everyday I might not grasp the real difference between what is red and blue for me.

[Boumtchack]

for what I know for now, I'm using api.getReviewPosition() and api.getNavigatorObject()

I wonder if the the problem with excel or after deactivating focus move on navigator object, might comes before I call the api?

Still looking into it atm

[Boumtchack]

I found the issue, as I was not calling the right api.

I'll be implementing a focusManager in utils to make things clean and so focus option will be better implemented in the future