We have an add-on developer here who is working on a custom app module for someone. Could any of you answer the below questions and CC your answer to the add-ons list please? Thank you.
From: firstname.lastname@example.org [mailto:email@example.com] On Behalf Of Flint Million
Sent: Sunday, November 10, 2013 9:02 AM
Subject: [nvda-addons] Re: Developer Add-on Reference(s)?
Thanks for the advice! My project is coming along very well.
The use case is for one specific user, who has access to an 80-cell Braille display, and the type of work being done is pretty dependent on Braille output. (It's a programming project) Using this with speech becomes pretty impractical (think: "Value at 4 is 0 x 0 0 0 0 f 0 0 c 1 0 a 0 0 4 f e" Try doing that without Braille and still keeping things in order!). Therefore for this specific use case Braille is pretty much required. My status cells implementation is working exactly as I expect it to.
Now, another question: Is there a way to subclass the config dialogs NVDA provides by default, and use them to put up my own configuration? I know NVDA has its own config store, which I think I could also make use of for the project to store the config (which would of course make the app module's config portable in the event of a portable install!). The only thing I need is a good way to get a GUI up quickly with a few options - say "Select item to display in status cell area" and have a combo box of a few possible options. I've done a *little* coding with wx in the past but it's pretty unwieldy to me and I'm feeling unless I can somehow borrow the NVDA config dialog code I'll be writing a *lot* of code for a simple dialog.
Also, is there a way to, temporarily, capture all keyboard input, say for one keystroke? Imagine this user experience. A user presses an NVDA shortcut key. NVDA speaks "Press desired function." The next key - whatever it is - is received by the app module, which can act on it, or simply say "Invalid keypress." This keypress would obviously not be forwarded to the app. Basically, the thing I'd need here is a way to, for one keypress, capture any and all keypresses within the script.
The desired user experience overall is as follows: When the user launches the app, NVDA starts running a script that extracts the desired information and puts it in the status cell; this is delegated to a thread which causes it to happen once per second so it can be updated in real-time. When the user presses a certain key, NVDA says "Press custom function." Then, the user can press various keys to cause different things to happen. For example, "U" toggles the live updater on and off. (A simple class-level flag can do this.) "C" brings up the configuration window. "Z" activates a specific item in the menu bars. "X" sends a completely different keypress to the application. And so on.
So basically, the things I do not yet know how to do for sure are:
* Bring up a config dialog box without a ton of work, which can save its options to the NVDA config store.
* Temporarily cause my script to receive all keyboard input for one keystroke.
* Send specific keystrokes to the application without the user having pressed them.
* Activate menu items, buttons, etc. in the application without the user having to directly manipulate them.
Any advice on these aspects?
On Wed, Nov 6, 2013 at 2:58 AM, Mesar Hameed <firstname.lastname@example.org> wrote:
On Tue 05/11/13,19:40, Flint Million wrote:
> The core of the project is to get data
> from edit fields in the application, parse it, and report the output
> after having distilled it down significantly.
> The method I have discovered thus far is to simply get the Desktop
> object, then search its children for the app’s main window (using the
> Name property, which to me feels prone to error - a folder could be
> open with the same name, for example, but there may be a better way
> that I’m not aware of yet to find an app’s window).
> The app has multiple “top level” windows, so getForegroundObject() is
> unreliable and if the user is in the wrong window, that function won’t
> get the right window - which is why I am starting at the desktop.
If you are in the appModule, and you call api.getForegroundObject() you
should get the currently focused window/dialog.
You can test its properties to check if it is the window/dialog that you
want to be working with, and discard if the user has the wrong dialog
> Then, in the main
> window, I drill into the app’s object tree to locate the edit fields.
> The edit fields all have the same control ID, and there is nothing
> uniquely identifying them in any NVDA object properties I can see, so
> I’ve simply resorted to capturing all the edit fields then checking
> each one to see if it is the data of interest, then parsing.
If there is absolutely no unique attributes for the object to enable you
to identify it specifically, then another approatch might be to fetch the
objects with relation to a known object.
From source/appModules/poedit.py, recycle the two functions getPath(obj, ancestor) and
For example once you know that the user has the main window focused, you
know the status1text is at some relative location.
You can descover the path in the python consol,
then hardcode this path into the appModule.
status1Text = fetchObject(mainWindow, path)
status2Text = fetchObject(mainWindow, path2)
Have a look over the poedit.py to see a concrete example of locating
points of interest.
Those two functions should probably be migrated to api.py to make them
more usable by appModule developers.
> The other aspect to this is that I need to essentially implement
> custom Braille status cells. Basically, certain text in certain edit
> fields changes frequently, and it needs to be instantly viewable on
> her 80-cell display, rather than having to repeatedly wade through the
> huge text box to find the information of relevance, only to have the
> focus moved as the text box updates again.
> To this end, I modified the FS Focus driver in such a way that it
> exposes a new function that lets me directly control some of the
> display’s cells - and in turn reports a fewer number of cells to NVDA
> so that NVDA will work the remaining cells as if it’s just a shorter
> display. I worked out all of the necessary modifications to make the
> driver still function exactly as intended, just reporting a smaller
> display size to NVDA, while still giving me full control of the
> remaining cells. All of this is working exactly the way I need it to -
> it’s kludgy and not the best solution by far (and some of you will be
> frowning upon me), but given the time constraints involved, it proved
> to be the fastest method that I knew of that worked. (The app module
> init checks dir(braille.handler.display) to see if my custom function
> is there - if not, it throws a warning and none of the functions work,
> but if it is then I simply call
> braille.handler.display.my_custom_function when I need to update the
> status cells.)
> So, my questions are as follows:
> 1. Is there any other “sanctioned” way to do customized status cells
> in NVDA, other than my very scotch-tape method of modifying the stock
> display driver for my friend’s display such so that it allows this to
> be done programmatically? (If not, I am willing to, after the course
> is over and I have more time, to consider looking into adding
> something like this to the core NVDA Braille output code, so it’ll
> work with all displays, and submitting it as a possible feature
At the moment NVDA does not support status cells, but what you have done
sounds not too bad. It would be great to generalize it and get it
In the meantime, status cells are suppose to be useful but not chritical for efficiency.
You might also consider that the user might want access to the
information without always having their braille display to hand, or want
to hear the information via speech.
Maybe it might be better to implement shortcuts for displaying
information on the fly via scripts.
fg = api.getForeground()
# check if the main window is focused.
if not fg ...: return
status1Text = fetchObject(api.getForeground(), path)
message1 = myDataParserAndSummerizer(status1text.value)
> 2. Other than simply retrieving information about objects, is there
> any way to “do” things to objects? I.e. click a button, change the
> text in an edit field, send a keystroke to the application (e.g.
> control+O) and so on?
As Joseph suggested, looking over miranda32.py might be useful.
If the user presses control+o or whatever and you have not bound it to a
script in your appModule, it will be passed through to the application,
and it will be acted on as standard.
What is your sinario for wanting to send a shortcut from code?
> 3. How can I find out what events are available for a given control?
> Is there some way to query that from the object itself, and then in
> code bind a function to that event? Suppose I want to have a function
> fire off every time a particular edit box is modified in any way, or
> any time the value of a slider is changed, or whenever the user clicks
> on a certain object that may not normally be considered clickable, or
> when the user is closing the application, or when the caret is moved
> to a different edit field, or...
For each object of interest, or class of objects, you might need to
create a custom nvda overlay class.
Look at how list items in poedit are being overwridden using the
There are other good examples in the appModules directory for custom
For each custom class, you would override valueChanged, or whatever
event you are interested in for that object.
> 4. Can an app module throw up a GUI of its own, say, to configure it?
> I’m only vaguely familiar with WX but I could use some of the NVDA
> configuration dialogs themselves as a good baseline for how to code my
> own config dialog for my app module…
What are the sort of things that you are wanting to expose as configurable?
> The documentation I’ve found on NVDA development is quite sparse so
> I’m looking for a good reference of some kind to these sorts of coding
> techniques for NVDA plugins.
Having a look at existing code, and asking questions is probably the
If you do end up making useful notes that might be helpful to others,
then sending in a patch for the developer guide would be very welcome.
> Thanks for any guidance, and I’m excited to see what I can do to
> enhance accessibility through NVDA!
Welcome to open source accessibility, hope you are having fun for a
a reasonably sized posative, non zero value. :D