Most terminal user interfaces can be used by blind users pretty easily. But why is that, actually? After all, the content of a terminal is pretty much unstructured data. A rectangular grid of characters with no extra information except colours. But is that really true?
The most important bit of metadata for a terminal screen reader is actually the cursor location. This is the one bit of information that is required to make a unstructured grid of characters surprisingly useful to a person mostly constraint to linear reading.
The terminal cursor acts as the focus. It indicates which part of the screen is currently being edited or selected. For editors, this is a pretty well understood concept. Where the cursor is, the next character will be inserted. However, there is more to a text-based user interface then just editing fields. And the cursor is not always visible.
Enter brick: a TUI for Haskell
A while ago, I looked at the brick terminal UI library for Haskell. When I tried its various demo programs, I noticed that my screen reader was reporting the cursor being in the lower right corner of the application when a menu item was selected. I had to manually investigate the screen and look at attributes (colours) to figure out which item in a list was currently selected. Amongst BRLTTY developers, we’ve decided long ago to not work around these issues on the screen reader side, rather try and make use of open source and fix the problems whenever we see them in the wild. Behind the scenes, we have fixed a bunch of frameworks and applications to place the cursor at the locus of focus. So I set out to understand brick internals to fix this.
A collection of cursors
brick works different from most TUI frameworks I know. You don’t have a single cursor which you set to a particular location. Rather, all the different components drawn on the screen can declare their own cursor location, and the composition mechansim ultimately chooses which cursor should be used.
This, while being pretty flexible, looked fundamentally wrong to me. Why? It doesn’t reflect the reality of a terminal.
Der Cursor ist immer unter überall!
There is no such thing as “no cursor” in a terminal. The cursor might be hidden, so it is not rendered on the screen. But the cursor still has a location where it sits and waits to print the next output character to the screen. A screen reader will pick that location up, no matter if the cursor is visible or hidden.
Visibility is key
So after patching brick to declare a cursor when rendering certain list items, checkboxes and radio items, I realized the actual missing bit. brick had no concept of cursor visibility. Rather, if the composition mechansim did not see a cursor declared, it would hide the cursor on-screen and “pretend” there was none. However, as we have learnt above, thats just not true and programs like screen readers actually rely on the cursor location to indicate the locus of fucs.
Raising this issue with the brick maintainer uncovered the fact that the low level vty library used by brick to do the actual terminal output did not have a concept of a hidden cursor either. Jonathan fixed this in vty 5.33.
Tying the knot
And since vty 5.33 is now in stack LTS, I thought to myself yesterday it is time to finally add support for invisible cursors to brick.
There is now a new function putCursor which has the same type signature as the already existing showCursor, but will make sure the cursor is not visible on-screen. This can and should be used to place a cursor at the locus of focus, even if that location is already visually indicated by different attributes.
The stock widgets that come with brick should now all be screen reader friendly. If you happen to maintain a brick application which provides a render function to something like renderList, please consider extending it to use showCursor. Once brick 0.64 is released, you can change to putCursor to clean up the visual appearance of your program.