Re: A Terible Problem with accessibility of Gnome
Matan Safriel via gnome-accessibility-list <[email protected]> Sat, 29 May 2021 13:18:15 +0300
| Newsgroups | gmane.comp.gnome.accessibility.general |
|---|---|
| Message-ID | <CAAiVq_1j5AWzP6RV0xL7r=bEiFzo1s4qMVCdgdT9+T4xAkXL1Q@mail.gmail.com> |
--===============3363772750124218830== Content-Type: multipart/alternative; boundary="0000000000001954c105c37550d6" --0000000000001954c105c37550d6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks, just to say I really appreciate this input! On Sat, May 29, 2021 at 11:29 AM Rynhardt Kruger via gnome-accessibility-list <[email protected]> wrote: > I definitely think image recognition has improved a lot, both in speed > and accuracy. However, even a difference like 50 milliseconds may be > noticeable by an experienced screen reader user, especially if one > uses speech at 400 words per minute or more. This is one of the > reasons why many blind users (including myself) still prefer the > text-mode console over a graphical terminal for command line work. The > graphical terminal is certainly very usable and works well for some > scenarios,, but there is quite a noticeable difference in performance > when using the text-mode console. > > Regards, > > Rynhardt > > On Sat, May 29, 2021 at 9:48 AM Matan Safriel <[email protected]> wrote= : > > > > Of course, the approach whereby the application developers interleave > semantics about the content beyond the particular way it is laid out in 2= D, > as you mention, is very sensible and robust as long as GUI toolkits and > development processes enable it as a default piece of development, and > application developers and designers put that extra semantic information > in, such as pointing out that a grid actually bears plain list semantics > and no special significance to it being a grid in a particular case. > > > > Do bear in mind that the potential for robust text reading from images > has significantly improved since the time that the current accessibility > paradigm took over, so I am not sure I see a reason it would be "always > slower" and things like that =E2=80=95 these affirmations are probably no= t true > today =E2=80=95 if one embarked on a machine learning project on this. > > > > On Sat, May 29, 2021 at 9:52 AM Rynhardt Kruger via > gnome-accessibility-list <[email protected]> wrote: > >> > >> I agree with you, it would be a useful fallback. It would never be a > primary solution though, it is essentially screen-scraping, and would hav= e > the same disadvantages as screen-scraping approaches that were used befor= e > accessibility API: > >> > >> > >> * Accessibility APIs make it the app developer's responsibility to > implement propper accessibility, this is by design. App developers know n= ot > just their app, but also the content associated with it, and therefore ca= n > implement an accessible experience that may be different from the visual > layout and yet more efficient for AT users. An example is the list of > recommendations on Youtube. Visually they are in a grid, but the screen > reader sees it as a list of headings. > >> * One of the most important requirements for a screen reader is > responsiveness. The quicker a blind user knows about an update in the > interface, the better. Even a slight delay before an announcement may > result in an interface that feels sluggish. This is why blind users often > use speech at a very fast speaking rate. A pattern-based approach will > always be slower than just reading the state via the accessibility API. > >> * It is dependent on the visual layout, which means accessibility may > break just because an app got new icons, or a few components shifted > position. These things are independent from the accessibility API. > >> > >> > >> Still, it may be a useful approach to consider for special edge cases, > and I am interested to see what happens in this space. > >> > >> > >> Regards, > >> > >> > >> Rynhardt > >> > >> > >> On Fri, 28 May 2021, 20:10 Matan Safriel via gnome-accessibility-list,= < > [email protected]> wrote: > >>> > >>> Hi Shadyar, > >>> > >>> Not an immediate solution at all, but I would say that AI (Machine > Learning) which snapshots the screen or window and is able to extract the > text from the snapshot image to then read it aloud, might be superior to > legacy accessibility API paradigms which rely on the application develope= rs > to interleave "accessibility" (ARIA etc.) information in each and every > field. > >>> > >>> Or at least as an augmentation that should be able to provide a reall= y > great fallback to any ARIA-like paradigm. > >>> > >>> It would be a project, sure, but it is really very accomplishable at > this time and age. > >>> > >>> Hopefully one day our desktops will be more fluid than only providing > voice services on top a graphical interaction interface, but a lot can be > done till then by leveraging Computer Vision AI in this space. Sorry agai= n > that this is not any immediate solution. > >>> > >>> Matan > >>> > >>> On Fri, May 28, 2021 at 4:04 AM Shadyar Khodayari via > gnome-accessibility-list <[email protected]> wrote: > >>>> > >>>> Hello > >>>> I'm a blind computer engineer, a developer and familiar with OS > >>>> Windows and Screen reader NVDA entirely. > >>>> I recently installed Linux Ubuntu v20.4 using Orca. > >>>> I read Accessibility section of Ubuntu documentation as well as Orca > >>>> documentation. > >>>> After logging in, > >>>> 1. When I am at Desktop through either pressing Super key + D or > >>>> holding Alt + CTRL and pressing Tab, next pressing arrow keys or Tab= , > >>>> Orca does not read desktop icons. > >>>> 2. 3. When I open a windows Settings, I press Tab numerous times but > >>>> Orca does not read the Setting categories like wireless, Bluetooth E= TC > >>>> in the window. It seems focus never moves on this part of the window= . > >>>> 3. When I open a windows like application files or file trash, I pre= ss > >>>> Tab numerous times but Orca does not read the main part of the windo= w. > >>>> It seems focus never moves on this part of the window. > >>>> 4. Should I do a specific config on Gnome? > >>>> 5. should I install another desktop environment? > >>>> I will be appreciate if you would help me. > >>>> Thanks and Regards > >>>> Shadyar KHodayari > >>>> _______________________________________________ > >>>> gnome-accessibility-list mailing list > >>>> [email protected] > >>>> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list > >>> > >>> _______________________________________________ > >>> gnome-accessibility-list mailing list > >>> [email protected] > >>> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list > >> > >> _______________________________________________ > >> gnome-accessibility-list mailing list > >> [email protected] > >> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list > _______________________________________________ > gnome-accessibility-list mailing list > [email protected] > https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list > --0000000000001954c105c37550d6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">Thanks, just to say I really appreciate t= his input! </div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"g= mail_attr">On Sat, May 29, 2021 at 11:29 AM Rynhardt Kruger via gnome-acces= sibility-list <<a href=3D"mailto:[email protected]">gno= [email protected]</a>> wrote:<br></div><blockquote class= =3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg= b(204,204,204);padding-left:1ex">I definitely think image recognition has i= mproved a lot, both in speed<br> and accuracy. However, even a difference like 50 milliseconds may be<br> noticeable by an experienced screen reader user, especially if one<br> uses speech at 400 words per minute or more. This is one of the<br> reasons why many blind users (including myself) still prefer the<br> text-mode console over a graphical terminal for command line work. The<br> graphical terminal is certainly very usable and works well for some<br> scenarios,, but there is quite a noticeable difference in performance<br> when using the text-mode console.<br> <br> Regards,<br> <br> Rynhardt<br> <br> On Sat, May 29, 2021 at 9:48 AM Matan Safriel <<a href=3D"mailto:dev.mat= [email protected]" target=3D"_blank">[email protected]</a>> wrote:<br> ><br> > Of course, the approach whereby the application developers interleave = semantics about the content beyond the particular way it is laid out in 2D,= as you mention, is very sensible and robust as long as GUI toolkits and de= velopment processes enable it as a default piece of development, and applic= ation developers and designers put that extra semantic information in, such= as pointing out that a grid actually bears plain list semantics and no spe= cial significance to it being a grid in a particular case.<br> ><br> > Do bear in mind that the potential for robust text reading from images= has significantly improved since the time that the current accessibility p= aradigm took over, so I am not sure I see a reason it would be "always= slower" and things like that =E2=80=95 these affirmations are probabl= y not true today =E2=80=95 if one embarked on a machine learning project on= this.<br> ><br> > On Sat, May 29, 2021 at 9:52 AM Rynhardt Kruger via gnome-accessibilit= y-list <<a href=3D"mailto:[email protected]" target=3D"= _blank">[email protected]</a>> wrote:<br> >><br> >> I agree with you, it would be a useful fallback. It would never be= a primary solution though, it is essentially screen-scraping, and would ha= ve the same disadvantages as screen-scraping approaches that were used befo= re accessibility API:<br> >><br> >><br> >> * Accessibility APIs make it the app developer's responsibilit= y to implement propper accessibility, this is by design. App developers kno= w not just their app, but also the content associated with it, and therefor= e can implement an accessible experience that may be different from the vis= ual layout and yet more efficient for AT users. An example is the list of r= ecommendations on Youtube. Visually they are in a grid, but the screen read= er sees it as a list of headings.<br> >> * One of the most important requirements for a screen reader is re= sponsiveness. The quicker a blind user knows about an update in the interfa= ce, the better. Even a slight delay before an announcement may result in an= interface that feels sluggish. This is why blind users often use speech at= a very fast speaking rate. A pattern-based approach will always be slower = than just reading the state via the accessibility API.<br> >> * It is dependent on the visual layout, which means accessibility = may break just because an app got new icons, or a few components shifted po= sition. These things are independent from the accessibility API.<br> >><br> >><br> >> Still, it may be a useful approach to consider for special edge ca= ses, and I am interested to see what happens in this space.<br> >><br> >><br> >> Regards,<br> >><br> >><br> >> Rynhardt<br> >><br> >><br> >> On Fri, 28 May 2021, 20:10 Matan Safriel via gnome-accessibility-l= ist, <<a href=3D"mailto:[email protected]" target=3D"_b= lank">[email protected]</a>> wrote:<br> >>><br> >>> Hi Shadyar,<br> >>><br> >>> Not an immediate solution at all, but I would say that AI (Mac= hine Learning) which snapshots the screen or window and is able to extract = the text from the snapshot image to then read it aloud, might be superior t= o legacy accessibility API paradigms which rely on the application develope= rs to interleave "accessibility" (ARIA etc.) information in each = and every field.<br> >>><br> >>> Or at least as an augmentation that should be able to provide = a really great fallback to any ARIA-like paradigm.<br> >>><br> >>> It would be a project, sure, but it is really very accomplisha= ble at this time and age.<br> >>><br> >>> Hopefully one day our desktops will be more fluid than only pr= oviding voice services on top a graphical interaction interface, but a lot = can be done till then by leveraging Computer Vision AI in this space. Sorry= again that this is not any immediate solution.<br> >>><br> >>> Matan<br> >>><br> >>> On Fri, May 28, 2021 at 4:04 AM Shadyar Khodayari via gnome-ac= cessibility-list <<a href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a>> wrote:<br> >>>><br> >>>> Hello<br> >>>> I'm a blind computer engineer, a developer and familia= r with OS<br> >>>> Windows and Screen reader NVDA entirely.<br> >>>> I recently installed Linux Ubuntu v20.4 using Orca.<br> >>>> I read Accessibility section of Ubuntu documentation as we= ll as Orca<br> >>>> documentation.<br> >>>> After logging in,<br> >>>> 1. When I am at Desktop through either pressing Super key = + D or<br> >>>> holding Alt + CTRL and pressing Tab, next pressing arrow k= eys or Tab,<br> >>>> Orca does not read desktop icons.<br> >>>> 2. 3. When I open a windows Settings, I press Tab numerous= times but<br> >>>> Orca does not read the Setting categories like wireless, B= luetooth ETC<br> >>>> in the window. It seems focus never moves on this part of = the window.<br> >>>> 3. When I open a windows like application files or file tr= ash, I press<br> >>>> Tab numerous times but Orca does not read the main part of= the window.<br> >>>> It seems focus never moves on this part of the window.<br> >>>> 4. Should I do a specific config on Gnome?<br> >>>> 5. should I install another desktop environment?<br> >>>> I will be appreciate if you would help me.<br> >>>> Thanks and Regards<br> >>>> Shadyar KHodayari<br> >>>> _______________________________________________<br> >>>> gnome-accessibility-list mailing list<br> >>>> <a href=3D"mailto:[email protected]" targ= et=3D"_blank">[email protected]</a><br> >>>> <a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-a= ccessibility-list" rel=3D"noreferrer" target=3D"_blank">https://mail.gnome.= org/mailman/listinfo/gnome-accessibility-list</a><br> >>><br> >>> _______________________________________________<br> >>> gnome-accessibility-list mailing list<br> >>> <a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a><br> >>> <a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-acces= sibility-list" rel=3D"noreferrer" target=3D"_blank">https://mail.gnome.org/= mailman/listinfo/gnome-accessibility-list</a><br> >><br> >> _______________________________________________<br> >> gnome-accessibility-list mailing list<br> >> <a href=3D"mailto:[email protected]" target=3D"_b= lank">[email protected]</a><br> >> <a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-accessibi= lity-list" rel=3D"noreferrer" target=3D"_blank">https://mail.gnome.org/mail= man/listinfo/gnome-accessibility-list</a><br> _______________________________________________<br> gnome-accessibility-list mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">gno= [email protected]</a><br> <a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list= " rel=3D"noreferrer" target=3D"_blank">https://mail.gnome.org/mailman/listi= nfo/gnome-accessibility-list</a><br> </blockquote></div></div> --0000000000001954c105c37550d6-- --===============3363772750124218830== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ gnome-accessibility-list mailing list [email protected] https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list --===============3363772750124218830==--