Re: A Terible Problem with accessibility of Gnome

--===============2219148818608780550==
Content-Type: multipart/alternative; boundary="000000000000fa738405c37336a7"

--000000000000fa738405c37336a7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Of course, the approach whereby the application developers interleave
semantics about the content beyond the particular way it is laid out in 2D,
as you mention, is very sensible and robust as long as GUI toolkits and
development processes enable it as a default piece of development, and
application developers and designers put that extra semantic information
in, such as pointing out that a grid actually bears plain list semantics
and no special significance to it being a grid in a particular case.

Do bear in mind that the potential for robust text reading from images has
significantly improved since the time that the current accessibility
paradigm took over, so I am not sure I see a reason it would be "always
slower" and things like that =E2=80=95 these affirmations are probably not =
true
today =E2=80=95 if one embarked on a machine learning project on this.

On Sat, May 29, 2021 at 9:52 AM Rynhardt Kruger via
gnome-accessibility-list <[email protected]> wrote:

> I agree with you, it would be a useful fallback. It would never be a
> primary solution though, it is essentially screen-scraping, and would hav=
e
> the same disadvantages as screen-scraping approaches that were used befor=
e
> accessibility API:
>
>
> * Accessibility APIs make it the app developer's responsibility to
> implement propper accessibility, this is by design. App developers know n=
ot
> just their app, but also the content associated with it, and therefore ca=
n
> implement an accessible experience that may be different from the visual
> layout and yet more efficient for AT users. An example is the list of
> recommendations on Youtube. Visually they are in a grid, but the screen
> reader sees it as a list of headings.
> * One of the most important requirements for a screen reader is
> responsiveness. The quicker a blind user knows about an update in the
> interface, the better. Even a slight delay before an announcement may
> result in an interface that feels sluggish. This is why blind users often
> use speech at a very fast speaking rate. A pattern-based approach will
> always be slower than just reading the state via the accessibility API.
> * It is dependent on the visual layout, which means accessibility may
> break just because an app got new icons, or a few components shifted
> position. These things are independent from the accessibility API.
>
>
> Still, it may be a useful approach to consider for special edge cases, an=
d
> I am interested to see what happens in this space.
>
>
> Regards,
>
>
> Rynhardt
>
> On Fri, 28 May 2021, 20:10 Matan Safriel via gnome-accessibility-list, <
> [email protected]> wrote:
>
>> Hi Shadyar,
>>
>> Not an immediate solution at all, but I would say that AI (Machine
>> Learning) which snapshots the screen or window and is able to extract th=
e
>> text from the snapshot image to then read it aloud, might be superior to
>> legacy accessibility API paradigms which rely on the application develop=
ers
>> to interleave "accessibility" (ARIA etc.) information in each and every
>> field.
>>
>> Or at least as an augmentation that should be able to provide a really
>> great fallback to any ARIA-like paradigm.
>>
>> It would be a project, sure, but it is really very accomplishable at thi=
s
>> time and age.
>>
>> Hopefully one day our desktops will be more fluid than only providing
>> voice services on top a graphical interaction interface, but a lot can b=
e
>> done till then by leveraging Computer Vision AI in this space. Sorry aga=
in
>> that this is not any immediate solution.
>>
>> Matan
>>
>> On Fri, May 28, 2021 at 4:04 AM Shadyar Khodayari via
>> gnome-accessibility-list <[email protected]> wrote:
>>
>>> Hello
>>> I'm a blind computer engineer, a developer and familiar with OS
>>> Windows and Screen reader NVDA entirely.
>>> I recently installed Linux Ubuntu v20.4 using Orca.
>>> I read Accessibility section of Ubuntu documentation as well as Orca
>>> documentation.
>>> After logging in,
>>> 1. When I am at Desktop through either pressing Super key + D or
>>> holding Alt + CTRL and pressing Tab, next pressing arrow keys or Tab,
>>> Orca does not read desktop icons.
>>> 2. 3. When I open a windows Settings, I press Tab numerous times but
>>> Orca does not read the Setting categories like wireless, Bluetooth ETC
>>> in the window. It seems focus never moves on this part of the window.
>>> 3. When I open a windows like application files or file trash, I press
>>> Tab numerous times but Orca does not read the main part of the window.
>>> It seems focus never moves on this part of the window.
>>> 4. Should I do a specific config on Gnome?
>>> 5. should I install another desktop environment?
>>> I will be appreciate if you would help me.
>>> Thanks and Regards
>>> Shadyar KHodayari
>>> _______________________________________________
>>> gnome-accessibility-list mailing list
>>> [email protected]
>>> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list
>>>
>> _______________________________________________
>> gnome-accessibility-list mailing list
>> [email protected]
>> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list
>>
> _______________________________________________
> gnome-accessibility-list mailing list
> [email protected]
> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list
>

--000000000000fa738405c37336a7
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Of course, the approach whereby the application developers=
 interleave semantics about the content beyond the particular way it is lai=
d out in 2D, as you mention, is very sensible and robust as long as GUI too=
lkits and development processes enable=C2=A0it as a default piece of develo=
pment,=C2=A0and application developers and designers put that extra semanti=
c information in, such as pointing out that a grid actually bears plain lis=
t semantics and no special significance to it being a grid in a particular =
case.=C2=A0<div><br></div><div>Do bear in mind that the potential for robus=
t text reading from images has significantly improved since the time that t=
he current accessibility paradigm took over, so I am not sure I see a reaso=
n it would be &quot;always slower&quot; and things like that =E2=80=95 thes=
e affirmations are probably not true today =E2=80=95 if one embarked on a m=
achine learning project on this.</div></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Sat, May 29, 2021 at 9:52 AM Rynha=
rdt Kruger via gnome-accessibility-list &lt;<a href=3D"mailto:gnome-accessi=
[email protected]">[email protected]</a>&gt; wrote:<br=
></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;=
border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><=
p dir=3D"ltr">I agree with you, it would be a useful fallback. It would nev=
er be a primary solution though, it is essentially screen-scraping, and wou=
ld have the same disadvantages as screen-scraping approaches that were used=
 before accessibility API:</p><p dir=3D"ltr"><br></p>
<p dir=3D"ltr">* Accessibility APIs make it the app developer&#39;s respons=
ibility to implement propper accessibility, this is by design. App develope=
rs know not just their app, but also the content associated with it, and th=
erefore can implement an accessible experience that may be different from t=
he visual layout and yet more efficient for AT users. An example is the lis=
t of recommendations on Youtube. Visually they are in a grid, but the scree=
n reader sees it as a list of headings.<br>
* One of the most important requirements for a screen reader is responsiven=
ess. The quicker a blind user knows about an update in the interface, the b=
etter. Even a slight delay before an announcement may result in an interfac=
e that feels sluggish. This is why blind users often use speech at a very f=
ast speaking rate. A pattern-based approach will always be slower than just=
 reading the state via the accessibility API.<br>
* It is dependent on the visual layout, which means accessibility may break=
 just because an app got new icons, or a few components shifted position. T=
hese things are independent from the accessibility API.</p><p dir=3D"ltr"><=
br></p>
<p dir=3D"ltr">Still, it may be a useful approach to consider for special e=
dge cases, and I am interested to see what happens in this space.</p><p dir=
=3D"ltr"><br></p>
<p dir=3D"ltr">Regards,</p><p dir=3D"ltr"><br></p>
<p dir=3D"ltr">Rynhardt</p></div>
<br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri=
, 28 May 2021, 20:10 Matan Safriel via gnome-accessibility-list, &lt;<a hre=
f=3D"mailto:[email protected]" target=3D"_blank">gnome-acc=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20=
4,204);padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><div>Hi Shadyar,=
</div><div><br></div><div>Not an immediate solution at all, but I would say=
 that AI (Machine Learning) which snapshots the screen or window and is abl=
e to extract the text from the snapshot image to then read it aloud, might =
be superior to legacy accessibility API paradigms which rely on the applica=
tion developers to interleave &quot;accessibility&quot; (ARIA etc.) informa=
tion in each and every field.=C2=A0<br><br>Or at least as an augmentation t=
hat should be able to provide a really great fallback to any ARIA-like para=
digm.=C2=A0</div><div><br></div><div>It would be a project, sure, but it is=
 really very accomplishable at this time and age.=C2=A0</div><div><br></div=
><div>Hopefully one day our desktops will be more fluid than only providing=
 voice services on top a graphical interaction interface, but a lot can be =
done till then by leveraging Computer Vision AI in this space. Sorry again =
that this is not any immediate solution.=C2=A0</div><div><br></div><div>Mat=
an<br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"=
gmail_attr">On Fri, May 28, 2021 at 4:04 AM Shadyar Khodayari via gnome-acc=
essibility-list &lt;<a href=3D"mailto:[email protected]" r=
el=3D"noreferrer" target=3D"_blank">[email protected]</a>&=
gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello=
<br>
I&#39;m a blind computer engineer, a developer and familiar with OS<br>
Windows and Screen reader NVDA entirely.<br>
I recently installed Linux Ubuntu v20.4 using Orca.<br>
I read Accessibility section of Ubuntu documentation as well as Orca<br>
documentation.<br>
After logging in,<br>
1. When I am at Desktop through either pressing Super key + D or<br>
holding Alt + CTRL and pressing Tab, next pressing arrow keys or Tab,<br>
Orca does not read desktop icons.<br>
2. 3. When I open a windows Settings, I press Tab numerous times but<br>
Orca does not read the Setting categories like wireless, Bluetooth ETC<br>
in the window. It seems focus never moves on this part of the window.<br>
3. When I open a windows like application files or file trash, I press<br>
Tab numerous times but Orca does not read the main part of the window.<br>
It seems focus never moves on this part of the window.<br>
4. Should I do a specific config on Gnome?<br>
5. should I install another desktop environment?<br>
I will be appreciate if you would help me.<br>
Thanks and Regards<br>
Shadyar KHodayari<br>
_______________________________________________<br>
gnome-accessibility-list mailing list<br>
<a href=3D"mailto:[email protected]" rel=3D"noreferrer" ta=
rget=3D"_blank">[email protected]</a><br>
<a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list=
" rel=3D"noreferrer noreferrer" target=3D"_blank">https://mail.gnome.org/ma=
ilman/listinfo/gnome-accessibility-list</a><br>
</blockquote></div></div>
_______________________________________________<br>
gnome-accessibility-list mailing list<br>
<a href=3D"mailto:[email protected]" rel=3D"noreferrer" ta=
rget=3D"_blank">[email protected]</a><br>
<a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list=
" rel=3D"noreferrer noreferrer" target=3D"_blank">https://mail.gnome.org/ma=
ilman/listinfo/gnome-accessibility-list</a><br>
</blockquote></div>
_______________________________________________<br>
gnome-accessibility-list mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">gno=
[email protected]</a><br>
<a href=3D"https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list=
" rel=3D"noreferrer" target=3D"_blank">https://mail.gnome.org/mailman/listi=
nfo/gnome-accessibility-list</a><br>
</blockquote></div>

--000000000000fa738405c37336a7--

--===============2219148818608780550==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
gnome-accessibility-list mailing list
[email protected]
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

--===============2219148818608780550==--