Mousetweaks: integrate into GNOME Shell or keep in its current form?
Magdalen Berns <[email protected]> Thu, 21 Nov 2013 00:18:25 +0000
| Newsgroups | gmane.comp.gnome.shell,gmane.comp.gnome.accessibility.devel |
|---|---|
| Message-ID | <1384993105.17603.12.camel@localhost> |
Hi all, Sorry for the epic email. I was speaking to mclasen and halfline about the mousetweaks discussion I started in a private email thread with the MT developers Gerd and Francesco and, rather than ccing people at random intervals it seems more appropriate to forward to the gnome-shell and accessibility lists so others can comment! Hope the thread below makes sense, but just to summarise the discussion it is about whether or not to build Mousetweaks into GNOME Shell as there were a couple of of considerations which need to be thrashed out before a decision can be arrived at. Unfortunately I had to paste the thread from different parts or bits would have got missed so let us know if there's anything too confusing to figure out from the text below and I can try to help. Magdalen Magdalen Berns <[email protected]> 22 Oct to Gerd, Francesco Hi Gerd, Francesco, How are you both doing? I wonder if you got my email about mousetweaks a while back? It may have got lost in the lists. Thanks, Magdalen Gerd Kohlberger 23 Oct to marmuta, me, Francesco Hi Magdalen, yeah I didn't see your mail until now. Sorry. Francesco and I haven't really talked about the future of mousetweaks since your gsoc mails. (congratulations by the way) I think porting mousetweaks in its current form to wayland is not going to work. To implement hover-click we would need 2 things: a) get global mouse-motion events and b) synthesize/fake mouse button events. A regular application can do neither under wayland. I can think of 2 ways to solve this, either move everything into gnome-shell as you suggest, or keep mousetweaks as a standalone daemon and use at-spi. at-spi has API for global mouse-events (AtspiEventListener + "mouse:*" events) and for faking mouse events (atspi_generate_mouse_event()), but both operations don't work under wayland at the moment. Until those things are implemented we can't start our port. Reimplementing mousetweaks in gnome-shell would have a lot of advantages. It would be easy to get mouse-events and the click-selection window could be integrated directly into the chrome. It's currently impossible to drag things around in the Activites overview, because you can't change the hover-click type from within the overview. Another advantage would be that we save a lot of code. Probably 90% of mousetweaks only deals with being a daemon and getting various events, the actual hover-click logic is only a tiny part. There is however one problem, if all the features move into gnome-shell then mousetweaks becomes a GNOME-only solution. At the moment mousetweaks works fine on Unity, lxde and even KDE. It can also be used as a service. For example the default on-screen keyboard on Ubuntu (onboard) integrates mousetweaks into its interface. You can activate hover-click and select the different click-types directly with the keyboard. I would actually prefer to just start working on a hoverclick.js component for gnome-shell (or you could Magdalen if you like), but loosing the onboard integration is a big problem. Francesco, what do you think? I'm guessing you're not even running gnome-shell. Cheers. CC'ing marmuta (main onboard developer) marmuta, what do you think about droping mousetweaks and also implementing hover-click? The at-spi dependency is already there and it might even be easier to do it ourselves than to do ipc with mousetweaks. It would at least resolve the issues with hiding the click-selection window. Magdalen Berns <[email protected]> 23 Oct to Gerd, Francesco, marmuta Hi Gerd, On 23 October 2013 09:20, Gerd Kohlberger <[email protected]> wrote: Hi Magdalen, yeah I didn't see your mail until now. Sorry. Francesco and I haven't really talked about the future of mousetweaks since your gsoc mails. (congratulations by the way) Thanks! I think porting mousetweaks in its current form to wayland is not going to work. To implement hover-click we would need 2 things: a) get global mouse-motion events and b) synthesize/fake mouse button events. A regular application can do neither under wayland. I can think of 2 ways to solve this, either move everything into gnome-shell as you suggest, or keep mousetweaks as a standalone daemon and use at-spi. atspi is now in GNOME Shell too but there are problems because it is making synchronous calls at the moment. at-spi has API for global mouse-events (AtspiEventListener + "mouse:*" events) and for faking mouse events (atspi_generate_mouse_event()), but both operations don't work under wayland at the moment. Until those things are implemented we can't start our port. Reimplementing mousetweaks in gnome-shell would have a lot of advantages. It would be easy to get mouse-events and the click-selection window could be integrated directly into the chrome. It's currently impossible to drag things around in the Activites overview, because you can't change the hover-click type from within the overview. Another advantage would be that we save a lot of code. Probably 90% of mousetweaks only deals with being a daemon and getting various events, the actual hover-click logic is only a tiny part. I will try this out. There is however one problem, if all the features move into gnome-shell then mousetweaks becomes a GNOME-only solution. At the moment mousetweaks works fine on Unity, lxde and even KDE. It can also be used as a service. For example the default on-screen keyboard on Ubuntu (onboard) integrates mousetweaks into its interface. You can activate hover-click and select the different click-types directly with the keyboard. This might be moving to gnome-shell as well. There was talk of it in GUADEC but I don't know what was decided. I would actually prefer to just start working on a hoverclick.js component for gnome-shell (or you could Magdalen if you like), but loosing the onboard integration is a big problem. Francesco, what do you think? I'm guessing you're not even running gnome-shell. Me too. Cheers. CC'ing marmuta (main onboard developer) marmuta, what do you think about droping mousetweaks and also implementing hover-click? The at-spi dependency is already there and it might even be easier to do it ourselves than to do ipc with mousetweaks. It would at least resolve the issues with hiding the click-selection window. Can I cc Alejandro into this thread as well? Francesco Fumanti 23 Oct to Gerd, me, marmuta Hi Magdalen, I am pleased to read that you are still interested in the accessibility features, and particularly in mousetweaks. On 2013-10-23 10:20, Gerd Kohlberger wrote: Hi Magdalen, yeah I didn't see your mail until now. Sorry. Francesco and I haven't really talked about the future of mousetweaks since your gsoc mails. (congratulations by the way) I think porting mousetweaks in its current form to wayland is not going to work. To implement hover-click we would need 2 things: a) get global mouse-motion events and b) synthesize/fake mouse button events. A regular application can do neither under wayland. I can think of 2 ways to solve this, either move everything into gnome-shell as you suggest, or keep mousetweaks as a standalone daemon and use at-spi. at-spi has API for global mouse-events (AtspiEventListener + "mouse:*" events) and for faking mouse events (atspi_generate_mouse_event()), but both operations don't work under wayland at the moment. Until those things are implemented we can't start our port. Reimplementing mousetweaks in gnome-shell would have a lot of advantages. It would be easy to get mouse-events and the click-selection window could be integrated directly into the chrome. It's currently impossible to drag things around in the Activites overview, because you can't change the hover-click type from within the overview. Another advantage would be that we save a lot of code. Probably 90% of mousetweaks only deals with being a daemon and getting various events, the actual hover-click logic is only a tiny part. There is however one problem, if all the features move into gnome-shell then mousetweaks becomes a GNOME-only solution. At the moment mousetweaks works fine on Unity, lxde and even KDE. It can also be used as a service. For example the default on-screen keyboard on Ubuntu (onboard) integrates mousetweaks into its interface. You can activate hover-click and select the different click-types directly with the keyboard. I would actually prefer to just start working on a hoverclick.js component for gnome-shell (or you could Magdalen if you like), but loosing the onboard integration is a big problem. Francesco, what do you think? I'm guessing you're not even running gnome-shell. You are right: I have not been running gnome-shell for many months. The advantages of going GNOME-only seem to be primarily from the point of view of the implementation. The dwell user would get tied to gnome-shell unless each desktop environment implements its own click accessibility features. This would be a clear regression for people relying on these features compared to the current situation. Assuming Wayland finally offers the things that the mousetweaks daemon needs to be able to work. Could mousetweaks not be enhanced that it checks which display server is running and run for that particular server if it is already supported by mousetweaks? (I suppose that MIR might have also to be considered in the near future.) The different desktop environments could (continue to) use the daemon similarly to how Onboard is doing it. Am I missing something, when I assume that gnome-shell could offer some sort of GUI that uses the mousetweaks daemon as a service? Going the at-spi2 way: The first incarnation of mousetweaks used the old at-spi until it was ported to X to remove the at-spi dependency. at-spi2 is a clear improvement in confront to the old at-spi. I see the advantage of using at-spi2, as (I assume) the daemon does not have to be adapted to the display servers. The code will probably be simpler and easier to maintain. But, what are the drawbacks compared to a daemon based on the display server? I don't have the necessary know-how to evaluate it, but I wonder whether it will work for example with all the toolkits. CC'ing marmuta (main onboard developer) marmuta, what do you think about droping mousetweaks and also implementing hover-click? The at-spi dependency is already there and it might even be easier to do it ourselves than to do ipc with mousetweaks. It would at least resolve the issues with hiding the click-selection window. Gerd, does this mean that the implementation of the ipc is more complicated than the implementation of the click features by using at-spi2? Moreover, this leads to the question of whether a mousetweaks based on at-spi2 makes sense at all, unless we want to avoid that everybody implements its own version of the click accessibility features. Cheers, Francesco PS: Gerd, could you please add some pieces of information to the mousetweaks section of the following page? https://wiki.gnome.org/Accessibility/Wayland marmuta 23 Oct to Francesco, Gerd, me Technically we probably could, at least until we move on with Onboard too and start targeting Wayland and/or MIR. I can't tell yet how much of this would survive the porting, same for all the other X tricks in Onboard. Assuming we did it, how would that work with Onboard in gnome-shell? There are two competing hover-click providers then, would we enable the gnome way in gnome-shell and fall-back to Onboard's implementation anywhere else? Wouldn't we then still need to be able to communicate with the new shell-mousetweaks? Magdalen Berns <[email protected]> 24 Oct to marmuta, Francesco, Gerd Hi all, A problem we have is that mousetweaks in its current form won't work under Wayland. Or at least I do not think so based on what I am finding out about it. The easiest way to make it work in gnome is to implement it into GNOME Shell but I do not know whether that could/should mean the standalone is abandoned altogether. If so that is not ideal. I do not see a reason to abandon mousetweaks as a standalone (other than lack of time and resources) but I do not know enough about the what work (if any) is currently needed for mousetweaks to continue to work in as a standalone outside of wayland. What can you all tell me about that? Kind regards, Magdalen marmuta 25 Oct to me, Francesco, Gerd Hi, > > A problem we have is that mousetweaks in its current form won't work > under Wayland. Or at least I do not think so based on what I am > finding out about it. I'm wondering, is there any way to implement mousetweaks at the evdev level? Events could probably be generated with /dev/uinput. > The easiest way to make it work in gnome is to > implement it into GNOME Shell but I do not know whether that > could/should mean the standalone is abandoned altogether. If so that > is not ideal. > > I do not see a reason to abandon mousetweaks as a standalone (other > than lack of time and resources) but I do not know enough about the > what work (if any) is currently needed for mousetweaks to continue to > work in as a standalone outside of wayland. What can you all tell me > about that? Only Gerd can answer that, but I guess once the main desktops switched away from X, there will be few incentives to keep maintaining it. Moving parts of it to Onboard then, as he suggested, may make sense, because for many desktops there is no other convenient way to enable mousetweaks anyway. Here's a thought, though: how about keeping standalone mousetweaks as a cross-desktop unified interface. It would provide the old functionality for X-based desktops, but only act as a proxy where desktop specific implementations are available, e.g. the shell implementation in GNOME shell. Would that make sense? > > > Kind regards, > Magdalen Best regards, M. Gerd Kohlberger 25 Oct to me, Francesco, marmuta On 23.10.2013 17:16, Magdalen Berns wrote: Hi Gerd, On 23 October 2013 09:20, Gerd Kohlberger <[email protected] <mailto:[email protected]>> wrote: Hi Magdalen, yeah I didn't see your mail until now. Sorry. Francesco and I haven't really talked about the future of mousetweaks since your gsoc mails. (congratulations by the way) Thanks! I think porting mousetweaks in its current form to wayland is not going to work. To implement hover-click we would need 2 things: a) get global mouse-motion events and b) synthesize/fake mouse button events. A regular application can do neither under wayland. I can think of 2 ways to solve this, either move everything into gnome-shell as you suggest, or keep mousetweaks as a standalone daemon and use at-spi. atspi is now in GNOME Shell too but there are problems because it is making synchronous calls at the moment. at-spi has API for global mouse-events (AtspiEventListener + "mouse:*" events) and for faking mouse events (atspi_generate_mouse_event())__, but both operations don't work under wayland at the moment. Until those things are implemented we can't start our port. Reimplementing mousetweaks in gnome-shell would have a lot of advantages. It would be easy to get mouse-events and the click-selection window could be integrated directly into the chrome. It's currently impossible to drag things around in the Activites overview, because you can't change the hover-click type from within the overview. Another advantage would be that we save a lot of code. Probably 90% of mousetweaks only deals with being a daemon and getting various events, the actual hover-click logic is only a tiny part. I will try this out. I found 2 things that don't work in the overview. You can't move windows between viewports and you can't rearrange the icons in the launcher. Both things aren't show-stoppers but it would be nice to have everything work. There is however one problem, if all the features move into gnome-shell then mousetweaks becomes a GNOME-only solution. At the moment mousetweaks works fine on Unity, lxde and even KDE. It can also be used as a service. For example the default on-screen keyboard on Ubuntu (onboard) integrates mousetweaks into its interface. You can activate hover-click and select the different click-types directly with the keyboard. This might be moving to gnome-shell as well. There was talk of it in GUADEC but I don't know what was decided. Do you mean hover-click controls in caribou? This could solve the overview problem because caribou can pop up there. I would actually prefer to just start working on a hoverclick.js component for gnome-shell (or you could Magdalen if you like), but loosing the onboard integration is a big problem. Francesco, what do you think? I'm guessing you're not even running gnome-shell. Me too. Cheers. CC'ing marmuta (main onboard developer) marmuta, what do you think about droping mousetweaks and also implementing hover-click? The at-spi dependency is already there and it might even be easier to do it ourselves than to do ipc with mousetweaks. It would at least resolve the issues with hiding the click-selection window. Can I cc Alejandro into this thread as well? Sure. Just cc people you think might be interested. Cheers. Gerd Kohlberger 25 Oct to marmuta, Francesco, me Yes that's my main concern. Most mousetweaks users I've had contact with were using Ubuntu and probably Unity. The advantages of going GNOME-only seem to be primarily from the point of view of the implementation. The dwell user would get tied to gnome-shell unless each desktop environment implements its own click accessibility features. This would be a clear regression for people relying on these features compared to the current situation. Yes. Assuming Wayland finally offers the things that the mousetweaks daemon needs to be able to work. Could mousetweaks not be enhanced that it checks which display server is running and run for that particular server if it is already supported by mousetweaks? (I suppose that MIR might have also to be considered in the near future.) The different desktop environments could (continue to) use the daemon similarly to how Onboard is doing it. If we only depend on atspi and not use any display server specific functions then we don't need different backends. The backends would be in the atspi registry. I don't know if there are any plans for a MIR backend in atspi. Am I missing something, when I assume that gnome-shell could offer some sort of GUI that uses the mousetweaks daemon as a service? No that would still be possible. Going the at-spi2 way: The first incarnation of mousetweaks used the old at-spi until it was ported to X to remove the at-spi dependency. at-spi2 is a clear improvement in confront to the old at-spi. I see the advantage of using at-spi2, as (I assume) the daemon does not have to be adapted to the display servers. The code will probably be simpler and easier to maintain. But, what are the drawbacks compared to a daemon based on the display server? I don't have the necessary know-how to evaluate it, but I wonder whether it will work for example with all the toolkits. The mouse-event parts of atspi are independent from the toolkit support so that shouldn't be a problem. I can think of 2 regressions. The cursor animations are X specific and have to be removed. I don't think it's possible to implement them under wayland or mir. It could work from within gnome-shell, the shell magnifier also manipulates the cursor sprite. But I also wouldn't mind loosing this feature, it never worked reliably even on X. I'm curious, do any of you see the cursor animations? When hover-click is activated the mouse cursor should fill up with a color before a click happens. The other thing are mouse wheel events. Currently mousetweaks stops the hover-click timer if the wheel is used. eg. if you scroll down on a web page with the wheel you don't want mousetweaks to randomly click somewhere in between. atspi only supports button 1-3. CC'ing marmuta (main onboard developer) marmuta, what do you think about droping mousetweaks and also implementing hover-click? The at-spi dependency is already there and it might even be easier to do it ourselves than to do ipc with mousetweaks. It would at least resolve the issues with hiding the click-selection window. Technically we probably could, at least until we move on with Onboard too and start targeting Wayland and/or MIR. I can't tell yet how much of this would survive the porting, same for all the other X tricks in Onboard. Onboard without xkb, xinput and xtest sounds almost impossible. Assuming we did it, how would that work with Onboard in gnome-shell? There are two competing hover-click providers then, would we enable the gnome way in gnome-shell and fall-back to Onboard's implementation anywhere else? Wouldn't we then still need to be able to communicate with the new shell-mousetweaks? I didn't think of that. Maybe always use onboard's and disable the other one, but that could get complicated fast, particularly if there are multiple other implementations. Gerd, does this mean that the implementation of the ipc is more complicated than the implementation of the click features by using at-spi2? hover-click is really just a timer. If the timer finishes we send a fake click, if the mouse is moved before that, we reset the timer and start again. If an app can already get global mouse events and synthesize clicks then this requires only a couple lines of code. Moreover, this leads to the question of whether a mousetweaks based on at-spi2 makes sense at all, unless we want to avoid that everybody implements its own version of the click accessibility features. Yes, but what I fear will happen is that nobody implements them. We can work on shell and onboard, but it would disappear everywhere else. With Canonical's CLA I certainly won't contribute anything to Ubuntu. Cheers. Gerd Kohlberger 25 Oct to marmuta, me, Francesco Hi, On 25.10.2013 02:17, marmuta wrote: Hi, A problem we have is that mousetweaks in its current form won't work under Wayland. Or at least I do not think so based on what I am finding out about it. I'm wondering, is there any way to implement mousetweaks at the evdev level? Events could probably be generated with /dev/uinput. evdev and uinput was my first thought as well. Unfortunately all the files are root:root, mousetweaks would have to be root or use some sort of suid helper. It's also complicated to use evdev directly, you need udev for PNP, figure out the device types yourself and you have to disconnect from all devices once a VT switch occurs. (requires talking to the DRM, again root only) Examples for that are in weston-launch.c and the weston evdev files. The easiest way to make it work in gnome is to implement it into GNOME Shell but I do not know whether that could/should mean the standalone is abandoned altogether. If so that is not ideal. I do not see a reason to abandon mousetweaks as a standalone (other than lack of time and resources) but I do not know enough about the what work (if any) is currently needed for mousetweaks to continue to work in as a standalone outside of wayland. What can you all tell me about that? mousetweaks uses a lot of X specific function calls throughout the code. All of those would have to be replaced with at-spi equivalents. Once this is done mousetweaks should work everywhere at-spi does, including wayland and maybe even MIR in the future. I'll start working on a port to atspi over the weekend to see if there are any unexpected problems. Only Gerd can answer that, but I guess once the main desktops switched away from X, there will be few incentives to keep maintaining it. Moving parts of it to Onboard then, as he suggested, may make sense, because for many desktops there is no other convenient way to enable mousetweaks anyway. Here's a thought, though: how about keeping standalone mousetweaks as a cross-desktop unified interface. It would provide the old functionality for X-based desktops, but only act as a proxy where desktop specific implementations are available, e.g. the shell implementation in GNOME shell. Would that make sense? Wouldn't this require that mousetweaks is always installed? It's installed by default on Ubuntu but I don't know what other distributions do. In a way gnome-settings-daemon already acts as that proxy. It monitors the 'dwell-click-enabled' key and launches mousetweaks. Maybe it could instead check if gnome-shell is running and use mousetweaks only as a fallback. Cheers. Kind regards, Magdalen Best regards, M. Francesco Fumanti 25 Oct to Gerd, marmuta, me Hi, I am probably wrong, but I interpreted marmuta's remark more broadly: By standalone mousetweaks, I understood a package containing the daemon and a standalone gui application that could start and control the daemon by itself; for example over dbus (I am assuming here, that the daemon can be started over dbus). Thus, people on distributions not offering mousetweaks, could install the package and make use of it through the provided gui application. And distributions wanting a better integration, could only use the daemon and offer their own graphical interface. Cheers, Francesco Magdalen Berns <[email protected]> 25 Oct to Piñeiro, Francesco, Gerd, marmuta cc Alejandro On 25 October 2013 13:59, Francesco Fumanti <[email protected]> wrote: Hi, On 2013-10-25 13:55, Gerd Kohlberger wrote: A problem we have is that mousetweaks in its current form won't work under Wayland. Or at least I do not think so based on what I am finding out about it. I'm wondering, is there any way to implement mousetweaks at the evdev level? Events could probably be generated with /dev/uinput. evdev and uinput was my first thought as well. Unfortunately all the files are root:root, mousetweaks would have to be root or use some sort of suid helper. It's also complicated to use evdev directly, you need udev for PNP, figure out the device types yourself and you have to disconnect from all devices once a VT switch occurs. (requires talking to the DRM, again root only) Examples for that are in weston-launch.c and the weston evdev files. The easiest way to make it work in gnome is to implement it into GNOME Shell but I do not know whether that could/should mean the standalone is abandoned altogether. If so that is not ideal. I do not see a reason to abandon mousetweaks as a standalone (other than lack of time and resources) but I do not know enough about the what work (if any) is currently needed for mousetweaks to continue to work in as a standalone outside of wayland. What can you all tell me about that? mousetweaks uses a lot of X specific function calls throughout the code. All of those would have to be replaced with at-spi equivalents. Once this is done mousetweaks should work everywhere at-spi does, including wayland and maybe even MIR in the future. I'll start working on a port to atspi over the weekend to see if there are any unexpected problems. Only Gerd can answer that, but I guess once the main desktops switched away from X, there will be few incentives to keep maintaining it. Moving parts of it to Onboard then, as he suggested, may make sense, because for many desktops there is no other convenient way to enable mousetweaks anyway. Here's a thought, though: how about keeping standalone mousetweaks as a cross-desktop unified interface. It would provide the old functionality for X-based desktops, but only act as a proxy where desktop specific implementations are available, e.g. the shell implementation in GNOME shell. Would that make sense? Wouldn't this require that mousetweaks is always installed? It's installed by default on Ubuntu but I don't know what other distributions do. In a way gnome-settings-daemon already acts as that proxy. It monitors the 'dwell-click-enabled' key and launches mousetweaks. Maybe it could instead check if gnome-shell is running and use mousetweaks only as a fallback. I am probably wrong, but I interpreted marmuta's remark more broadly: By standalone mousetweaks, I understood a package containing the daemon and a standalone gui application that could start and control the daemon by itself; for example over dbus (I am assuming here, that the daemon can be started over dbus). Thus, people on distributions not offering mousetweaks, could install the package and make use of it through the provided gui application. And distributions wanting a better integration, could only use the daemon and offer their own graphical interface. Cheers, Francesco Piñeiro 25 Oct to jdiggs, me, Francesco, Gerd, marmuta On 10/25/2013 05:18 PM, Magdalen Berns wrote: > cc Alejandro Hi, thanks for the forward. I have some comments, suggestions below. > > > On 25 October 2013 13:59, Francesco Fumanti <[email protected]> wrote: > > Hi, > > > On 2013-10-25 13:55, Gerd Kohlberger wrote: > > A problem we have is that mousetweaks in its current form won't work > under Wayland. Or at least I do not think so based on what I am > finding out about it. > > I'm wondering, is there any way to implement mousetweaks at the evdev > level? Events could probably be generated with /dev/uinput. > > > evdev and uinput was my first thought as well. Unfortunately all the files are root:root, mousetweaks would have to be root or use some sort of suid helper. It's also complicated to use evdev directly, you need udev for PNP, figure out the device types yourself and you have to disconnect from all devices once a VT switch occurs. (requires talking to the DRM, again root only) Examples for that are in weston-launch.c and the weston evdev files. > Well, although I don't have a lot of experience with Wayland, the feeling that I have is that the main idea is Wayland being more simple that X, and moving a lot of the functionality to the clients (example: window decorators now is a responsibilit of the client), but at the same time right now is really complex from the client side to implement some of the stuff that was possible before with X. As far as I understand that paragrah, before X exposed that functionality, so clients didn't have to talk directly to evdev/uinput, and now is not possible. > > The easiest way to make it work in gnome is to > implement it into GNOME Shell but I do not know whether that > could/should mean the standalone is abandoned altogether. If so that > is not ideal. > > I do not see a reason to abandon mousetweaks as a standalone (other > than lack of time and resources) but I do not know enough about the > what work (if any) is currently needed for mousetweaks to continue to > work in as a standalone outside of wayland. What can you all tell me > about that? > > > mousetweaks uses a lot of X specific function calls throughout the code. All of those would have to be replaced with at-spi equivalents. Once this is done mousetweaks should work everywhere at-spi does, including wayland and maybe even MIR in the future. I'll start working on a port to atspi over the weekend to see if there are any unexpected problems. > Well, that idea makes sense in the theory, but there are some problems: * On Wayland: there are still several questions about how to implement some of the current API on Wayland. For example, you mentioned some problems using evdev/uinput on mousetweaks. I foresee a similar problem with at-spi (that also runs as a user process). * On X: one of the outcomes of investigating how to reimplement some of the current functionality of at-spi2 on Wayland, is the conclusion that the usage of X on at-spi is obsolete. Needs and update, and probably some of the current functionality is not working. * at-spi2 itself: other of the outcomes of that investigation is realizing that at-spi2 API needs an update. Current API is confusing. Hopefully can be partially solved improving the documentation, but likely, it would be required to tweak the API itself. Any feedback would be welcome. Finally, FWIW, there are some running threads about Wayland and accessibility on gnome-accessibility-devel list and on the wayland-devel list itself [1]. Probably it would be good to collaborate publicly on those discussions, and see what the people with more experience on Wayland thinks. Thanks for all this information Best regards [1] http://lists.freedesktop.org/archives/wayland-devel/2013-October/011487.html marmuta 26 Oct to Gerd, Francesco, me I don't see the animation for the default arrow, but all other cursors, caret, resizing, etc. fill up with what looks like the orange selection color. > > The other thing are mouse wheel events. Currently mousetweaks stops > the hover-click timer if the wheel is used. eg. if you scroll down on > a web page with the wheel you don't want mousetweaks to randomly > click somewhere in between. atspi only supports button 1-3. > > >> > >>> CC'ing marmuta (main onboard developer) > >>> > >>> marmuta, what do you think about droping mousetweaks and also > >>> implementing hover-click? The at-spi dependency is already there > >>> and it might even be easier to do it ourselves than to do ipc with > >>> mousetweaks. It would at least resolve the issues with hiding the > >>> click-selection window. > > Technically we probably could, at least until we move on with > > Onboard too and start targeting Wayland and/or MIR. I can't tell > > yet how much of this would survive the porting, same for all the > > other X tricks in Onboard. > > Onboard without xkb, xinput and xtest sounds almost impossible. It won't be easy, I agree, but I suspect there will be alternatives, no way there won't be a replacement for xkb. xinput might not be essential, scanner and click generation could perhaps switch to GdkDevice. xtest could perhaps be replaced with uinput (daemon?). I have an old uinput branch of Onboard that kinda worked. If a rudimentary osk_uinput.c would help you anything with mousetweaks let me know. Else we could revisit at-spi2 for generating key-strokes, but there's no consistent separation between press and release, unfortunately. All the GUI stuff will hopefully just work, I had it start up in Weston a year ago or so, and it looked alright. > > > > Assuming we did it, how would that work with Onboard in gnome-shell? > > There are two competing hover-click providers then, would we enable > > the gnome way in gnome-shell and fall-back to Onboard's > > implementation anywhere else? Wouldn't we then still need to be > > able to communicate with the new shell-mousetweaks? > > I didn't think of that. Maybe always use onboard's and disable the > other one, but that could get complicated fast, particularly if there > are multiple other implementations. I think, if there indeed will be multiple implementations, we better yield to them in Onboard. No collisions that way and Onboard wouldn't have to deal with the platform's intricacies. Activities view and the locked down password dialog in GS come to mind. > > > >> > >> Gerd, does this mean that the implementation of the ipc is more > >> complicated than the implementation of the click features by using > >> at-spi2? > >> > > hover-click is really just a timer. If the timer finishes we send a > fake click, if the mouse is moved before that, we reset the timer and > start again. If an app can already get global mouse events and > synthesize clicks then this requires only a couple lines of code. > > > >> Moreover, this leads to the question of whether a mousetweaks based > >> on at-spi2 makes sense at all, unless we want to avoid that > >> everybody implements its own version of the click accessibility > >> features. > > Yes, but what I fear will happen is that nobody implements them. We > can work on shell and onboard, but it would disappear everywhere > else. With Canonical's CLA I certainly won't contribute anything to > Ubuntu. > > Cheers. > Cheers, M. Gerd Kohlberger 28 Oct to marmuta, Francesco, me Thanks. It's the same here. The other thing are mouse wheel events. Currently mousetweaks stops the hover-click timer if the wheel is used. eg. if you scroll down on a web page with the wheel you don't want mousetweaks to randomly click somewhere in between. atspi only supports button 1-3. CC'ing marmuta (main onboard developer) marmuta, what do you think about droping mousetweaks and also implementing hover-click? The at-spi dependency is already there and it might even be easier to do it ourselves than to do ipc with mousetweaks. It would at least resolve the issues with hiding the click-selection window. Technically we probably could, at least until we move on with Onboard too and start targeting Wayland and/or MIR. I can't tell yet how much of this would survive the porting, same for all the other X tricks in Onboard. Onboard without xkb, xinput and xtest sounds almost impossible. It won't be easy, I agree, but I suspect there will be alternatives, no way there won't be a replacement for xkb. xinput might not be essential, scanner and click generation could perhaps switch to GdkDevice. xtest could perhaps be replaced with uinput (daemon?). I have an old uinput branch of Onboard that kinda worked. If a rudimentary osk_uinput.c would help you anything with mousetweaks let me know. How did you open fd? I'm getting gerd@box01:~$ ll /dev/uinput crw------- 1 root root 10, 223 Okt 21 17:34 /dev/uinput Else we could revisit at-spi2 for generating key-strokes, but there's no consistent separation between press and release, unfortunately. All the GUI stuff will hopefully just work, I had it start up in Weston a year ago or so, and it looked alright. Assuming we did it, how would that work with Onboard in gnome-shell? There are two competing hover-click providers then, would we enable the gnome way in gnome-shell and fall-back to Onboard's implementation anywhere else? Wouldn't we then still need to be able to communicate with the new shell-mousetweaks? I didn't think of that. Maybe always use onboard's and disable the other one, but that could get complicated fast, particularly if there are multiple other implementations. I think, if there indeed will be multiple implementations, we better yield to them in Onboard. No collisions that way and Onboard wouldn't have to deal with the platform's intricacies. Activities view and the locked down password dialog in GS come to mind. ok. Gerd Kohlberger 28 Oct to Piñeiro, me, Francesco, marmuta, jdiggs Hi Alejandro, On 25.10.2013 18:24, Piñeiro wrote: On 10/25/2013 05:18 PM, Magdalen Berns wrote: cc Alejandro Hi, thanks for the forward. I have some comments, suggestions below. On 25 October 2013 13:59, Francesco Fumanti <[email protected] <mailto:[email protected]>> wrote: Hi, On 2013-10-25 13:55, Gerd Kohlberger wrote: A problem we have is that mousetweaks in its current form won't work under Wayland. Or at least I do not think so based on what I am finding out about it. I'm wondering, is there any way to implement mousetweaks at the evdev level? Events could probably be generated with /dev/uinput. evdev and uinput was my first thought as well. Unfortunately all the files are root:root, mousetweaks would have to be root or use some sort of suid helper. It's also complicated to use evdev directly, you need udev for PNP, figure out the device types yourself and you have to disconnect from all devices once a VT switch occurs. (requires talking to the DRM, again root only) Examples for that are in weston-launch.c and the weston evdev files. Well, although I don't have a lot of experience with Wayland, the feeling that I have is that the main idea is Wayland being more simple that X, and moving a lot of the functionality to the clients (example: window decorators now is a responsibilit of the client), but at the same time right now is really complex from the client side to implement some of the stuff that was possible before with X. As far as I understand that paragrah, before X exposed that functionality, so clients didn't have to talk directly to evdev/uinput, and now is not possible. On X the XTest extension makes it easy to synthesize mouse events and you can poll for the mouse position and button state. The code in mousetweaks to do that is exactly the same as in the at-spi-registryd. At the time we didn't want to use at-spi because you had to do the enable a11y/restart the session dance and it seemed better to just rewrite those parts. But now that a11y is always-on it's just code duplication. The easiest way to make it work in gnome is to implement it into GNOME Shell but I do not know whether that could/should mean the standalone is abandoned altogether. If so that is not ideal. I do not see a reason to abandon mousetweaks as a standalone (other than lack of time and resources) but I do not know enough about the what work (if any) is currently needed for mousetweaks to continue to work in as a standalone outside of wayland. What can you all tell me about that? mousetweaks uses a lot of X specific function calls throughout the code. All of those would have to be replaced with at-spi equivalents. Once this is done mousetweaks should work everywhere at-spi does, including wayland and maybe even MIR in the future. I'll start working on a port to atspi over the weekend to see if there are any unexpected problems. Well, that idea makes sense in the theory, but there are some problems: * On Wayland: there are still several questions about how to implement some of the current API on Wayland. For example, you mentioned some problems using evdev/uinput on mousetweaks. I foresee a similar problem with at-spi (that also runs as a user process). * On X: one of the outcomes of investigating how to reimplement some of the current functionality of at-spi2 on Wayland, is the conclusion that the usage of X on at-spi is obsolete. Needs and update, and probably some of the current functionality is not working. mousetweaks only uses 2 at-spi functions, the generateMouseEvent method and an EventListener to listen for "mouse:*" events. Does this mean that those things will no longer work on X once wayland support is added? I've now uploaded a new 'wayland' branch to git. On X everything seems to work as before but on wayland mousetweaks just sits there and waits for at-spi events that never arrive. And I've also updated the wayland accessibility page. Hope that's ok. https://git.gnome.org/browse/mousetweaks/tree/?h=wayland https://wiki.gnome.org/Accessibility/Wayland * at-spi2 itself: other of the outcomes of that investigation is realizing that at-spi2 API needs an update. Current API is confusing. Hopefully can be partially solved improving the documentation, but likely, it would be required to tweak the API itself. Any feedback would be welcome. Finally, FWIW, there are some running threads about Wayland and accessibility on gnome-accessibility-devel list and on the wayland-devel list itself [1]. Probably it would be good to collaborate publicly on those discussions, and see what the people with more experience on Wayland thinks. Ok, thanks for the info. Cheers, Gerd marmuta 28 Oct to Gerd, Francesco, me I probably just changed permissions to 666. That code was for investigating lost key-strokes back then, not really meant to be shipped. However, when cornered in Wayland/MIR, with uinput the only way out, I would consider building a uinput daemon that opens the device as root. Piñeiro 28 Oct to Gerd, me, Francesco, marmuta, jdiggs Hi, On 10/28/2013 11:35 AM, Gerd Kohlberger wrote: > Hi Alejandro, > >> Well, although I don't have a lot of experience with Wayland, the >> feeling that I have is >> that the main idea is Wayland being more simple that X, and moving a >> lot of the >> functionality to the clients (example: window decorators now is a >> responsibilit of the >> client), but at the same time right now is really complex from the >> client side to >> implement some of the stuff that was possible before with X. As far >> as I understand that >> paragrah, before X exposed that functionality, so clients didn't have >> to talk directly to >> evdev/uinput, and now is not possible. > > On X the XTest extension makes it easy to synthesize mouse events and > you can poll for the mouse position and button state. The code in > mousetweaks to do that is exactly the same as in the at-spi-registryd. > At the time we didn't want to use at-spi because you had to do the > enable a11y/restart the session dance and it seemed better to just > rewrite those parts. But now that a11y is always-on it's just code > duplication. During GUADEC, Keith Packard, a X developer, attended the accessibility BOF. He mentioned that using XTest to synthesize mouse events is ok, but not the best for tracking mouse position. Basically due the need of polling. He mentioned that it would be better to use recent additions to X, specifically xinput2. He event bloggued about that: http://keithp.com/blogs/Cursor_tracking/ Anyway, it is true that if mousetweaks were doing the same that at-spi2, I agree that just using at-spi2 would be better, to avoid replicate code. >> >> Well, that idea makes sense in the theory, but there are some problems: >> * On Wayland: there are still several questions about how to >> implement some of the >> current API on Wayland. For example, you mentioned some problems >> using evdev/uinput on >> mousetweaks. I foresee a similar problem with at-spi (that also runs >> as a user process). >> * On X: one of the outcomes of investigating how to reimplement >> some of the current >> functionality of at-spi2 on Wayland, is the conclusion that the usage >> of X on at-spi is >> obsolete. Needs and update, and probably some of the current >> functionality is not working. > > mousetweaks only uses 2 at-spi functions, the generateMouseEvent > method and an EventListener to listen for "mouse:*" events. Does this > mean that those things will no longer work on X once wayland support > is added? Sorry if I didn't explain myself properly. What I mentioned about X doesn't have any relation with adding wayland. What I wanted to say is that some of the code is using somewhat old X methods that it would be to replace (ie, with some xinput2 stuff). But we don't foresee any regression there. > > I've now uploaded a new 'wayland' branch to git. On X everything seems > to work as before Good to know. As I said, although ideally we would like to update the code that depends on X, with the proper X methods, we will not do that if that means a regression. If we do that, we will ping you. > but on wayland mousetweaks just sits there and waits for at-spi events > that never arrive. Well, the problem is that on at-spi2 there are not a equivalent implementation relying on wayland. Basically, we are on a situation of not being sure how to do that using just wayland methods, with something similar to what you commented on previous emails. > And I've also updated the wayland accessibility page. Hope that's ok. Yes it is ok, and in fact, welcomed. Thanks for the update. BR _______________________________________________ gnome-shell-list mailing list [email protected] https://mail.gnome.org/mailman/listinfo/gnome-shell-list