Wayland support

[vith]

Wayland is now the default display server on some popular distros such as Fedora 25.

It would be great if ActivityWatch worked on Wayland!

On v0.7.0b1 I just get this log line repeatedly:

2017-08-02 09:35:59,457 [WARNING]: Failed to find active window, id found was 0x0 (aw_watcher_window.lib:19)

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[ErikBjare]

I've been anticipating this... See this issue: ActivityWatch/aw-watcher-window#18

Unfortunately, I'm not sure how to solve this. It seems like we can't have Wayland-support since Wayland itself doesn't have any notion of active window.

Are you using Gnome? I'd be interested in getting this working on Gnome or some other large desktop environment with Wayland support to start with and then see how we should go from there.

Edit: It seems Gnome is deprecating their get_active_window API which only ever worked on X11. Will investigate further.

Edit 2: Done quite a few searches but have come up emptyhanded. Considering asking a question on Stack Exchange and/or sending a mail to some Gnome mailinglist.

[vith]

Ah, that does complicate things.

Yep, I'm using Gnome on Arch Linux actually. ActivityWatch is the only thing keeping me using the Gnome on Xorg session.

[ErikBjare]

Created a question on Stack Overflow, will try to find someone on IRC to help.

Edit: Also asked on the Gnome IRC, seems a bit quiet right now.

Edit: Has now been ~20h since I asked, no one replied to me in that channel. Might want to try a mailinglist further down the line.

[ErikBjare]

Alright, just asked in the #wayland channel on Freenode. Got a response at least, but the situation seems pretty bad.

15:20:44  ErikBjare    Hello everybody. I'm working on a piece of self-tracking software called ActivityWatch (https://github.com/ActivityWatch/activitywatch). I know this isn't exactly the right place to ask, but I was wondering if anyone knew anything about getting the active window in any Wayland-using DE.
15:20:57  ErikBjare    Created a question on SO: https://stackoverflow.com/questions/45465016/how-do-i-get-the-active-window-on-gnome-wayland
15:21:25  ErikBjare    Here's the issue in my repo for it: https://github.com/ActivityWatch/activitywatch/issues/92
15:22:54  ErikBjare    There are a bunch of other applications that depend on it (RescueTime, selfspy, arbtt, ulogme, etc.) so they'd need it as well
15:24:23  blocage      ErikBjare, in the core protocol you cannot know which windnow has the keyboard or cursor focus
15:24:39  blocage      ErikBjare, in the wayland core protocol *
15:25:10  blocage      ErikBjare, you can just know if your window has the focus or not, it a design choise
15:25:23  blocage      avoid client spying each other
15:25:25  ErikBjare    blocage: I'm aware, that's my reason for concern. I'm not saying it should be included or anything, but as it looks now every DE would need to implement it themselves if these kind of applications are to be supported
15:25:46  ErikBjare    So wondering if anyone knew the teams working with Wayland on Gnome for example
15:26:11  ErikBjare    But thanks for confirming
15:26:29  blocage      ErikBjare, DE should create a custom extension, or use D-bus or other IPC
15:27:31  blocage      ErikBjare, I guess some compositor are around here, but I do not know myself if there is such extension already
15:27:44  blocage      compositor developers *
15:28:36  ErikBjare    I don't think there is (I've done quite a bit of searching), so I guess I need to catch the attention of some DE developers
15:29:16  ErikBjare    Thanks a lot though
15:29:42  ErikBjare    blocage: Would you mind if I shared logs of our conversation in the issue?                                     
15:30:05  blocage      just use it :) it's public                                                                                               
15:30:19  ErikBjare    ty :)                                                                                                            

I guess our only option is to contact some DE developers and petition them to create a way to get the info.

[osule]

You may be able to get the active window from the root window.

• https://github.com/wayland-project/weston/blob/master/xwayland/window-manager.c#L2244
• https://developer.gnome.org/gdk3/stable/GdkScreen.html#gdk-screen-get-active-window

[ErikBjare]

@osule Unfortunately that won't work. That Gnome API is X11 only (that's why its deprecated) and that thing you found in weston is just xwayland.

Wayland has no concept of active window, and Gnome has no internal concept of it that it exposes. The reasons for this seems to be privacy as blocage mentioned in the IRC convo I pasted.

[ErikBjare]

Filed an issue on the bug tracker for Gnome's mutter.

[Awerick]

Hi @ErikBjare, I stumbled upon this issue while searching for a related problem.

Excuse me, in case this is completely off track but I just came across the "Wayland Security Module" ("WSM", see below for links) and thought, this might be interesting / relevant for this issue as well:
Its a proposal for a standardized, secure way to let programs perform "sensitive" tasks under Wayland (if the Desktop Environments/Wayland compositors would support the WSM).
Maybe long-term this could also be a solution for this issue of "getting the active window under Wayland".

More info on Wayland Security Module:
X.Org Presentation 2014 ; official GitHub repo ; Wikipedia: Wayland#WSM

[ErikBjare]

@Awerick Thanks a lot for reaching out!

WSM looks like a great initiative, and I had no idea it existed. It's probably the closest thing to a solution I've seen so far. Let's hope there is interest in it from the Wayland community, I might try prodding them in the future and see if they have considered our usecase 🙂

[Awerick]

Glad it was helpful!
I am actually not sure whether WSM is something that Wayland itself has to support or whether it is something "additional" to the Wayland protocol that the DEs would have to provide (e.g. weston as Waylands reference compositor implementation, Gnome's mutter etc.) …

[da2x]

So Mutter/GNOME support may be out, but have have you investigated KWin/Wayland (Plasma) support?

Update: The org.gnome.Mutter.ScreenCast API in libmutter + OCR could solve this for GNOME. Obviously at a terrible performance cost.

[johan-bjareholt]

@da2x I currently don't have a desktop with KWin/Wayland installed, but would love to try out in case it has some API that exposes window information (DBus?).

In the long run however we don't have enough manpower to write one window watcher for each Wayland compositor, so we will have to push for some standardized wayland protocol for exposing the information about the currently focused window.

[jancborchardt]

I am using GNOME (on Ubuntu) and while I’m mainly a designer, I could help if something needs to be tried – if I can get some pointers. :)

In the long run however we don't have enough manpower to write one window watcher for each Wayland compositor, so we will have to push for some standardized wayland protocol for exposing the information about the currently focused window.

Maybe it’s best to focus on GNOME first, cause it seems like a standardized way is more difficult. And GNOME is used by Ubuntu and thus makes it the most used desktop environment – which will solve it for a lot of people already such as @vith and myself.

[jancborchardt]

Another possible enhancement, while it might result in false positives, could be to adjust the output in the interface. So when ActivityWatch is running on Arch/Ubuntu/etc., we could kind-of assume that "Unknown" is a system applicaton and simply write "System app".

As this might be wrong, it could be further specified like "Unknown (likely system app)" or something like that. But it’s already better than a plain "Unknown".

[johan-bjareholt]

@jancborchardt There's a separate issue with more info in aw-watcher-window where we have discussed this a bit. TL;DR: there is a WIP branch of gnome-shell which has implemented a DBus API which exposes this and they are considering making it into a Freedesktop standard. Progress is slow though.

ActivityWatch/aw-watcher-window#18 (comment)

Another possible enhancement, while it might result in false positives, could be to adjust the output in the interface. So when ActivityWatch is running on Arch/Ubuntu/etc., we could kind-of assume that "Unknown" is a system applicaton and simply write "System app".
As this might be wrong, it could be further specified like "Unknown (likely system app)" or something like that. But it’s already better than a plain "Unknown".

I don't really agree with this. If the appname is not available and "Unknown" is used it only means that the application has not used APIs properly to expose its name to the display manager. There's really nothing that indicates what kind of app it is. Whether its more likely to be one type of app or another will likely vary a lot depending on the OS and the quality of the apps which the user uses so an assumption is very likely to be wrong.

[hughwilliams94]

Just to mention that swaywm (a tiling Wayland compositor), has a command which returns the focused window (swaymsg -t get_tree). The result of the command is a json formatted list of the currently open windows with a key boolean key for focused (and lots of other info)

[johan-bjareholt]

@hughwilliams94 Thanks for the heads up, that's pretty cool!
Might be fun to try just for prototyping, but as it's exclusive to sway it's probably never going to be used. We really need a proper Wayland API for this.

[johan-bjareholt]

Asked questions about this to wlroots devs on reddit and got this answer: https://www.reddit.com/r/linux/comments/as1dd0/we_are_the_sway_wlroots_developers_ask_us_anything/egskqq9

Apparently this interface can give us support for all wlroots based window managers which sounds like a decent solution.
https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-foreign-toplevel-management-unstable-v1.xml

[hughwilliams94]

Great to hear, the devs on sway and wlroots are very responsive and have been working well with the KDE gang. Hopefully this can help you to make some progress.

[johan-bjareholt]

@hughwilliams94 Yeah, they are very good at communicating with their users (in contrast to GNOME/KDE devs).

In the end we are likely not going to be using this solution, but it seems like a reliable base to start prototyping with.

[johan-bjareholt]

Just want to mention, if someone is using sway I have written a prototype watcher for that which works with a few flaws which you can read about in the README

https://github.com/activitywatch/aw-watcher-sway

[johan-bjareholt]

First of all the Gnome introspection code was merged: https://gitlab.gnome.org/GNOME/gnome-shell/merge_requests/326/diffs
So now it should be possible to write window detection to Gnome Shell under wayland. Still no afk detection though as far as I know.

Another good thing is that phosh (the Librem 5 compositor) has now implemented the wlr-foreign-toplevel-management protocol https://source.puri.sm/Librem5/phosh/commit/532cfaf085cd440c3f849e92da8c8d65681c2a9c

Another good thing is that sway got a PR to get wlr-foreign-toplevel-management implemented as well which means that we could get one watcher which could support both sway and phosh. It's not merged yet though, but it looks promising!
swaywm/sway#4476

[real-or-random]

If I understand correctly, this is an approach to support GNOME (on Wayland) in the future.

[okan-instrumentl]

For anyone looking, I'm using this since like 2 weeks and everything works perfectly.

ActivityWatch/aw-watcher-window#79 (comment)

[renyuneyun]

Just to give some information as a KDE user: Spectacle, the screenshot tool for KDE, has a mode to get screenshot of only the active window. Therefore, there must be a way for it to know which is the current active window. Maybe it's worth looking into its code.

This KDE bug report may be of relevance. It is resolved already.

I also found this reddit post which gave some hints. D-bus may be a place where the WM/DE expose such information? But unfortunately my exploration didn't find anything that shows which window is active.

[renyuneyun]

For KDE, you can get the current active window and everything through KWin script (dev doc).

For example, the following script prints the window title on changing the active window:

function printWindowTitle(client) {
    print(client.caption);
}

workspace.clientActivated.connect(printWindowTitle);

It will encounter issues with null. I guess this means switching window will result in activating null first and then the new window.

[Steinhagen]

Is there any update for this issue?

[real-or-random]

This works well on Wayland, last time I checked: https://github.com/flexagoon/aw-watcher-gnome (specific to GNOME of course)

@flexagoon Do you plan to PR this here?

[flexagoon]

@real-or-random what do you want me to PR? Since my watcher is GNOME specific, I don't think it can be merged into main AW repo. Although perhaps I could add a mention about it to the documentation, if that's allowed

[real-or-random]

Yeah sorry, my comment was a bit unspecific. And I have no idea what the official policy here is. :)

Although perhaps I could add a mention about it to the documentation, if that's allowed

Yes, it should be added to docs here: https://docs.activitywatch.net/en/latest/watchers.html That page specifically calls for additions to the list.

Since my watcher is GNOME specific, I don't think it can be merged into main AW repo.

Perhaps not merged in the main repo, but apparently some watchers contributed by external people live in the ActivityWatch GitHub organization. Perhaps that's desirable also in this case (only if you like the idea, of course!). The watcher is GNOME-specific, but GNOME is not at all a niche desktop. @ErikBjare What's your opinion on this?

[2e3s]

With the help of this and a few other threads, I have compiled all supported platforms and workarounds for Linux in a single binary executable https://github.com/2e3s/awatcher with a few bells and whistles to work on X11 and Wayland, including workarounds for KDE and Gnome (with the help of @flexagoon ). I've made 2 binaries:

• Just a idle+active window watcher to use with ActivityWatch distribution instead of the original Python watchers
• All-in-one with bundled aw-server-rust and StatusNotifier tray icon to be a standalone binary (which I prefer and use as a DEB package).

I tested all environments to some extent, except KDE+Wayland one 1st PC and KDE+X11 on 2nd PC which I use every day.

Screenshot

[Steinhagen]

Is there any plan to have this integrated as the default for aw?

[2e3s]

Here I asked about it in April. I experimented with Tauri (lighter Electron) for a day, but frankly it feels very clunky, bulky for such a background utility. If this direction will be chosen, I would use and maintain my slim app. Nevertheless, I can help to integrate Linux watchers into whatever, or transfer the repository if needed.

Anyway, not to wait for anything, in May I've already committed efforts into a standalone app which is usable. I'll do more polishing and maintenance.

[johan-bjareholt]

There is a new wayland protocol in the staging section called ext-foreign-toplevel-list[1] which allows clients to get all windows as well as their appid+title. This has of now only been implemented in a draft commit for the cosmic DE[2], hopefully more will follow. Compared to aw-watcher-window-wayland that uses the wlr-foreign-toplevel-management[3], this new protocol is more limited yet pretty close to what we need for activitywatch. The only thing missing is to be able to see which window is focused, which is being worked on in a new protocol called foreign-toplevel-state[4].

What is more convincing about these two protocols compared to wlr-foreign-toplevel-management is that they have an intention to get them into wayland-protocols. There are still two big drawbacks however that still makes it unlikely that we will be able to use these two protocols anytime soon. First is simply that most compositors probably won't implement them. Secondly is that these protocols will only be accesible from so called "priviliged" clients. Exactly how to make a client like aw-watcher-window "priviliged" is not really clear to me.

Regardless, I am happy to see that at least something is happening in the wayland ecosystem and it's good to see that we are not alone in missing this functionality. Thank you @i509VCB for working on this, if you need a pilot client implementation I'd be happy to help!

[1] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/staging/ext-foreign-toplevel-list/ext-foreign-toplevel-list-v1.xml
[2] pop-os/cosmic-comp#76
[3] https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-foreign-toplevel-management-unstable-v1.xml
[4] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/196

[i509VCB]

There is a new wayland protocol in the staging section called ext-foreign-toplevel-list[1] which allows clients to get all windows as well as their appid+title. This has of now only been implemented in a draft commit for the cosmic DE[2], hopefully more will follow.

I have had a plan to continue the extension protocols to make the base protocol worth implementing across the ecosystem. I suspect that the wlroots ecosystem will adopt this protocol once there is a justification beyond the base protocol to do so. My own personal compositor (Smithay based) is also an implementor but it's not exactly ready enough to be a test bed.

Compared to aw-watcher-window-wayland that uses the wlr-foreign-toplevel-management[3], this new protocol is more limited yet pretty close to what we need for activitywatch. The only thing missing is to be able to see which window is focused, which is being worked on in a new protocol called foreign-toplevel-state[4].

Now that I think about it there isn't really a way to ask what window has focus beyond an activated state. I think the activated state is probably how you'd want to use this. If you can do double check the foreign-toplevel-state protocol to make sure I didn't forget it (and make a review comment if I did)

What is more convincing about these two protocols compared to wlr-foreign-toplevel-management is that they have an intention to get them into wayland-protocols. There are still two big drawbacks however that still makes it unlikely that we will be able to use these two protocols anytime soon. First is simply that most compositors probably won't implement them. Secondly is that these protocols will only be accesible from so called "priviliged" clients. Exactly how to make a client like aw-watcher-window "priviliged" is not really clear to me.

The protocol says "compositor policy". Sadly this is about as precise as I could get with the current way Wayland protocols are.

I did talk to @Drakulix a bit about that and I think she said there will be a way to enable privileged protocol access in cosmic, but I'm not sure how fine grained that will be.

There is the proposed security context protocol for giving extra stuff to sandboxes, but that is also subject to providing a UX to enable the protocols we need which will still be compositor dependent.

Regardless, I am happy to see that at least something is happening in the wayland ecosystem and it's good to see that we are not alone in missing this functionality. Thank you @i509VCB for working on this, if you need a pilot client implementation I'd be happy to help!

I'd definitely be interested in that, I plan on making some small changes first to the foreign-toplevel-state protocol. I planned to make the state query and management in the same protocol, but seperately advertiseable by having two different extension and global objects.

[1] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/staging/ext-foreign-toplevel-list/ext-foreign-toplevel-list-v1.xml [2] pop-os/cosmic-comp#76 [3] https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-foreign-toplevel-management-unstable-v1.xml [4] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/196

[johan-bjareholt]

Now that I think about it there isn't really a way to ask what window has focus beyond an activated state. I think the activated state is probably how you'd want to use this. If you can do double check the foreign-toplevel-state protocol to make sure I didn't forget it (and make a review comment if I did)

Added a comment about it in the PR. Maybe activated isn't a perfect solution. We can discuss that there.

The protocol says "compositor policy". Sadly this is about as precise as I could get with the current way Wayland protocols are.

I did talk to Drakulix a bit about that and I think she said there will be a way to enable privileged protocol access in cosmic, but I'm not sure how fine grained that will be.

There is the proposed security context protocol for giving extra stuff to sandboxes, but that is also subject to providing a UX to enable the protocols we need which will still be compositor dependent.

The security context protocol seems to at least have quite a decent backing, ACKs from Gnome and implementation already done in flatpak and sway. While it is interesting how fine-grained it will be in the long term, my hope is that compositors can start having just a basic "on/off" switch for which apps are priviliged to begin with. My guess is that it will be hard to guess what those fine-grained controls will be before more priviliged protocols exist.

[hrqmonteiro]

Will this be finally solved or what? Not being able to use on Wayland is preventing me from using AW. Using X11 is a step back, literally.

Every distro will move to Wayland as default.

[flexagoon]

@hrqmonteiro you can already use AW on Wayland by installing Awatcher

[piegamesde]

@hrqmonteiro your rude complain won't accelerate the development. What you are doing is the equivalent of honking at a red traffic light.

[Humam-Hamdan]

Anything new about this? :)

[eerootsus]

Another possible enhancement, while it might result in false positives, could be to adjust the output in the interface. So when ActivityWatch is running on Arch/Ubuntu/etc., we could kind-of assume that "Unknown" is a system applicaton and simply write "System app".

As this might be wrong, it could be further specified like "Unknown (likely system app)" or something like that. But it’s already better than a plain "Unknown".

Just my 5 cents, as I started using the app today on PopOS on Wayland. Amazingly, the only "unknown" so far is Firefox, for which I have a browser extension. So in my case it would be very fair to assume, that unknown screen time comes from Firefox. Currently looking for a solution to do just that.

I of course could switch the watcher daemon as explained in the docs...

[Ghurir]

It could be better to have an API that allows users to manually report active windows, allowing them to add support for each of their systems themselves.
E.G. you could run
aw-watch-event focused "firefox.exe" "Wayland support · Issue #92 · ActivityWatch/activitywatch"
and activitywatch handles the rest. This way you can support all systems.

[flexagoon]

@Ghurir you mean this?

https://docs.activitywatch.net/en/latest/api.html