No module anymore & perfect zoom feature

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

No module anymore & perfect zoom feature

Samuel Thibault
Hello,

So, I also saw the removal of generic modules.

Unfortunately we currently need it for implementing perfect zoom feature
:)

The context is that visual-impaired users need magnification of the
desktop. Changing font sizes / dpi etc. have their limit, at some point
we need to just have a zoomed view of a piece of the screen. Currently
compiz' ezoom takes the piece of the screen, and magnify it to show it
on the screen, with obviously awful pixelization effects.

Our idea was very similar to gtk-vector-screenshot : instead of taking
the output as it is displayed on the screen, get a module loaded within
the application, with which ezoom can discuss to make the application
produce a magnified rendering of its window, which ezoom can then show
in the magnification glass, thus getting perfect zoom.

Without module loading, I don't know how to implement it :) Or perhaps
this could be added as an AT-SPI interface?

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Alejandro-2
Not too much time to get involved on this, but I will at least make the
obvious question:


On 26/02/18 11:49, Samuel Thibault wrote:

> Hello,
>
> So, I also saw the removal of generic modules.
>
> Unfortunately we currently need it for implementing perfect zoom feature
> :)
>
> The context is that visual-impaired users need magnification of the
> desktop. Changing font sizes / dpi etc. have their limit, at some point
> we need to just have a zoomed view of a piece of the screen. Currently
> compiz' ezoom takes the piece of the screen, and magnify it to show it
> on the screen, with obviously awful pixelization effects.
>
> Our idea was very similar to gtk-vector-screenshot : instead of taking
> the output as it is displayed on the screen, get a module loaded within
> the application, with which ezoom can discuss to make the application
> produce a magnified rendering of its window, which ezoom can then show
> in the magnification glass, thus getting perfect zoom.
>
> Without module loading, I don't know how to implement it :) Or perhaps
> this could be added as an AT-SPI interface?

Being added as an AT-SPI interface would depend on what API do you need.
You previously mention that you have a module that "discuss" with ezoom.
What that discussion involves? Are you able to express it as an API?

>
> Samuel
> _______________________________________________
> gnome-accessibility-devel mailing list
> [hidden email]
> https://mail.gnome.org/mailman/listinfo/gnome-accessibility-devel
>

_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Samuel Thibault
Hello,

Alejandro, on mar. 27 févr. 2018 08:11:26 +0100, wrote:
> On 26/02/18 11:49, Samuel Thibault wrote:
> > Without module loading, I don't know how to implement it :) Or perhaps
> > this could be added as an AT-SPI interface?
>
> Being added as an AT-SPI interface would depend on what API do you need.
> You previously mention that you have a module that "discuss" with ezoom.
> What that discussion involves? Are you able to express it as an API?

Basically it could be something like

Pixmap GtkComponent::render(int width, int height)

In gtk-vector-screenshot this boils down to creating a cairo
surface with cairo_xlib_surface_create_for_bitmap, calling
gtk_widget_draw(widget, cr), and returning the result to ezoom which can
render it as it wishes.

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: No module anymore & perfect zoom feature

Matthias Clasen-2
In reply to this post by Samuel Thibault
On Mon, Feb 26, 2018 at 5:49 AM, Samuel Thibault <[hidden email]> wrote:
Hello,

So, I also saw the removal of generic modules.

Unfortunately we currently need it for implementing perfect zoom feature
:)

The context is that visual-impaired users need magnification of the
desktop. Changing font sizes / dpi etc. have their limit, at some point
we need to just have a zoomed view of a piece of the screen. Currently
compiz' ezoom takes the piece of the screen, and magnify it to show it
on the screen, with obviously awful pixelization effects.

Our idea was very similar to gtk-vector-screenshot : instead of taking
the output as it is displayed on the screen, get a module loaded within
the application, with which ezoom can discuss to make the application
produce a magnified rendering of its window, which ezoom can then show
in the magnification glass, thus getting perfect zoom.

Without module loading, I don't know how to implement it :) Or perhaps
this could be added as an AT-SPI interface?


If it is a toolkit-level feature that is needed desktop-wide, it needs to be implemented in the toolkit proper, not added through the backdoor via a module. I know that this will require some rearchitecting and may not be super-easy, but I still believe that this is the right way forward.

_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: No module anymore & perfect zoom feature

Emmanuele Bassi
In reply to this post by Samuel Thibault
On 26 February 2018 at 17:49, Samuel Thibault
<[hidden email]> wrote:
> Hello,
>
> So, I also saw the removal of generic modules.
>
> Unfortunately we currently need it for implementing perfect zoom feature
> :)

I don't know what a "perfect zoom feature" is — but zooming on a
window should be part of the display server.

Having said that, we do have a magnifier inside GTK, used by the
Inspector. We could make that feature public, and improve it.

We definitely do not want to let people inject code into running applications.

Ciao,
 Emmanuele.

--
https://www.bassi.io
[@] ebassi [@gmail.com]
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: No module anymore & perfect zoom feature

Samuel Thibault
[Re-sending without the attached pictures, too big for the list]

Hello,

Emmanuele Bassi, on jeu. 01 mars 2018 14:42:27 +0700, wrote:

> On 26 February 2018 at 17:49, Samuel Thibault
> <[hidden email]> wrote:
> > Hello,
> >
> > So, I also saw the removal of generic modules.
> >
> > Unfortunately we currently need it for implementing perfect zoom feature
> > :)
>
> I don't know what a "perfect zoom feature" is —

Please compare the two examples on

https://people.debian.org/~sthibault/zoom-gimp.png
https://people.debian.org/~sthibault/zoom-perfect.png

zoom-gimp.png is the kind of zoom you can get with state-of-the-art
zooming heuristics. zoom-perfect.png is simply obtained by getting gtk
to redraw the window into a bigger pixmap.

> but zooming on a window should be part of the display server.

The display server can not invent information, at best it could
achieve the zoom-gimp.png result, which is really not enough for
visually-impaired people. Here I have only magnified a couple of times,
people quite often request for 10x-30x magnification.

Also, the control on zooming should really not implemented in the
server. Usually you'll also want color inversion or mangling, adding
position hints etc. I don't think freedesktop people will be happy to
see that added to the display server, so an external solution is needed,
currently implemented in Compiz (but lacking access to re-rendering on a
bigger pixmap).

> Having said that, we do have a magnifier inside GTK, used by the
> Inspector. We could make that feature public, and improve it.

Interesting.  Having mentioned adding the feature to AT-SPI, I'm
however interested in putting the interface there, so that not only GTK
application can benefit from it, but also Qt, etc. and GTK can just plug
its support into AT-SPI.

> We definitely do not want to let people inject code into running applications.

Ok :)

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: No module anymore & perfect zoom feature

Emmanuele Bassi
In reply to this post by Emmanuele Bassi

On Thu, 1 Mar 2018 at 20:48, Samuel Thibault <[hidden email]> wrote:
Hello,

Emmanuele Bassi, on jeu. 01 mars 2018 14:42:27 +0700, wrote:
> On 26 February 2018 at 17:49, Samuel Thibault
> <[hidden email]> wrote:
> > Hello,
> >
> > So, I also saw the removal of generic modules.
> >
> > Unfortunately we currently need it for implementing perfect zoom feature
> > :)
>
> I don't know what a "perfect zoom feature" is —

Please compare the two attached examples :)

zoom-gimp.png is the kind of zoom you can get with state-of-the-art
zooming heuristics. zoom-perfect.png is simply obtained by getting gtk
to redraw the window into a bigger pixmap.

> but zooming on a window should be part of the display server.

The display server can not invent information, at best it could
achieve the zoom-gimp.png result, which is really not enough for
visually-impaired people. Here I have only magnified a couple of times,
people quite often request for 10x-30x magnification.

The compositor can say to the toolkit to render with a different scaling factor - we do that for hidpi so toolkits already know how to deal with it. No need to do vector rendering; after all, things are rendered to pixmaps anyway, not using vector based API.


Also, the control on zooming should really not implemented in the
server. Usually you'll also want color inversion or mangling, adding
position hints etc. I don't think freedesktop people will be happy to
see that added to the display server, so an external solution is needed,
currently implemented in Compiz (but lacking access to re-rendering on a
bigger pixmap).

Don’t really agree at all. Considering that Compiz is mostly dead, and that the current GNOME Shell already has logic for zoom, color inversion, and other effects, it’s perfectly capable of dealing with these requirements. Targeting an obsolete graphics stack is not going to get you anywhere - especially for a new major version of the toolkit, where we don’t have legacy code.

Ciao,
 Emmanuele.


> Having said that, we do have a magnifier inside GTK, used by the
> Inspector. We could make that feature public, and improve it.

Interesting.  Having mentioned adding the feature to AT-SPI, I'm
however interested in putting the interface there, so that not only GTK
application can benefit from it, but also Qt, etc. and GTK can just plug
its support into AT-SPI.

> We definitely do not want to let people inject code into running applications.

Ok :)

Samuel
--

_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Samuel Thibault
Emmanuele Bassi, on jeu. 01 mars 2018 15:32:37 +0000, wrote:
>     The display server can not invent information, at best it could
>     achieve the zoom-gimp.png result, which is really not enough for
>     visually-impaired people. Here I have only magnified a couple of times,
>     people quite often request for 10x-30x magnification.
>
>
> The compositor can say to the toolkit to render with a different scaling factor
> - we do that for hidpi so toolkits already know how to deal with it.

Can it do so several times?  I mean, users like to have both the
magnified version and the non-magnified version.

> No need to do vector rendering; after all, things are rendered to
> pixmaps anyway, not using vector based API.

I'm not asking for vector rendering, but bigger pixmap rendering.

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Alex ARNAUD
In reply to this post by Emmanuele Bassi
Le 01/03/2018 à 16:32, Emmanuele Bassi a écrit :
> that the current GNOME Shell already has logic for zoom, color
> inversion, and other effects, it’s perfectly capable of dealing with
> these requirements.

You can enable the GNOME Shell zoom feature, zoom to the factor 10 and
tell me if it works without blurry effects. I'm visual-impaired with a
vision of 1/10 and I see embarrassing blurry effect. You can try the
ZoomText magnifier on Windows, it provides trial version and you'll see
a real perfect zoom.

Best regards,
Alex ARNAUD.
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Emmanuele Bassi
In reply to this post by Emmanuele Bassi

On Fri, 2 Mar 2018 at 00:03, Alex ARNAUD <[hidden email]> wrote:
Le 01/03/2018 à 16:32, Emmanuele Bassi a écrit :
> that the current GNOME Shell already has logic for zoom, color
> inversion, and other effects, it’s perfectly capable of dealing with
> these requirements.

You can enable the GNOME Shell zoom feature, zoom to the factor 10 and
tell me if it works without blurry effects. I'm visual-impaired with a
vision of 1/10 and I see embarrassing blurry effect. You can try the
ZoomText magnifier on Windows, it provides trial version and you'll see
a real perfect zoom.

Thanks.

I was not claiming that the Shell’s zoom is perfect; I’m saying that the Shell is where things need to be fixed, as it’s where things are implemented already. The shell does not currently ask the toolkit to render an area at a different scaling factor, but it could. The shell can also change the rendering pipeline to apply an effect like color inversion.

Ciao,
 Emmanuele.
--

_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Alex ARNAUD
Le 01/03/2018 à 21:27, Emmanuele Bassi a écrit :
> Thanks.
>
> I was not claiming that the Shell’s zoom is perfect; I’m saying that the
> Shell is where things need to be fixed, as it’s where things are
> implemented already. The shell does not currently ask the toolkit to
> render an area at a different scaling factor, but it could. The shell
> can also change the rendering pipeline to apply an effect like color
> inversion.

Do you mean you can change the scaling factor during a session?
Visual-impaired, including me, use the zoom in/zoom out shortcut all the
time, they don't stay on the same zoom level. When a colleague wants to
work with me, I change the zoom, enable it, disable it each minutes to
make our collaboration easier. The zoom in/zoom out shortcut seems
instant on the screen.

Best regards and thanks for your feedback on this important subjects to
make GTK usable for all,
Alex ARNAUD.
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Samuel Thibault
In reply to this post by Emmanuele Bassi
Hello,

Emmanuele Bassi, on jeu. 01 mars 2018 20:27:04 +0000, wrote:
> I was not claiming that the Shell’s zoom is perfect; I’m saying that the Shell
> is where things need to be fixed, as it’s where things are implemented already.

Well, that is for gnome.  Users shouldn't be tied with gnome just to get
good zoom of gtk applications, and we want good zoom of other widget
libraries too, so we need a window magnification interface which is
independent from gnome.

> The shell does not currently ask the toolkit to render an area at a different
> scaling factor, but it could. The shell can also change the rendering pipeline
> to apply an effect like color inversion.

But that means it'll only benefit the gnome shell, and not other
desktops.  Writing a *good* magnifier, which means not only just the
graphical effect, but also mouse tracking, focus tracking, smooth
transition, etc. is not something that we want to see having to be
implemented several times for several desktops, there are not enough
people who have the skills, context, and time to do that.

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Bastien Nocera
On Mon, 2018-03-05 at 15:05 +0100, Samuel Thibault wrote:

> Hello,
>
> Emmanuele Bassi, on jeu. 01 mars 2018 20:27:04 +0000, wrote:
> > I was not claiming that the Shell’s zoom is perfect; I’m saying
> > that the Shell
> > is where things need to be fixed, as it’s where things are
> > implemented already.
>
> Well, that is for gnome.  Users shouldn't be tied with gnome just to
> get
> good zoom of gtk applications, and we want good zoom of other widget
> libraries too, so we need a window magnification interface which is
> independent from gnome.

If you implement the feature in GTK+ and gnome-shell in a way that
makes either of those replaceable, you can then add support for non-
GTK+ apps, or non-gnome-shell compositors.

I don't see how you could implement it without the compositor knowing
about the feature. The compositor already knows how to modify what
appears on the screen, can modify on-screen data quickly (for all the
brightness, contrast, etc. changes), and using the GPU knows how to
scale.

> > The shell does not currently ask the toolkit to render an area at a
> > different
> > scaling factor, but it could. The shell can also change the
> > rendering pipeline
> > to apply an effect like color inversion.
>
> But that means it'll only benefit the gnome shell, and not other
> desktops.  Writing a *good* magnifier, which means not only just the
> graphical effect, but also mouse tracking, focus tracking, smooth
> transition, etc. is not something that we want to see having to be
> implemented several times for several desktops, there are not enough
> people who have the skills, context, and time to do that.

Perhaps, but you'd be doing a disservice to your users trying to
implement this as a "can run anywhere" solution. I don't think there's
any way you can implement this generically so that there's no work
needed on other desktops.

Implementing this in GTK+ and gnome-shell gives you the opportunity to
experiment with an interface that should hopefully be applicable to
other compositors, and compositing window managers.
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Samuel Thibault
Bastien Nocera, on lun. 05 mars 2018 15:21:47 +0100, wrote:
> Perhaps, but you'd be doing a disservice to your users trying to
> implement this as a "can run anywhere" solution. I don't think there's
> any way you can implement this generically so that there's no work
> needed on other desktops.
>
> Implementing this in GTK+ and gnome-shell gives you the opportunity to
> experiment with an interface that should hopefully be applicable to
> other compositors, and compositing window managers.

It looks like there is a misunderstanding.

I'm not saying that it has to be completely external to gtk and its
compositor. I'm saying that it has to be not only internal to gtk and
its compositor. The gnome desktop can be a testbed, sure, I'm just
saying that we have to keep in mind to have a generic interface that can
be implemented by other widget libraries, and used by other compositors.

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Alejandro-2


On 05/03/18 15:32, Samuel Thibault wrote:

> Bastien Nocera, on lun. 05 mars 2018 15:21:47 +0100, wrote:
>> Perhaps, but you'd be doing a disservice to your users trying to
>> implement this as a "can run anywhere" solution. I don't think there's
>> any way you can implement this generically so that there's no work
>> needed on other desktops.
>>
>> Implementing this in GTK+ and gnome-shell gives you the opportunity to
>> experiment with an interface that should hopefully be applicable to
>> other compositors, and compositing window managers.
> It looks like there is a misunderstanding.
>
> I'm not saying that it has to be completely external to gtk and its
> compositor. I'm saying that it has to be not only internal to gtk and
> its compositor. The gnome desktop can be a testbed, sure, I'm just
> saying that we have to keep in mind to have a generic interface that can
> be implemented by other widget libraries, and used by other compositors.

And related to misunderstandings and generic interfaces:

That generic interface is what I asked you, and you replied with a
really Gtk-tied API. I think that you misunderstood my question, and
replied with the API that your zoom application is using. But I will try
again.

Lets say that we have a zoom application. And a specific application
that you want to get zoomed. You originally asked if it would be
possible to add a generic API on at-spi to do that. So my question is:
do you have an approximate idea of that generic API to be included on
at-spi that any toolkit could implement?

Having said so, right now Im biased for what other people is already
saying. There are already one compositor with a zoom feature,
gnome-shell. It is true that  it is not "Zoom perfect", but it is really
well integrated with gnome libraries, included gtk+. So I would try to
check there first. After all, as Emmanuele said, the compositor can say
to the toolkit to render on different scaling factors (AFAIU, thats the
basis of hdpi). Although your question if if it possible to change the
scaling factor in-the-fly is good too.

FWIW, I'm not saying that we should force ourselves for that case. Just
saying that seems the best candidate to see how things should be done,
in order to have a good idea of how it could be expressed in a generic way.

BR

_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Reply | Threaded
Open this post in threaded view
|

Re: [g-a-devel] No module anymore & perfect zoom feature

Samuel Thibault
Alejandro, on mar. 06 mars 2018 09:35:01 +0100, wrote:

> On 05/03/18 15:32, Samuel Thibault wrote:
> > Bastien Nocera, on lun. 05 mars 2018 15:21:47 +0100, wrote:
> >> Perhaps, but you'd be doing a disservice to your users trying to
> >> implement this as a "can run anywhere" solution. I don't think there's
> >> any way you can implement this generically so that there's no work
> >> needed on other desktops.
> >>
> >> Implementing this in GTK+ and gnome-shell gives you the opportunity to
> >> experiment with an interface that should hopefully be applicable to
> >> other compositors, and compositing window managers.
> > It looks like there is a misunderstanding.
> >
> > I'm not saying that it has to be completely external to gtk and its
> > compositor. I'm saying that it has to be not only internal to gtk and
> > its compositor. The gnome desktop can be a testbed, sure, I'm just
> > saying that we have to keep in mind to have a generic interface that can
> > be implemented by other widget libraries, and used by other compositors.
>
> And related to misunderstandings and generic interfaces:
>
> That generic interface is what I asked you, and you replied with a
> really Gtk-tied API.

Again misunderstanding, sorry.  When I wrote

Pixmap GtkComponent::render(int width, int height)

Gtk slipped in from my fingers due to my work on libreoffice widgets.
I meant at-spi's Component, I was talking about an AT-SPI interface,
not something else.  I only mentioned gtk-vector-screenshot to explain
how it already worked with a gtk-ish interface, which we could try to
transpose to AT-SPI.

If only the world was letting us time to properly explain what we are
thinking...

> Lets say that we have a zoom application. And a specific application
> that you want to get zoomed. You originally asked if it would be
> possible to add a generic API on at-spi to do that. So my question is:
> do you have an approximate idea of that generic API to be included on
> at-spi that any toolkit could implement?

That was

Pixmap Component::render(int width, int height)

Basically, ask an at-spi Component to render itself with a given
width/height, and return a Pixmap.  Of course, how to transmit the
pixmap to the screen magnifier remains to be defined, but that was the
idea.

I agree that rather putting it in the compositor makes a lot of
technical sense, however, and it's easier to put screen magnification
there, my concern is about sharing the implementation.  Perhaps we could
make this a library that various compositors could use, I'm just a bit
afraid of the interface, not for the library to tell the compositor
what to do, but for the library to get the input it needs.  Currently
compiz' ezoom uses the mouse position only, but the extensions we are
experimenting with use at-spi's components positions and sizes, so it'd
mean that the compositor would end up talking with at-spi through that
library?

> FWIW, I'm not saying that we should force ourselves for that case. Just
> saying that seems the best candidate to see how things should be done,
> in order to have a good idea of how it could be expressed in a generic way.

Our concern is that while gnome is in general the most accessible
desktop thanks to these features, the rest of gnome poses a lot of
issues to our customers with really low vision or even blind, thus we
are currently sticking with MATE, it's hard to work on something which
we'll likely not use :)

Samuel
_______________________________________________
gtk-devel-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-devel-list