Async HTTP download with Glib event loop doesn't work on windows

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Async HTTP download with Glib event loop doesn't work on windows

juhaszp
Dear Gtk-Perl experts,

We have a legacy Gtk2 application that occasionally has to send HTTP requests asynchronously.
We've noticed that it hangs on Windows with recent perls.

The issue can be reduced to the following simple test case:

########################
#!/usr/bin/perl

use strict;
use warnings;
use feature qw/say/;
#use Gtk2 -init;

use AnyEvent;
use AnyEvent::DNS;
use AnyEvent::Socket;
use AnyEvent::Handle;
use AnyEvent::Impl::Glib;
#use AnyEvent::Impl::EV;
#use AnyEvent::Impl::Perl;

my $cv = AnyEvent->condvar;

tcp_connect #'localhost', 8000,
            "www.google.com", "http",
      sub {
         my ($fh) = @_
            or die "unable to connect: $!";
 
         my $handle; # avoid direct assignment so on_eof has it in scope.
         $handle = new AnyEvent::Handle
            fh     => $fh,
            on_error => sub {
               AE::log error => $_[2];
               $_[0]->destroy;
            },
            on_eof => sub {
               $handle->destroy; # destroy handle
               AE::log info => "Done.";
               $cv->send;
            };
 
         $handle->push_write ("GET / HTTP/1.0\015\012\015\012");
 
         $handle->push_read (line => "\015\012\015\012", sub {
            my ($handle, $line) = @_;
 
            # print response header
            print "HEADER\n$line\n\nBODY\n";
 
            $handle->on_read (sub {
               # print response body
               print $_[0]->rbuf;
               $_[0]->rbuf = "";
            });
           
            });
      };

$cv->recv;
#Gtk2->main();

########################

This program (taken almost verbatim from the AnyEvent::Socket documentation) works as expected on linux (downloads and prints the Google homepage), and breaks on Windows.
The reported error is sometimes a Broken Pipe, sometimes it is the following:

A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.

(This apparently corresponds to the error code WSANOTCONN)

If I use a webserver on localhost as the target to avoid any DNS requests, I (seemingly) always get a broken pipe.

And here is the main point, which is why this report belongs on this mailing list: the problem only appears if I explicitly select the Glib event loop implementation. The pure perl or EV backends work even on Windows.

Affected version numbers:

Windows 7 (Version 6.1 Build 7601 Service Pack 1)
perl 5.24.1 (ActiveState)
Glib 1.305 (from the sisyphusion.tk PPM repo advertised here a few weeks ago)
AnyEvent 7.13

Finally, my question:
is there any hope that this issue will be fixed, or at least its cause be found, or should we abandon this approach entirely?

best regards,
Peter Juhasz


_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

Brian Manning-2
On Wed, Mar 8, 2017 at 4:58 AM, Peter Juhasz <[hidden email]> wrote:
> Dear Gtk-Perl experts,
>
> We have a legacy Gtk2 application that occasionally has to send HTTP
> requests asynchronously.
> We've noticed that it hangs on Windows with recent perls.

What was the last versions of Perl/Gtk2/Glib/AnyEvent that worked for
you on Windows?

> Finally, my question:
> is there any hope that this issue will be fixed, or at least its cause be
> found, or should we abandon this approach entirely?

I'm going to guess that you're the only person trying to do this; you
may want to take that into consideration.

Thanks,

Brian
_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

juhaszp
On Thu, 2017-03-09 at 08:32 -0800, Brian Manning wrote:

> On Wed, Mar 8, 2017 at 4:58 AM, Peter Juhasz <[hidden email]> wrote:
> > Dear Gtk-Perl experts,
> >
> > We have a legacy Gtk2 application that occasionally has to send HTTP
> > requests asynchronously.
> > We've noticed that it hangs on Windows with recent perls.
>
> What was the last versions of Perl/Gtk2/Glib/AnyEvent that worked for
> you on Windows?
>

I have an old Vista VM here from around 2009 with Camelbox* installed:
perl 5.10.0
Glib.pm 1.221
Gtk2.pm 1.220
AnyEvent.pm 7.13
glib 2.20.0

This combination seems to work.

I also have anecdotal evidence from my colleague that our application
works with perl 5.12, I don't know the version numbers for the rest of
the modules though.

In theory, the next steps would be bisecting the Perl and Glib versions
until the change that broke our use case can be identified, but in
practice it is quite hard and time consuming to assemble even one
working combination of Perl/Gtk2 on Windows.

Which is why I liked to use *Camelbox in the past. It delivered exactly
what it promised: a pain-free way to install Perl with Gtk2 and
everything else needed on Windows. Pity it was never updated after a
while.

> > Finally, my question:
> > is there any hope that this issue will be fixed, or at least its cause be
> > found, or should we abandon this approach entirely?
>
> I'm going to guess that you're the only person trying to do this; you
> may want to take that into consideration.
>

I do realize that the world has moved on and Gtk2-Perl is obsolete, but
we have to support our legacy application, and we'd like to upgrade to
the latest and greatest Perl version for its performance and syntax
enhancements, but we can't because of this bug.

BTW there are multiple relevant-looking bugs mentioned in glib's source
in https://github.com/GNOME/glib/blob/master/glib/giowin32.c , perhaps
it'd worth to report this issue to them so that they know that there is
still something wrong with glib's socket support on Windows, if not
anything else.

> Thanks,
>
> Brian

thanks for the reply in any case,
Peter Juhasz


_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

juhaszp
In reply to this post by Brian Manning-2
The plot thickens.

I've got a very helpful tip about Berrybrew ( https://github.com/stevieb9/berrybrew ), which is a Windows analog to Perlbrew, allowing multiple versions of Strawberry Perl to coexist on the same system. I've installed Glib (1.305), Gtk2 (1.2498), AnyEvent (7.13), AnyEvent::HTTP (2.23) on all versons between 5.12 and 5.24, and ran the test script on all of them. 5.16.3 and below works, 5.18.4 and above fails.

The following test program demonstrates the issue even more dramatically:
##################
#!/usr/bin/perl

use strict;
use warnings;
use feature qw/say/;
use Gtk2 -init;

use AnyEvent;
use AnyEvent::HTTP;

my $w = Gtk2::Window->new();
my $l = Gtk2::Label->new();

$l->set_text("init");
$w->add($l);
$w->show_all();

my $wa; $wa = AnyEvent->timer(after => 2.0, interval => 2.0, cb => sub {
    http_request('GET',
        'http://www.fsz.bme.hu/cgi-bin/ltime',
        sub {
                my ($response_body, $response_headers) = @_;
                $l->set_text($response_body);
        }
    );

    $l->set_text("ready");
});


Gtk2->main();
#######################

On Perl 5.16.3 and below it fetches the data from the network and displays it in a window, on 5.18.4 and above it hangs.

I've looked at the perldelta docs for 5.18 but nothing obvious stood out.

Now what?

Peter


On Thu, Mar 9, 2017 at 5:32 PM, Brian Manning <[hidden email]> wrote:
On Wed, Mar 8, 2017 at 4:58 AM, Peter Juhasz <[hidden email]> wrote:
> Dear Gtk-Perl experts,
>
> We have a legacy Gtk2 application that occasionally has to send HTTP
> requests asynchronously.
> We've noticed that it hangs on Windows with recent perls.

What was the last versions of Perl/Gtk2/Glib/AnyEvent that worked for
you on Windows?

> Finally, my question:
> is there any hope that this issue will be fixed, or at least its cause be
> found, or should we abandon this approach entirely?

I'm going to guess that you're the only person trying to do this; you
may want to take that into consideration.

Thanks,

Brian


_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

Emmanuele Bassi
Without knowing how AnyEvent works, if the handler involves threads
then calling *any* GTK+ API from different threads than the one that
called `gtk_main()` is going to blow up your application. Even if it
worked in one release, during a specific alignment of planets, or if
you placed the machines in a pentacle drawn in goat blood, there's no
guarantee that it'll keep working. GTK+ simply does *not* support
multi-threaded access of the windowing system resources.

GTK+ and the G* platform has a whole bunch of API to deal with these
cases, like GTask[0]. Sadly, it's not very Perlish because nobody
wrote introspection overrides for it. Additionally, for things like
HTTP requests, there's a whole library called libsoup[1] which
integrates with the GLib main loop.

Ciao,
 Emmanuele.

[0]: https://developer.gnome.org/gio/stable/GTask.html
[1]: https://developer.gnome.org/libsoup/stable/

On 10 March 2017 at 15:27, Peter Juhasz <[hidden email]> wrote:

> The plot thickens.
>
> I've got a very helpful tip about Berrybrew (
> https://github.com/stevieb9/berrybrew ), which is a Windows analog to
> Perlbrew, allowing multiple versions of Strawberry Perl to coexist on the
> same system. I've installed Glib (1.305), Gtk2 (1.2498), AnyEvent (7.13),
> AnyEvent::HTTP (2.23) on all versons between 5.12 and 5.24, and ran the test
> script on all of them. 5.16.3 and below works, 5.18.4 and above fails.
>
> The following test program demonstrates the issue even more dramatically:
> ##################
> #!/usr/bin/perl
>
> use strict;
> use warnings;
> use feature qw/say/;
> use Gtk2 -init;
>
> use AnyEvent;
> use AnyEvent::HTTP;
>
> my $w = Gtk2::Window->new();
> my $l = Gtk2::Label->new();
>
> $l->set_text("init");
> $w->add($l);
> $w->show_all();
>
> my $wa; $wa = AnyEvent->timer(after => 2.0, interval => 2.0, cb => sub {
>     http_request('GET',
>         'http://www.fsz.bme.hu/cgi-bin/ltime',
>         sub {
>                 my ($response_body, $response_headers) = @_;
>                 $l->set_text($response_body);
>         }
>     );
>
>     $l->set_text("ready");
> });
>
>
> Gtk2->main();
> #######################
>
> On Perl 5.16.3 and below it fetches the data from the network and displays
> it in a window, on 5.18.4 and above it hangs.
>
> I've looked at the perldelta docs for 5.18 but nothing obvious stood out.
>
> Now what?
>
> Peter
>
>
> On Thu, Mar 9, 2017 at 5:32 PM, Brian Manning <[hidden email]> wrote:
>>
>> On Wed, Mar 8, 2017 at 4:58 AM, Peter Juhasz <[hidden email]>
>> wrote:
>> > Dear Gtk-Perl experts,
>> >
>> > We have a legacy Gtk2 application that occasionally has to send HTTP
>> > requests asynchronously.
>> > We've noticed that it hangs on Windows with recent perls.
>>
>> What was the last versions of Perl/Gtk2/Glib/AnyEvent that worked for
>> you on Windows?
>>
>> > Finally, my question:
>> > is there any hope that this issue will be fixed, or at least its cause
>> > be
>> > found, or should we abandon this approach entirely?
>>
>> I'm going to guess that you're the only person trying to do this; you
>> may want to take that into consideration.
>>
>> Thanks,
>>
>> Brian
>
>
>
> _______________________________________________
> gtk-perl-list mailing list
> [hidden email]
> https://mail.gnome.org/mailman/listinfo/gtk-perl-list
>



--
https://www.bassi.io
[@] ebassi [@gmail.com]
_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

juhaszp

On Fri, Mar 10, 2017 at 4:38 PM, Emmanuele Bassi <[hidden email]> wrote:
Without knowing how AnyEvent works, if the handler involves threads
then calling *any* GTK+ API from different threads than the one that
called `gtk_main()` is going to blow up your application. Even if it
worked in one release, during a specific alignment of planets, or if
you placed the machines in a pentacle drawn in goat blood, there's no
guarantee that it'll keep working. GTK+ simply does *not* support
multi-threaded access of the windowing system resources.


It does not do anything like that. It only uses Glib::Timeout, Glib::IO, Glib::Idle watchers in fairly standard ways.
 
GTK+ and the G* platform has a whole bunch of API to deal with these
cases, like GTask[0]. Sadly, it's not very Perlish because nobody
wrote introspection overrides for it. Additionally, for things like
HTTP requests, there's a whole library called libsoup[1] which
integrates with the GLib main loop.


We're considering libsoup, but it depends on Glib::Object::Introspection for which there are no windows builds (that we know of).
 
Ciao,
 Emmanuele.


thanks,
Peter


_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

Daniel Kasak-4
I have some old(ish) binaries of Glib::Object::Introspection for Windows. Yes they were a major pain to build. I'd usually point you to my blog ( http://tesla.duckdns.org ) - but it's offline at the moment ... we're renovating the house and my ADLS2, home server, and other things are totally out of action at the moment. If you're interested, I can go and get the server and rescue stuff off it. I also have build instructions that worked when they were written ( 18 months ago ). I tried repeating the process a couple of months back, and there are unfortunately major issues again, and I've kinda lost interest in building for Windows. Anyway, me know if you want Windows and/or build instructions.

Dan

On Sat, Mar 11, 2017 at 3:16 AM, Peter Juhasz <[hidden email]> wrote:

On Fri, Mar 10, 2017 at 4:38 PM, Emmanuele Bassi <[hidden email]> wrote:
Without knowing how AnyEvent works, if the handler involves threads
then calling *any* GTK+ API from different threads than the one that
called `gtk_main()` is going to blow up your application. Even if it
worked in one release, during a specific alignment of planets, or if
you placed the machines in a pentacle drawn in goat blood, there's no
guarantee that it'll keep working. GTK+ simply does *not* support
multi-threaded access of the windowing system resources.


It does not do anything like that. It only uses Glib::Timeout, Glib::IO, Glib::Idle watchers in fairly standard ways.
 
GTK+ and the G* platform has a whole bunch of API to deal with these
cases, like GTask[0]. Sadly, it's not very Perlish because nobody
wrote introspection overrides for it. Additionally, for things like
HTTP requests, there's a whole library called libsoup[1] which
integrates with the GLib main loop.


We're considering libsoup, but it depends on Glib::Object::Introspection for which there are no windows builds (that we know of).
 
Ciao,
 Emmanuele.


thanks,
Peter


_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list



_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list
Reply | Threaded
Open this post in threaded view
|

Re: Async HTTP download with Glib event loop doesn't work on windows

Daniel Kasak-4
Sorry for the delay in getting stuff together - as I mentioned, we're
renovating, and everything upstairs is packed / covered in dust / not
functioning. Anyway, I've set up my home server again. I have binaries
available at:
https://tesla.duckdns.org/gtk3-perl-windows/ ... direct link:
http://tesla.duckdns.org/downloads/JewelKit-1.0_strawberry-5.20.1-gtk-3.14.3.zip

This includes gtk+-3.14.3 and whatever other libs + bindings were
current at the time.

The documentation of the build process is available at:
http://win32.arrozcru.org/wiki/index.php?title=Perl-gtk3_for_Windows

I didn't come up with these - I paid another guy to do it, but I did
follow his instructions, and they worked at the time. They don't work
now :( If you can make them work, please document things. I've been
meaning to try pushing some of the patches required to make things
build, but finding the time to do it has been challenging, and it's
also difficult to do, not being the person who actually did the work (
ie the patches ).

Anyway, hopefully this is useful to you. Enjoy :)

Dan

On Sat, Mar 11, 2017 at 7:42 PM, Daniel Kasak <[hidden email]> wrote:

> I have some old(ish) binaries of Glib::Object::Introspection for Windows.
> Yes they were a major pain to build. I'd usually point you to my blog (
> http://tesla.duckdns.org ) - but it's offline at the moment ... we're
> renovating the house and my ADLS2, home server, and other things are totally
> out of action at the moment. If you're interested, I can go and get the
> server and rescue stuff off it. I also have build instructions that worked
> when they were written ( 18 months ago ). I tried repeating the process a
> couple of months back, and there are unfortunately major issues again, and
> I've kinda lost interest in building for Windows. Anyway, me know if you
> want Windows and/or build instructions.
>
> Dan
>
> On Sat, Mar 11, 2017 at 3:16 AM, Peter Juhasz <[hidden email]>
> wrote:
>>
>>
>> On Fri, Mar 10, 2017 at 4:38 PM, Emmanuele Bassi <[hidden email]> wrote:
>>>
>>> Without knowing how AnyEvent works, if the handler involves threads
>>> then calling *any* GTK+ API from different threads than the one that
>>> called `gtk_main()` is going to blow up your application. Even if it
>>> worked in one release, during a specific alignment of planets, or if
>>> you placed the machines in a pentacle drawn in goat blood, there's no
>>> guarantee that it'll keep working. GTK+ simply does *not* support
>>> multi-threaded access of the windowing system resources.
>>>
>>
>> It does not do anything like that. It only uses Glib::Timeout, Glib::IO,
>> Glib::Idle watchers in fairly standard ways.
>>
>>>
>>> GTK+ and the G* platform has a whole bunch of API to deal with these
>>> cases, like GTask[0]. Sadly, it's not very Perlish because nobody
>>> wrote introspection overrides for it. Additionally, for things like
>>> HTTP requests, there's a whole library called libsoup[1] which
>>> integrates with the GLib main loop.
>>>
>>
>> We're considering libsoup, but it depends on Glib::Object::Introspection
>> for which there are no windows builds (that we know of).
>>
>>>
>>> Ciao,
>>>  Emmanuele.
>>>
>>
>> thanks,
>> Peter
>>
>>
>> _______________________________________________
>> gtk-perl-list mailing list
>> [hidden email]
>> https://mail.gnome.org/mailman/listinfo/gtk-perl-list
>>
>
_______________________________________________
gtk-perl-list mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/gtk-perl-list