Language execution speeds or "The Fibonacci Stats"

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Language execution speeds or "The Fibonacci Stats"

Tim Janik-6
Hey Stefan,

following our discussion last week, I thought it'd be useful to have at least a
light comparison of different scripting/programming languages available. Just to
put decisions against Guile, for Python or for Javascript (or C++ at the BSE
core) into perspective performance wise.

I've implemented a small program to print fibonacci(40) in various languages,
the stupid recursive variant of course. It's simple enough that the needed bits
are covered next to "Hello World" for most languages, and complex enough that
the compilers won't optimize it away and VMs are significantly pressured during
execution. For reference, here's how it looks in Python:
        def f (n):
          if n > 1:     return f (n - 2) + f (n - 1)
          if n == 1:    return 1
          else:         return 0
        print (f (40))

Execution times are measured with bash's "time", here's the ranking:

// user 0m0.328s (gcc -O6)
// user 0m0.636s (Java SE 8)
// user 0m0.716s (clang -O3)
// user 0m0.720s (rust)
// user 0m0.884s (D - gdc-5.4.0)
// user 0m1.596s (js - node v4.2.6, v8 4.5.103.35)
#  user 0m10.168s (jruby)
// user 0m14.904s (php7)
#  user 0m17.556s (ruby 2.3.1p112)
-- user 0m22.480s (lua)
// user 0m24.964s (Java SE 8, interpreted mode execution only)
// user 0m26.152s (pike)
#  user 0m27.292s (jython)
#  user 0m38.288s (python2)
#  user 0m41.260s (python3)
#  user 0m57.584s (awk)
;; user 1m4.724s (guile)
#  user 1m39.080s (perl5)
#  user 9m5.000s (perl6 - Rakudo 2016.06)

A couple interesting observations can be made:

- The statically typed languages are definitely fastest and all of them use
compilation.

- Java still beats the C implementation compiled by clang and that includes the
time needed by OpenJDK to JIT the function, impressive! Though I wonder if the
JIT here's clever enough to make some consexpr optimizations behind the scenes,
I have a hard time seeing how it can beat clang otherwise...
        https://www.infoq.com/articles/OpenJDK-HotSpot-What-the-JIT

- Thoguh the statically typed compiled languages are generally faster by an
order of magnitude, Javascript is fairly close. Impressive!

- So Javascript as provided by V8 is by far the fastest dynamically typed
execution environment.

- It's interesting to note that jruby and jpython both significantly outperform
their C implementations, i.e. benefit from using the JVM.

- I would have hoped that python3 improved over python2, but didn't... ;-(

- Using Guile as a scripting language is a particularly bad choice for
computation performance. I really would have thought Guile beats the pants off
of Python and Ruby, and probably be up there with PHP.

- The PHP7 speed is impressive, given that it's dynamically typed and *not*
JITed (just like Guile). Btw, PHP7 supports typehints so we can add 'int' to the
function arguments, but that actually makes it run *slower*, b/c there's not JIT
here that can take advantage of the hints. It runs in 0m18.766s with typehints
added.
        https://www.golem.de/news/programmiersprache-im-detail-mit-php-7-wird-das-internet-schneller-1512-116750.html

- Perl6 is just... off limits. Allmost makes me wonder if it even makes sense to
pursue that language implementation.


PS: The language selection was done mostly by reading the Wikipedia language
lists and picking the ones that I vaguely knew and have access to on Ubuntu.
Fell free to provide ports to anything you think is missing. ;-)


--
Yours sincerely,
Tim Janik

https://testbit.eu/timj/
Free software author.
_______________________________________________
beast mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/beast
Reply | Threaded
Open this post in threaded view
|

Re: Language execution speeds or "The Fibonacci Stats"

Stefan Westerfeld
   Hi!

On Tue, Feb 14, 2017 at 01:13:32AM +0100, Tim Janik wrote:
> Hey Stefan,
>
> following our discussion last week, I thought it'd be useful to have at least a
> light comparison of different scripting/programming languages available. Just to
> put decisions against Guile, for Python or for Javascript (or C++ at the BSE
> core) into perspective performance wise.

While you're at it, there are two more JITs you could benchbark:
 - PyPy for Python
 - LuaJIT for Lua

   Cu... Stefan
--
Stefan Westerfeld, http://space.twc.de/~stefan
_______________________________________________
beast mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/beast
Reply | Threaded
Open this post in threaded view
|

Re: Language execution speeds or "The Fibonacci Stats"

Tim Janik-6
On 14.02.2017 12:59, Stefan Westerfeld wrote:
> While you're at it, there are two more JITs you could benchbark:
>  - PyPy for Python
>  - LuaJIT for Lua

Sure, barking at the bench, just like you request ;-)

The new results have Lua and Python catch up with Javascript pretty closely, so
giving these JITs a try was totoally worth it!

// user 0m0.328s (gcc -O6)
// user 0m0.636s (Java SE 8)
// user 0m0.716s (clang -O3)
// user 0m0.720s (rustc 1.7.0)
// user 0m0.884s (D - gdc-5.4.0)
// user 0m1.596s (js - node v4.2.6, v8 4.5.103.35)
-- user 0m1.924s (LuaJIT 2.0.4)
#  user 0m2.888s (PyPy 5.1.2 / GCC 5.3.1)
#  user 0m10.168s (jruby)
// user 0m14.904s (php7)
#  user 0m17.556s (ruby 2.3.1p112)
-- user 0m22.480s (Lua 5.2.4)
// user 0m24.964s (Java SE 8, interpreted mode execution only)
// user 0m26.152s (pike)
#  user 0m27.292s (Jython 2.5.3)
#  user 0m38.288s (Python 2.7.12)
#  user 0m41.260s (Python 3.5.2)
#  user 0m57.584s (awk)
;; user 1m4.724s (guile)
#  user 1m39.080s (perl5)
#  user 9m5.000s (perl6 - Rakudo 2016.06)

I didn't know about libgccjit so far, have to look that up now ;-)

>
>    Cu... Stefan
>

--
Yours sincerely,
Tim Janik

https://testbit.eu/timj/
Free software author.
_______________________________________________
beast mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/beast
Reply | Threaded
Open this post in threaded view
|

Re: Language execution speeds or "The Fibonacci Stats"

Stefan Westerfeld
   Hi!

On Tue, Feb 14, 2017 at 08:37:22PM +0100, Tim Janik wrote:

> On 14.02.2017 12:59, Stefan Westerfeld wrote:
> > While you're at it, there are two more JITs you could benchbark:
> >  - PyPy for Python
> >  - LuaJIT for Lua
>
> Sure, barking at the bench, just like you request ;-)
>
> The new results have Lua and Python catch up with Javascript pretty closely, so
> giving these JITs a try was totoally worth it!
>
> // user 0m0.328s (gcc -O6)
> // user 0m0.636s (Java SE 8)
> // user 0m0.716s (clang -O3)
> // user 0m0.720s (rustc 1.7.0)
> // user 0m0.884s (D - gdc-5.4.0)
> // user 0m1.596s (js - node v4.2.6, v8 4.5.103.35)
> -- user 0m1.924s (LuaJIT 2.0.4)
> #  user 0m2.888s (PyPy 5.1.2 / GCC 5.3.1)
> #  user 0m10.168s (jruby)
> // user 0m14.904s (php7)
> #  user 0m17.556s (ruby 2.3.1p112)
> -- user 0m22.480s (Lua 5.2.4)
> // user 0m24.964s (Java SE 8, interpreted mode execution only)
> // user 0m26.152s (pike)
> #  user 0m27.292s (Jython 2.5.3)
> #  user 0m38.288s (Python 2.7.12)
> #  user 0m41.260s (Python 3.5.2)
> #  user 0m57.584s (awk)
> ;; user 1m4.724s (guile)
> #  user 1m39.080s (perl5)
> #  user 9m5.000s (perl6 - Rakudo 2016.06)

In this example, everything looks good with PyPy. However, there is one problem
why we currently can't use PyPy for the UI. I'd recommend using PyQt for the
Python UI. So we can

 * use Python as language for most UI code
 * implement only performance critical widgets directly in C++
 * use Qt standard widgets (buttons, checkboxes, filedialog, toolbars, menus...)
   where we need them

PyQt is a mature python extension for UI development. However, PyPy is in some
cases incompatible with C extensions designed for CPython, so unfortunately,
the PyQt extension doesn't work with PyPy. So this means that if we want
PyQt/Python for the UI development, we will have to use CPython. Still I
believe that since we have the option of implementing widgets in C++ when
performance is an issue, we should be able to use Python for most of the code,
and C++ where necessary (for instance: fancy realtime spectrum visualization).

As for PyPy support for PyQt, there is a mention on this page:
 * https://morepypy.blogspot.de/2016/11/pypy27-v56-released-stdlib-2712-support.html

"We continue to make incremental improvements to our C-API compatibility layer
(cpyext). We pass all but 12 of the over-6000 tests in the upstream NumPy test
suite, and have begun examining what it would take to support Pandas and PyQt."

So there may be hope that some day PyPy and PyQt could be used together. But
right now it is not supported.

   Cu... Stefan
--
Stefan Westerfeld, http://space.twc.de/~stefan
_______________________________________________
beast mailing list
[hidden email]
https://mail.gnome.org/mailman/listinfo/beast