Hi! Please consider following me on twitter: @hanekomu.
2009年02月23日
Perl benchmarks: Nested data structures
The conclusion first: Nested arrays and hashes in Perl are slow. They get slower the more levels you nest. For most applications this won't be a problem, but if you cache values in a hash, you might want to use as few levels of nesting as possible.
First, let's benchmark reading from and writing to nested array elements. We use different arrays for six different levels of nesting, and all element indices are constants.
use Benchmark qw(:all); our (@array1, @array2, @array3, @array4, @array5, @array6); cmpthese(timethese(10_000_000, { L1w => sub { $array1[1] = 1 }, L2w => sub { $array2[1][2] = 1 }, L3w => sub { $array3[1][2][3] = 1 }, L4w => sub { $array4[1][2][3][4] = 1 }, L5w => sub { $array5[1][2][3][4][5] = 1 }, L6w => sub { $array6[1][2][3][4][5][6] = 1 }, L1r => sub { $array1[1] }, L2r => sub { $array2[1][2] }, L3r => sub { $array3[1][2][3] }, L4r => sub { $array4[1][2][3][4] }, L5r => sub { $array5[1][2][3][4][5] }, L6r => sub { $array6[1][2][3][4][5][6] }, }));
The results:
Benchmark: timing 10000000 iterations of L1r, L1w, L2r, L2w, L3r, L3w, L4r, L4w, L5r, L5w, L6r, L6w...
L1r: 0 wallclock secs ( 0.20 usr + -0.00 sys = 0.20 CPU) @ 50000000.00/s (n=10000000)
(warning: too few iterations for a reliable count)
L1w: 1 wallclock secs ( 0.72 usr + 0.01 sys = 0.73 CPU) @ 13698630.14/s (n=10000000)
L2r: 1 wallclock secs ( 1.19 usr + 0.01 sys = 1.20 CPU) @ 8333333.33/s (n=10000000)
L2w: 1 wallclock secs ( 2.01 usr + 0.02 sys = 2.03 CPU) @ 4926108.37/s (n=10000000)
L3r: 2 wallclock secs ( 1.82 usr + 0.02 sys = 1.84 CPU) @ 5434782.61/s (n=10000000)
L3w: 3 wallclock secs ( 2.67 usr + 0.03 sys = 2.70 CPU) @ 3703703.70/s (n=10000000)
L4r: 3 wallclock secs ( 2.63 usr + 0.01 sys = 2.64 CPU) @ 3787878.79/s (n=10000000)
L4w: 4 wallclock secs ( 3.54 usr + 0.02 sys = 3.56 CPU) @ 2808988.76/s (n=10000000)
L5r: 3 wallclock secs ( 3.04 usr + 0.00 sys = 3.04 CPU) @ 3289473.68/s (n=10000000)
L5w: 4 wallclock secs ( 4.13 usr + 0.04 sys = 4.17 CPU) @ 2398081.53/s (n=10000000)
L6r: 4 wallclock secs ( 3.76 usr + 0.01 sys = 3.77 CPU) @ 2652519.89/s (n=10000000)
L6w: 5 wallclock secs ( 4.46 usr + 0.02 sys = 4.48 CPU) @ 2232142.86/s (n=10000000)
Rate L6w L5w L6r L4w L5r L3w L4r L2w L3r L2r L1w L1r
L6w 2232143/s -- -7% -16% -21% -32% -40% -41% -55% -59% -73% -84% -96%
L5w 2398082/s 7% -- -10% -15% -27% -35% -37% -51% -56% -71% -82% -95%
L6r 2652520/s 19% 11% -- -6% -19% -28% -30% -46% -51% -68% -81% -95%
L4w 2808989/s 26% 17% 6% -- -15% -24% -26% -43% -48% -66% -79% -94%
L5r 3289474/s 47% 37% 24% 17% -- -11% -13% -33% -39% -61% -76% -93%
L3w 3703704/s 66% 54% 40% 32% 13% -- -2% -25% -32% -56% -73% -93%
L4r 3787879/s 70% 58% 43% 35% 15% 2% -- -23% -30% -55% -72% -92%
L2w 4926108/s 121% 105% 86% 75% 50% 33% 30% -- -9% -41% -64% -90%
L3r 5434783/s 143% 127% 105% 93% 65% 47% 43% 10% -- -35% -60% -89%
L2r 8333333/s 273% 248% 214% 197% 153% 125% 120% 69% 53% -- -39% -83%
L1w 13698630/s 514% 471% 416% 388% 316% 270% 262% 178% 152% 64% -- -73%
L1r 50000000/s 2140% 1985% 1785% 1680% 1420% 1250% 1220% 915% 820% 500% 265% --
Next we do the same for hashes, again with constant hash keys:
use Benchmark qw(:all); our (%hash1, %hash2, %hash3, %hash4, %hash5, %hash6); cmpthese(timethese(10_000_000, { L1w => sub { $hash1{L1} = 1 }, L2w => sub { $hash2{L1}{L2} = 1 }, L3w => sub { $hash3{L1}{L2}{L3} = 1 }, L4w => sub { $hash4{L1}{L2}{L3}{L4} = 1 }, L5w => sub { $hash5{L1}{L2}{L3}{L4}{L5} = 1 }, L6w => sub { $hash6{L1}{L2}{L3}{L4}{L5}{L6} = 1 }, L1r => sub { $hash1{L1} }, L2r => sub { $hash2{L1}{L2} }, L3r => sub { $hash3{L1}{L2}{L3} }, L4r => sub { $hash4{L1}{L2}{L3}{L4} }, L5r => sub { $hash5{L1}{L2}{L3}{L4}{L5} }, L6r => sub { $hash6{L1}{L2}{L3}{L4}{L5}{L6} }, }));
The results:
Benchmark: timing 10000000 iterations of L1r, L1w, L2r, L2w, L3r, L3w, L4r, L4w, L5r, L5w, L6r, L6w...
L1r: 0 wallclock secs ( 0.63 usr + -0.00 sys = 0.63 CPU) @ 15873015.87/s (n=10000000)
L1w: 0 wallclock secs ( 1.53 usr + 0.01 sys = 1.54 CPU) @ 6493506.49/s (n=10000000)
L2r: 2 wallclock secs ( 1.58 usr + -0.00 sys = 1.58 CPU) @ 6329113.92/s (n=10000000)
L2w: 2 wallclock secs ( 2.35 usr + 0.02 sys = 2.37 CPU) @ 4219409.28/s (n=10000000)
L3r: 3 wallclock secs ( 2.58 usr + 0.01 sys = 2.59 CPU) @ 3861003.86/s (n=10000000)
L3w: 4 wallclock secs ( 3.67 usr + 0.02 sys = 3.69 CPU) @ 2710027.10/s (n=10000000)
L4r: 4 wallclock secs ( 3.07 usr + 0.02 sys = 3.09 CPU) @ 3236245.95/s (n=10000000)
L4w: 5 wallclock secs ( 4.69 usr + 0.01 sys = 4.70 CPU) @ 2127659.57/s (n=10000000)
L5r: 5 wallclock secs ( 4.67 usr + 0.02 sys = 4.69 CPU) @ 2132196.16/s (n=10000000)
L5w: 5 wallclock secs ( 5.20 usr + 0.02 sys = 5.22 CPU) @ 1915708.81/s (n=10000000)
L6r: 6 wallclock secs ( 5.77 usr + 0.01 sys = 5.78 CPU) @ 1730103.81/s (n=10000000)
L6w: 7 wallclock secs ( 6.35 usr + 0.01 sys = 6.36 CPU) @ 1572327.04/s (n=10000000)
Rate L6w L6r L5w L4w L5r L3w L4r L3r L2w L2r L1w L1r
L6w 1572327/s -- -9% -18% -26% -26% -42% -51% -59% -63% -75% -76% -90%
L6r 1730104/s 10% -- -10% -19% -19% -36% -47% -55% -59% -73% -73% -89%
L5w 1915709/s 22% 11% -- -10% -10% -29% -41% -50% -55% -70% -70% -88%
L4w 2127660/s 35% 23% 11% -- -0% -21% -34% -45% -50% -66% -67% -87%
L5r 2132196/s 36% 23% 11% 0% -- -21% -34% -45% -49% -66% -67% -87%
L3w 2710027/s 72% 57% 41% 27% 27% -- -16% -30% -36% -57% -58% -83%
L4r 3236246/s 106% 87% 69% 52% 52% 19% -- -16% -23% -49% -50% -80%
L3r 3861004/s 146% 123% 102% 81% 81% 42% 19% -- -8% -39% -41% -76%
L2w 4219409/s 168% 144% 120% 98% 98% 56% 30% 9% -- -33% -35% -73%
L2r 6329114/s 303% 266% 230% 197% 197% 134% 96% 64% 50% -- -3% -60%
L1w 6493506/s 313% 275% 239% 205% 205% 140% 101% 68% 54% 3% -- -59%
L1r 15873016/s 910% 817% 729% 646% 644% 486% 390% 311% 276% 151% 144% --
I hadn't expected the differences to be so dramatic...
Tags: benchmarks.
posted at: 16:38 | path: /benchmarks | permalink | 5 comments | 0 trackbacks
2009年02月19日
The Grand Perspective on CPAN
I've created a tree map for the minicpan mirror using GrandPerspective on Mac OS X. To quote from GrandPerspective:
GrandPerspective is a small utility application for Mac OS X that graphically shows the disk usage within a file system. It can help you to manage your disk, as you can easily spot which files and folders take up the most space. It uses a so called tree map for visualisation. Each file is shown as a rectangle with an area proportional to the file's size. Files in the same folder appear together, but their placement is otherwise arbitrary.
Within the application, you can mouse over the rectangles to see which files and folders they represent, but here I've just annotated a few interesting blocks. Somehow it feels like a city layout...
posted at: 17:50 | path: /misc | permalink | 0 comments | 0 trackbacks
2009年02月18日
any::feature - Backwards-compatible handling of new syntactic features
I've released any::feature. The development repo is on github.
The problem
Perl 5.10 introduces new syntactic features which you can activate and
deactivate with the feature module. You want to use the
say feature in a program that's supposed to run under both Perl
5.8 and 5.10. So your program looks like this:
use feature 'say'; say 'Hello, world!';
But this only works in Perl 5.10, because there is no feature
module in Perl 5.8. So you write
use Perl6::Say; say 'Hello, world!';
This works, but it's strange to force Perl 5.10 users to install
Perl6::Say when the say feature is included in Perl
5.10.
The solution
Use any::feature!
WARNING: This is just a proof-of-concept.
any::feature can be used like Perl 5.10's feature
and will try to "do the right thing", regardless of whether you use Perl 5.8 or
Perl 5.10.
At the moment, this is just a proof-of-concept and only handles the
say feature. If things work out, I plan to extend it with other
Perl 5.10 features.
The following programs should work and exhibit the same behaviour both in Perl 5.8 and Perl 5.10.
This program will work:
use any::feature 'say'; say 'Hello, world!';
This program will fail at compile-time:
use any::feature 'say'; say 'Hello, world!'; no any::feature 'say'; say 'Oops';
The features are lexically scoped, which is how they work in Perl 5.10:
{ use any::feature 'say'; say 'foo'; } say 'bar'; # dies at compile-time
posted at: 14:04 | path: /dev | permalink | 0 comments | 0 trackbacks
2009年02月16日
Benchmarking immutable Mouse
Dann has added benchmarks for immutable Mouse to App::Benchmark::Accessors; I've also updated the benchmarks page.
Tags: benchmarks, Moose.
posted at: 13:14 | path: /moose | permalink | 0 comments | 0 trackbacks
If you're on Twitter, would you change your avatar to black to show support for the fight against a "three accusations and you're offline" law in New Zealand?
- Details on the law that will come into effect on Feb 28 unless we mobilize
- The Techsploder article "Join New Zealand Internet blackout protest against insane copyright law"
Via gnat (Nathan Torkington).
posted at: 01:19 | path: /misc | permalink | 0 comments | 0 trackbacks
2009年02月15日
Dissecting the Moose Part 5 - Accessor Generator Benchmarks Updated
The results of the accessor generator benchmarks generated by App::Benchmark::Accessors now have a permanent page. I will update the page from time to time as new versions of the accessor generators are released.
Tags: benchmarks, Moose.
posted at: 21:12 | path: /moose | permalink | 0 comments | 0 trackbacks
2009年02月14日
Bounds checking with Variable::Magic
Vincent Pit's Variable::Magic is an evil module that allows you to define Perl variable magic in Perl space. I hope to be able to use it for new types of join points in aspect-oriented programming (see Aspect).
Playing around with the module, I thought it could reproduce the effects of
tie() without having to tie objects. For example, let's enhance a
variable so that it can only take values that lie within given lower and upper
bounds.
use Variable::Magic qw(wizard cast); use Carp qw(croak); sub bounds_spell { my ($lower, $upper) = @_; wizard set => sub { my $value = ${$_[0]}; return if $value >= $lower && $value <= $upper; croak "bounds violation: $lower <= $value <= $upper is not true\n"; }; } my $foo; cast $foo, bounds_spell(2,8) or die "wizard FAIL\n"; $foo = 7; # OK $foo += 2; # exception
Setting the variable to 7 works fine, but increasing its value to 9 produces the following error:
bounds violation: 2 <= 9 <= 8 is not true
at test.pl line 15
main::__ANON__('SCALAR(0x800c6c)', 'undef') called at test.pl line 23
posted at: 13:31 | path: /dev | permalink | 0 comments | 0 trackbacks
2009年02月13日
Error::Return
I've released Error-Return. It exports one function,
RETURN, to be used within a try-block (see Error). It is a more intuitive way of returning from the
subroutine that contains the try-block.
try() takes a coderef using the & prototype so
it looks more like a normal Perl block or like map() or
grep(). But the "block" is still just an anonymous subroutine, so
using return within the sub won't do what you think it will do.
For example:
use Error ':try'; sub doit { print " in doit, before try\n"; try { print " in try: start\n"; return 456; print " in try: end\n"; } catch Error with { my $E = shift; print " caught error [$E]\n"; }; print " in doit, after try\n"; } print "before doit\n"; my $x = doit(); print "doit() returned [$x]\n"; print "after doit\n";
The return in the try-block (we call it a block,
but it really isn't) looks like it should return from doit(), but
it doesn't - it just returns from the anonymous sub that was passed to
try(). Therefore, this program prints the following:
before doit in doit, before try in try: start in doit, after try doit() returned [1] after doit
So in doit, after try is still reached, and doit()
returns 1 because of its last print statement.
While that is the correct behaviour, it is unintuitive. This module provides a more powerful way of returning.
Error::Return exports one function, RETURN, which is like
return except that it doesn't just return to its upper scope but
smashes right through it to the next-higher scope. Actually, it skips two
scopes, because it has to return from the try() subroutine as
well. It does take care of the cleanup that try() would normally
perform.
So now you can say:
use Error ':try'; use Error::Return; sub doit { print " in doit, before try\n"; try { print " in try: start\n"; RETURN 456; print " in try: end\n"; } catch Error with { my $E = shift; print " caught error [$E]\n"; }; print " in doit, after try\n"; } print "before doit\n"; my $x = doit(); print "doit() returned [$x]\n"; print "after doit\n";
and it prints
before doit in doit, before try in try: start doit() returned [456] after doit
which is more intuitive.
posted at: 16:46 | path: /dev | permalink | 0 comments | 0 trackbacks
Bringing the C API into Perl space
Around the turn of the century, when a limit was reached on how far you can mess with pure Perl, inspired Perl (and C) hackers have turned to the Perl C API.
In the last few years many modules have been released that bring the C API into Perl space: Devel::Declare, Devel::Caller, Scope::Upper, Variable::Magic, B::OPCheck and many more. It's a whole new level of fun, or a new level of chaos, depending on how you look at it.
I believe that this is the best effect that the whole Perl 6 discussion and development had so far.
posted at: 13:37 | path: /cpan_gems | permalink | 0 comments | 0 trackbacks
2009年02月12日
callee.pm
I've released callee. It exports one function,
callee(), which allows anonymous functions to refer to themselves.
This is necessary for recursive anonymous functions.
A recursive function must be able to refer to itself. Typically, a function
refers to itself by its name. However, an anonymous function does not have a
name, and if there is no accessible variable referring to it, i.e. the function
is not assigned to any variable, the function cannot refer to itself. This is
where callee() comes in.
This module is just very thin syntactic sugar for Devel::Caller.
use callee; my $f = sub { my $x = shift; return 1 if $x <= 1; $x * callee->($x-1); }->(5); # $f is 120
Takesako-san wrote arguments.pm, which does practically the same thing; see also his blog entry (in Japanese). I released this module because arguments.pm is not on CPAN, and because Devel::Caller already existed on CPAN, but not with the syntax I wanted.
posted at: 22:17 | path: /dev | permalink | 0 comments | 0 trackbacks
Nordic Perl Workshop
I've signed up for the Nordic Perl Workshop, bought a plane ticket and booked a hotel (the Anker Best Western, recommended by Marcus Ramberg). The workshop will take place in Oslo, Norway, on 15-16 April; on Saturday and Sunday afterwards I'll participate in the Enlightened Perl Hackathon — think Catalyst, Moose and DBIx::Class, among other things.
It will be good to see Perl hacker friends again: Ingy, jrockway, nothingmuch, Marcus, Abigail, Gabor and many others. I'm thinking more and more that Perl conferences are opportunities to hang out with friends, and you can go to some talks as well if you want to.
Tags: conferences.
posted at: 21:43 | path: /conferences | permalink | 0 comments | 0 trackbacks
Going to YAPC::Asia? Learn Japanese!
YAPC::Asia 2009 will probably be in September. I hope to be able to go; last year was fun and interesting.
If you're thinking of going — or even if you don't — you could start learning a few fundamentals of the Japanese language and writing systems. There are some really good learning sites and courses that I can recommend:
- Start with the hiragana course on iKnow
- Then do the katakana course on iKnow
- Learn basic words and kanji with the Japanese Core 2000 course on iKnow
- Take Japanese classes on eduFire
Tags: Asia, conferences, YAPC.
posted at: 20:20 | path: /conferences | permalink | 0 comments | 0 trackbacks
Moose::Manual
The newest versions of Moose include Moose::Manual. The Changes file says:
* Moose::Manual
- This is a brand new, extensive manual for Moose. This aims to
provide a complete introduction to all of Moose's
features. This work was funded as part of the Moose docs grant
from TPF. (Dave Rolsky)
Tags: Moose.
posted at: 13:44 | path: /moose | permalink | 0 comments | 0 trackbacks
![[Valid Atom 1.0]](/images/icons/valid-atom.png "Validate my Atom 1.0 feed"){kind=icon}