跳ね込むの日記 - hanekomu's Perl blog : Title: Dissecting the Moose Part 3

Hi! Please consider following me on twitter: @hanekomu.

2008年11月01日

Dissecting the Moose Part 3 - Benchmarking Accessor Generators

Update: Now also includes Mojo.

Update 2: Now uses Mojo 0.8.

Update 3: Now includes Class::MethodMaker.

Update 4: Now includes Object::Tiny, Spiffy, Class::Spiffy and direct hash access.

I wrote a small benchmarking program to compare the speeds of various accessor generators. They don't do much, just create an object while setting an accessor, then reading that accessor again, all for 100,000 times.

Here is the program:

#!/usr/bin/env perl

use warnings;
use strict;
use Benchmark qw(cmpthese timethese :hireswallclock);


package WithMoose;
use Moose;
has myattr => ( is => 'rw' );


package WithMooseImmutable;
use Moose;
has myattr => ( is => 'rw' );
__PACKAGE__->meta->make_immutable;


package WithMouse;
use Mouse;
has myattr => ( is => 'rw' );


package WithClassAccessor;
use base qw(Class::Accessor);
__PACKAGE__->mk_accessors(qw/myattr/);


package WithClassAccessorFast;
use base qw(Class::Accessor::Fast);
__PACKAGE__->mk_accessors(qw/myattr/);


package WithClassAccessorFastXS;
use base qw(Class::Accessor::Fast::XS);
__PACKAGE__->mk_accessors(qw/myattr/);


package WithClassAccessorComplex;
use base qw(Class::Accessor::Complex);
__PACKAGE__->mk_new->mk_scalar_accessors(qw/myattr/);


package WithClassAccessorConstructor;
use base qw(Class::Accessor::Constructor Class::Accessor::Complex);
__PACKAGE__->mk_constructor->mk_scalar_accessors(qw/myattr/);


package WithMojo;
use base qw(Mojo::Base);
__PACKAGE__->attr('myattr');


package WithClassMethodMaker;
use Class::MethodMaker
    [ scalar => [ qw/myattr/ ],
      new    => [ qw/-hash new/ ],
    ];


package WithObjectTiny;
use Object::Tiny qw/myattr/;


package WithSpiffy;
use Spiffy -base;
field 'myattr';


package WithClassSpiffy;
use Class::Spiffy -base;
field 'myattr';


package main;


cmpthese(timethese(100_000,{
    moose => sub {
        my $obj = WithMoose->new(myattr => 27);
        my $x = $obj->myattr;
    },
    moose_immutable => sub {
        my $obj = WithMooseImmutable->new(myattr => 27);
        my $x = $obj->myattr;
    },
    mouse => sub {
        my $obj = WithMouse->new(myattr => 27);
        my $x = $obj->myattr;
    },
    class_accessor => sub {
        my $obj = WithClassAccessor->new({ myattr => 27 });
        my $x = $obj->myattr;
    },
    class_accessor_fast => sub {
        my $obj = WithClassAccessorFast->new({ myattr => 27 });
        my $x = $obj->myattr;
    },
    class_accessor_fast_xs => sub {
        my $obj = WithClassAccessorFastXS->new({ myattr => 27 });
        my $x = $obj->myattr;
    },
    class_accessor_complex => sub {
        my $obj = WithClassAccessorComplex->new(myattr => 27);
        my $x = $obj->myattr;
    },
    class_accessor_constructor => sub {
        my $obj = WithClassAccessorConstructor->new(myattr => 27);
        my $x = $obj->myattr;
    },
    mojo => sub {
        my $obj = WithMojo->new(myattr => 27);
        my $x = $obj->myattr;
    },
    class_methodmaker => sub {
        my $obj = WithClassMethodMaker->new(myattr => 27);
        my $x = $obj->myattr;
    },
    object_tiny => sub {
        my $obj = WithObjectTiny->new(myattr => 27);
        my $x = $obj->myattr;
    },
    spiffy => sub {
        my $obj = WithSpiffy->new(myattr => 27);
        my $x = $obj->myattr;
    },
    class_spiffy => sub {
        my $obj = WithClassSpiffy->new(myattr => 27);
        my $x = $obj->myattr;
    },
    direct_hash => sub {
        my $h = {};
        $h->{myattr} = 27;
        my $x = $h->{myattr};
    },
}));

And here are the results, on a 2.16 GHz MacBook with 2 GB RAM, Mac OS X 10.5.5, perl 5.8.8 (yes, it's not the newest one):

Benchmark: timing 100000 iterations of class_accessor, class_accessor_complex, class_accessor_constructor, class_accessor_fast, class_accessor_fast_xs, class_methodmaker, class_spiffy, direct_hash, mojo, moose, moose_immutable, mouse, object_tiny, spiffy...
class_accessor: 0.680644 wallclock secs ( 0.67 usr +  0.00 sys =  0.67 CPU) @ 149253.73/s (n=100000)
class_accessor_complex: 0.890903 wallclock secs ( 0.88 usr +  0.01 sys =  0.89 CPU) @ 112359.55/s (n=100000)
class_accessor_constructor: 1.99036 wallclock secs ( 1.96 usr +  0.01 sys =  1.97 CPU) @ 50761.42/s (n=100000)
class_accessor_fast: 0.561965 wallclock secs ( 0.54 usr +  0.00 sys =  0.54 CPU) @ 185185.19/s (n=100000)
class_accessor_fast_xs: 0.4521 wallclock secs ( 0.44 usr +  0.00 sys =  0.44 CPU) @ 227272.73/s (n=100000)
class_methodmaker: 1.51813 wallclock secs ( 1.48 usr +  0.01 sys =  1.49 CPU) @ 67114.09/s (n=100000)
class_spiffy: 0.620832 wallclock secs ( 0.60 usr +  0.01 sys =  0.61 CPU) @ 163934.43/s (n=100000)
direct_hash: 0.116149 wallclock secs ( 0.12 usr +  0.00 sys =  0.12 CPU) @ 833333.33/s (n=100000)
            (warning: too few iterations for a reliable count)
      mojo: 1.00943 wallclock secs ( 0.95 usr +  0.01 sys =  0.96 CPU) @ 104166.67/s (n=100000)
     moose: 11.4151 wallclock secs (10.87 usr +  0.09 sys = 10.96 CPU) @ 9124.09/s (n=100000)
moose_immutable: 0.899679 wallclock secs ( 0.87 usr +  0.01 sys =  0.88 CPU) @ 113636.36/s (n=100000)
     mouse: 5.28056 wallclock secs ( 5.05 usr +  0.05 sys =  5.10 CPU) @ 19607.84/s (n=100000)
object_tiny: 0.382038 wallclock secs ( 0.31 usr +  0.01 sys =  0.32 CPU) @ 312500.00/s (n=100000)
            (warning: too few iterations for a reliable count)
    spiffy: 0.620199 wallclock secs ( 0.60 usr +  0.01 sys =  0.61 CPU) @ 163934.43/s (n=100000)
                               Rate moose mouse class_accessor_constructor class_methodmaker mojo class_accessor_complex moose_immutable class_accessor class_spiffy spiffy class_accessor_fast class_accessor_fast_xs object_tiny direct_hash
moose                        9124/s    --  -53%                       -82%              -86% -91%                   -92%            -92%           -94%         -94%   -94%                -95%                   -96%        -97%        -99%
mouse                       19608/s  115%    --                       -61%              -71% -81%                   -83%            -83%           -87%         -88%   -88%                -89%                   -91%        -94%        -98%
class_accessor_constructor  50761/s  456%  159%                         --              -24% -51%                   -55%            -55%           -66%         -69%   -69%                -73%                   -78%        -84%        -94%
class_methodmaker           67114/s  636%  242%                        32%                -- -36%                   -40%            -41%           -55%         -59%   -59%                -64%                   -70%        -79%        -92%
mojo                       104167/s 1042%  431%                       105%               55%   --                    -7%             -8%           -30%         -36%   -36%                -44%                   -54%        -67%        -87%
class_accessor_complex     112360/s 1131%  473%                       121%               67%   8%                     --             -1%           -25%         -31%   -31%                -39%                   -51%        -64%        -87%
moose_immutable            113636/s 1145%  480%                       124%               69%   9%                     1%              --           -24%         -31%   -31%                -39%                   -50%        -64%        -86%
class_accessor             149254/s 1536%  661%                       194%              122%  43%                    33%             31%             --          -9%    -9%                -19%                   -34%        -52%        -82%
class_spiffy               163934/s 1697%  736%                       223%              144%  57%                    46%             44%            10%           --    -0%                -11%                   -28%        -48%        -80%
spiffy                     163934/s 1697%  736%                       223%              144%  57%                    46%             44%            10%           0%     --                -11%                   -28%        -48%        -80%
class_accessor_fast        185185/s 1930%  844%                       265%              176%  78%                    65%             63%            24%          13%    13%                  --                   -19%        -41%        -78%
class_accessor_fast_xs     227273/s 2391% 1059%                       348%              239% 118%                   102%            100%            52%          39%    39%                 23%                     --        -27%        -73%
object_tiny                312500/s 3325% 1494%                       516%              366% 200%                   178%            175%           109%          91%    91%                 69%                    38%          --        -62%
direct_hash                833333/s 9033% 4150%                      1542%             1142% 700%                   642%            633%           458%         408%   408%                350%                   267%        167%          --

Some points of note:

My Class::Accessor::Constructor is relatively slow, but then it does quite a lot.
Moose is the slowest of them all, but that's not surprising considering it uses a meta-object protocol.
When making classes that use Moose immutable, things get a lot faster.
I thought Mouse was supposed to be leaner than Moose, but it's still a lot slower than immutable Moose.
Class::Accessor::Fast::XS is very fast, but then it doesn't do very much compared to Moose or Class::Accessor::Constructor.
Object::Tiny is even faster than Class::Accessor::Fast::XS!