Re: [CDBI] Make CDBI go fast

On 4/20/07, Bill Moseley <[email protected]> wrote:
> On Thu, Feb 15, 2007 at 11:34:36AM -0500, John Siracusa wrote:
>
>> http://rose.sourceforge.net/wiki/index.php/RDBO/Benchmark
>
> It's very hard not to look at those benchmarks and go "wow!", but
> I'm not sure how to relate those benchmarks to the web applications
> where the different ORMs might be used.

It's important to understand what these benchmarks are really
measuring.  Each one measures one tiny, specific task, and takes great
pains to do so in isolation.

Micro-benchmarks like this may seem less useful than "real world"
tests, but such "macro-benchmarks" are impossible to generalize.  My
real app may be nothing like your real app.  Figuring out which
micro-benchmarks are relevant to your app will require some thought,
of course.  But leaving that determination to the developer is the
appropriate choice, IMO.

>     # Simple: accessor 1
>            Rate  CDBS  CDBI  DBIC  RDBO
>     CDBS 1.38/s    --   -0%  -11%  -93%
>     CDBI 1.38/s    0%    --  -11%  -93%
>     DBIC 1.54/s   12%   12%    --  -92%
>     RDBO 19.4/s 1308% 1305% 1156%    --
>
> Why the huge difference there?

For scalar accessors, RDBO generates each method by producing the
minimal Perl code required to do the job, and then string-eval-ing it
into existence  at class setup time.  Most other ORMs have some form
of indirection happening at runtime on every call.

Since there's no database involvement in an accessor call, this
benchmark is entirely measuring Perl overhead.  In most benches that
do involve the db, the (more-or-less) constant overhead of the
database affects all participants, and acts as an equalizer, reducing
the size of the difference between any two ORMs.  The accessor bench
lacks this equalizer, and therefore shows a big range.

> Also, I tend to drop to DBI to do anything complex.  I like how the
> ORMs provide object relationships, but I don't expect them to do
> everything.  I'm a heavy user of set_sql in CDBI, for example.
> I assume that evens the playing field a bit if not depending on the
> ORM to do everything.

You can see how much faster plain DBI is than any ORM by including it
in the benchmark.  Example here:

http://rose.sourceforge.net/wiki/index.php/RDBO/Benchmark/DBI

-John