Re: DateTime performance

On Saturday 05 May 2012 17:14:52 N Heinrichs wrote:
> ++ to using precomposed TZ object (as you observed, supplying only the
> name as a string still results in lengthy DT:TZ object creation
> overhead.)
> 
> If you use them, I would also precompose any necessary Formatter or
> Locale objects.
> 

Thanks for the hint, but it does not apply to my
situation: most of the time, I am creating DateTime
objects from Unix epoch seconds; very occasionally
from individual pieces (year, month, etc). So, stuff
is already broken down quite nicely.

I'll check out the NO_VALIDATION option.

> 
> Depending on how you're parsing the date strings and composing the DT
> objects, `local $Params::Validate::NO_VALIDATION = 1;` can speed
> things up for you.
> 
> If you're using a DateTime::Formatter class to parse strings into DT
> objects for you, you might investigate using your own regex to chop up
> the string and call DT->new directly.
> 
> This code (including comments) is from early 2011 and I unfortunately
> do not have benchmark data handy:
> 
>         # NOTE: This method is faster than using DateTime::Formatter::MySQL
>         # NOTE2: It's also faster than `split m#[ /:T-]#`
>         $timestamp =~ m!^
>             (?:\s+)?(\d{4,4})[/-](\d{1,2})[/-](\d{1,2}) # Required date
> portion (?:[T\s](\d{1,2}):(\d{1,2}):(\d{1,2}))?     # Optional time
> portion (?:\s?([\w/\+:]+))?                         # Optional timezone
> $!x;
>         my ($y, $m, $d, $hr, $min, $sec, $tz) = ($1, $2, $3, $4, $5, $6,
> $7);
> 
> On 4 May 2012 13:20, Philipp K. Janert <[email protected]> wrote:
> > On Thursday 03 May 2012 02:14:45 you wrote:
> >> > From: Philipp K. Janert [mailto:[email protected]]
> >> > Sent: Wednesday, 2 May 2012 8:29 AM
> >> > 
> >> > Question:
> >> > 
> >> > When using DateTime for a large number of
> >> > instances, it becomes a serious performance
> >> > drag.
> >> 
> >> ...
> >> 
> >> > Is this expected behavior? And are there access
> >> > patterns that I can use to mitigate this effect?
> >> > (I tried to supply a time_zone explicitly, but that
> >> > does not seem to improve things significantly.)
> >> 
> >> Hi Phillip,
> >> 
> >> My #1 tip is to pre-prepare/cache the DateTime::TimeZone object and 
pass
> >> it in to each creation of a DateTime object (via whatever mechanism
> >> you're using to do that). I have seen a case where we were using
> >> time_zone => 'local' in a reasonably tight datetime object creation
> >> loop and saw significant speed increases just by cutting out that chunk
> >> of processing.
> >> 
> >> In hindsight that was a silly thing to do but it became an easy win :-)
> >> 
> >> I apologise if this is what you meant by supplying a time_zone
> >> explicitly in your comment above.
> > 
> > I have tried to specify the timezone explicitly as a string:
> >  $dt = DateTime->new( ..., time_zone => "America/Chicago" )
> > which does not seem to help, but I have not tried to do:
> >  $tz = DateTime::TimeZone( 'America/Chicago' )
> >  $dt = DateTime->new( ..., time_zone => $tz )
> > 
> > I'll try that the next time I have to process one of my data
> > sets again. ;-)
> > 
> > Thanks for the hint.
> > 
> >> I can't recommend using a tool like NYTProf highly enough on a run of
> >> your tool to spot the low hanging fruit. See
> >> https://metacpan.org/module/Devel::NYTProf
> >> 
> >> Cheers,
> >> 
> >> Andrew