This Week on perl5-porters (27 November-3 December 2006)

This Week on perl5-porters - 27 November-3 December 2006

   "So I don't really have an answer for you. When faced with a similar
   situation, I punted and did a PerlIO_fdopen(fileno(stdio)) instead of
   a PerlIO_importFILE(stdio). This way you know you are creating a
   unixio layer which gets ref counted properly." -- Craig A. Berry,
   somewhere in the sprawling PerlIO layers saga.

Topics of Interest

"PerlIO::Scalar" buffering

   H.Merijn Brand wondered if he had uncovered a problem with performing
   IO to a scalar, since at the end of a series of operations where the
   scalar should have been emptied, a lone newline character remained.

     http://xrl.us/tqdv

Whither 5.8.9

   Jerry D. Hedden wondered if there was going to be a 5.8.9 release, and
   if so would arrive before 5.10.0. H.Merijn Brand confirmed that a
   5.8.9 release is going to happen some time, and it's likely to escape
   from the lab before 5.10. No dates have been set for either event as
   yet.

     when it's ready
     http://xrl.us/tqdw

Old "dmake"s for building perl on Win32...

   Steve Hay noticed that recent changes to the Makefiles for Windows are
   no longer understood by the older make programs floating around in the
   Windows world. He felt that the best approach was to update the README
   to explain the fix, in the unlikely event that someone encounters it,
   and to move on.

     time to go
     http://xrl.us/tqdx

... and archaic Visual C++ compilers

   Steve also noted that the compiler writers at Microsoft have played
   the compiler command line switch fandango in recent versions, and now
   warnings are emitted when certain deprecated switches are used. To get
   things into shape means drawing the line at VC++ 5.0. This equates to
   roughly a 10 year old compiler. The Makefile states that things should
   be compatible all the way back to 2.0, but Steve has his doubts.

   Nicholas Clark pointed out that if people really need to use a pre-5.0
   compiler, all they have to do is change the new-name switch back to
   old-name, and things should work. Steve updated README.win32 to note
   this point.

     http://xrl.us/tqdy

Trouble with PERLIO_INIT/PERLIO_TERM changes

   Steve Hay and Jarkko Hietaniemi tried to make sense of the ongoing
   PerlIO problems, in that reference counts were going too far negative
   and bad things were happening as a result. And the code was a bit too
   twisty for people to understand exactly what was happening. My eyes
   sort of glazed over on this thread.

   It eventually wound up being filed as a bug, see below.

     http://xrl.us/tqdz

   Also, in another thread, Steve also wondered whether
   PerlIO_importFILE() needed to do a "refcnt_inc". Craig A. Berry showed
   him how to test the hypothesis. One thing that came out of it is that
   Steve noticed that the code was leaking file descriptors.

     http://xrl.us/tqd2

   Similarly, Yves Orton found a segfault on Windows due to one of the
   many changes that were attempted this week to come to grips with these
   issues. Rafaël was pretty sure about which patch was the problem, and
   when Yves reverted it, his build succeeded in starting the test suite.

     http://xrl.us/tqd3

"PathTools" now at svk.perl.org

   Ken Williams announced that he'd thrown up "File::Spec" and "Cwd", on
   a Subversion repository, and asked if there were any volunteers who
   wanted to poke around at them. Philippe Bruhat bravely stepped
   forward.

     http://xrl.us/tqd4

Change #26513 breaks "Devel::SmallProf"

   Nicholas Clark found that the addition of an assertion to ensure an AV
   is always non-null made "Devel::SmallProf" curl up and die, and didn't
   what to do about it.

     http://xrl.us/tqd5

   In the new month, the thread continued: Gisle Aas observed that the
   problem went away of the build was configured with "usethreads".
   Rafaël Garcia-Suarez realised that the heart of the problem lay in
   toke.c, and made the appropriate change there.

     http://xrl.us/tqd6

Minor patch for "Benchmark.pm"

   Christophe Grosjean patched what he thought was a minor problem in
   "Benchmark", but was really quite a significant bug when you think
   about it: "cmpthese" used to ignore child process times, which led to
   the fascinating outcome that a process that spawned an external
   command appeared to be faster than something done in native Perl.

     Highly illogical
     http://xrl.us/tqd7

Patches of Interest

Abstract mempool header testing

   Jarkko did a bit of work on memory pools, (that approximates the
   conditions of Win32 on Unix), where memory allocation (especially in
   threads) is a much trickier issue. Nicholas Clark gave a clear
   explanation of how the mechanisms present in the source code fitted
   together, and Jan Dubois completed the picture.

   Based especially upon Jan's information, Jarkko produced a patch to
   attempt to clear up some PerlIO problems. Jan, however, thought that
   the code in question traded one set of problems (the mempool
   allocations) for another (leaking stuff from the opcode tree instead),
   and explained how to fix that problem

     http://xrl.us/tqd8

   As the month rolled over into December, Yves Orton took a stab at the
   matter, pointing to a couple of spots in the code that he felt relied
   on flawed assumptions. The patches went back and forth, and I lost
   track of which patch undid what, and which ones were reverted.

   Jan Dubois suggested one strategy for getting things straightened out,
   and Jarkko coded it up as a patch. This seemed to fix things
   sufficiently well that Steve Hay was able to unrevert a previously
   reverted patch with everything still behaving itself.

   Yves was most impressed, but also confused, and asked if there was
   some documentation that explained The Big Picture. Steve Hay did his
   best to illuminate the part of the canvas that he understood, hoping
   that Jan Dubois would chip in and complete as necessary. And so he
   did. Jan gave a thorough best exposé on the issue of mempools here:

     http://xrl.us/tqd9

   Now all we need is for someone to draw a diagram of the myriad ways in
   which one may allocate memory in perl. The thread in the December,
   from the top:

     http://xrl.us/tqea

Building 5.8.8 on OS/2

   Ilya Zakharevich delivered a patch to get 5.8.8 compiling again on
   OS/2. There are still some tests that fail, and Ilya promised to
   address them in a future patch.

     http://xrl.us/tqeb

"perl5db" on "miniperl"

   Similarly, Ilya noticed that perl5db.pl no longer worked with
   "miniperl", due to a reliance on "IO::Handle", which was not yet fully
   operational at that stage of the build process. So he tweaked things
   to handle this situation gracefully.

     http://xrl.us/tqec

"Test::Class" and blead

   A test in "Test::Class" tested for a bogus regular expression, and was
   supposed to die. The funny part was that thanks to Yves Orton's
   tireless efforts, what used to be a meaningless string of characters
   is now a legal pattern, so the pattern matches and the test fails.

   So Nicholas Clark made the pattern even more bogus, to force the
   pattern to fail again. Which judging by Yves's prodigious output,
   should see the test safe for a least three weeks.

     NO CARRIER
     http://xrl.us/tqed

MacOSX debugger fork support

   perldb.pl received some more patching goodness from Bo Lindbergh to
   allow the debugger to be fired up in a separate terminal window on Mac
   OS/X.

     http://xrl.us/tqee

Make "B::Lint" use "Module::Pluggable"

   Joshua ben Jore was one of the first people to take advantage of
   "Module::Pluggable"'s new promotion to core, and rewrote "B::Lint" to
   make use of it. He was immensely pleased, since it meant that he was
   able to rip out the previous clumsy efforts.

     The street finds its own uses for things
     http://xrl.us/tqef

No code specified for -.

   Jarkko discovered that an 'e' was dropped in the message "No code
   specified for -e" if in fact there was no code.

     dash before e except after p
     http://xrl.us/tqeg

Perl regexp structures bifurcation

   Yves Orton continued to deliver a stream of changes to introduce a
   higher level of abstraction to the regular expression engine. This is
   part of his ongoing work to permit pluggable engines.

     http://xrl.us/tqeh

"\R" means something else in POSIX

   Yves discovered that not only does the Unicode consortium care about
   regular expressions, they even published a specification on the
   matter. Yves read the spec, and learnt that they recommend using "\R"
   as a best-practice sequence for specifying a generic line-break
   marker.

   This unfortunately clashes with "\R" as a relative back-reference
   assertion in Perl's regular expression, but since we are still not out
   of beta, Yves was quite happy to cast around for a different unused
   character. After looking around at what was left of the alphabet, he
   settled in "\g" (as in group). He implemented a generalised syntax
   with curly braces, so that now "\g1" or "\g{1}" is the same as the
   existing "\1", and conversely, that "\g-1" and "\g{-1}" as a
   replacement for the now-defunct "\R1" syntax. (Yves, if you're reading
   this, doesn't that mean you could drop "\k<foo>" and subsume it into
   "\g<foo>"? You have 15 minutes to write the patch).

   Yves also thought that implement "\R" shouldn't be too difficult
   either, noting in passing that Philip Hazel will be delivering it in
   the next release of PCRE.

     14 minutes and counting
     http://xrl.us/tqei

New and old bugs from RT

Segfault due to a semicolon inside an array reference (#40995)

   Shlomi Fish discovered that a semi-colon, such as in "[ $i->func2();
   ]" gets the tokeniser in a tizz. Rafaël noted that this misbehaviour
   was fixed in blead.

     http://xrl.us/tqej

Setting $0 invalidates environment shown by "ps" (#41008)

   Alexander Bluhm noted that assigning to $0 causes unwanted
   side-effects on OpenBSD, and suggested a patch to fix things up.
   Rafaël took it and applied it to "blead".

     http://xrl.us/tqek

"(?(COND))" in pattern matching not working properly (#41010)

   Karl Williamson discovered a bug where an alternation ("(a|b)") did
   not match the same thing as a character class ("[ab]"). What was worse
   was that the original program, when it didn't crash, the values of
   other assorted variables in the program mysteriously changed. Yves
   Orton figured out what the problem was, and developed a patch to fix
   things up.

     Applied
     http://xrl.us/tqem

"backslash-G" does not appear to work in a while loop (#41011)

   Jim Woodworth filed what he thought was a bug, but Dr. Ruud suspected
   that he was a victim of crypto-context.

     http://xrl.us/tqen

Changing $0 on darwin leads to excessive padding in "ps" (#38868)

   The week continued in a $0 vein, with Darwin also causing trouble.
   Chris Dolan sorted things out with the help of the C preprocessor.

     You were expecting \0?
     http://xrl.us/tqeo

Regexp optimizer loses its hopes too soon (#39096)

   Yves Orton explored the reasons why the optimiser was ignoring obvious
   shortcuts and pointed out why a naive solution wouldn't work, and the
   problems that a generalised solution would have to consider.

     It used to be simple
     http://xrl.us/tqep

"LD_LIBRARY_PATH" changes not exported to Dynaloader,Perl 5.8.8 On 
Solaris 8 (#40655)

   A mysterious person reported that this bug was resolved.

     Private and Confidential
     http://xrl.us/tqeq

SIGSEGV with "pos()" in regexp (#40989)

   Yves Orton noted that this bug no longer existed in "blead". This is
   no doubt a pleasing side-effect of all the work that the engine has
   received in recent times, although he was hard pressed to say which
   change would have been responsible. By extension, he didn't expect the
   fix to be backported to "maint".

     One more reason
     http://xrl.us/tqer

   Yves noted that bug #39893 was probably another facet of the same
   problem.

     http://xrl.us/tqes

"File::Glob" braces with character classes erratic behaviour (#41027)

   Kolano discovered some inconsistencies with globs, curly braces and
   character classes.

     http://xrl.us/tqet

Nested closures result in scoping related segmentation fault (#41028)

   ... and they are fixed in blead.

     http://xrl.us/tqeu

"PerlIOUnix_refcnt_dec()" croaks in bleadperl since #29065 (#41030)

   Steve Hay reported that since change #29065, his
   "Win32-SharedFileOpen" croaked with a strange error from somewhere
   within PerlIO. He traced the problem to a couple of patches, thinking
   that while they were not incorrect, perhaps there was an underlying
   bug which had only now been exposed as a result.

     http://xrl.us/tqev

MSWin32: can't disable crlf trans. on STDxxx using :raw or binmode (#41034)

   Vaclav Ovsik reported a problem with end-of-line markers on Windows.
   He was not able to disable end-of-line translations on files opened by
   default (such as "STDOUT" and "STDERR", but files opened afterwards
   were just fine.

     http://xrl.us/tqew

Perl5 Bug Summary

   We almost crept under the 1500 barrier this week.

     Once more into the breech
     http://xrl.us/tqex

     http://rt.perl.org/rt3/NoAuth/perl5/Overview.html

New Core Modules

   *   "IPC::Cmd" achieved core status this week. Steve Hay tweaked it
       slightly to fix a test problem on Windows.

         http://xrl.us/tqey

   *   "threads" moved onto version 1.53. This corrected some of the
       problems that Steve Hay was experiencing with his smoke machine,
       but now he saw that the test suite was hanging on a different
       test.

         http://xrl.us/tqez

   *   "File::Fetch" made it to the core. Johan Vromans wondered why this
       functionality was not added to "LWP::UserAgent". The reason is
       that "File::Fetch" is a wrapper for any number of command-line
       tools and protocols than can be used to fetch a file (such as
       "rsync", "LWP" itself, "curl", "wget" and the like).

         http://xrl.us/tqe2

In Brief

   Nicholas Clark taught the Aho-Corasick regexp patch how to behave in
   the presence of threads.

     http://xrl.us/tqe3

   H.Merijn Brand discovered that "PerlIO::Scalar" ignores "format" and
   "write", preferring instead to send the output to "STDOUT", instead of
   the associated variable.

     The last user of formats in the world
     http://xrl.us/tqe4

   "Time::Piece" is back from the future.

     http://xrl.us/tqe5

   David Landgren discovered that a literal before "\k<foo>" (a named
   back reference) can really ruin a conference talk. Yves Orton resolved
   the problem in 20 minutes, give or take a few.

     http://xrl.us/tqe6

   The 5.8.3 "sprintf" patch made its way into the Mac OS X Security
   Update 2006-007.

     http://xrl.us/tqe7

   Paul Marquess independently rediscovered the joy of testing
   filehandles with threaded 5.6.x perls, and decided to give it a wide
   berth.

     http://xrl.us/tqe8
     http://xrl.us/tqe9

   Bo Lindbergh proposed adding lexical scope to "SelfLoader", complete
   with patch, but elicited no comments.

     http://xrl.us/tqfa

   Gabor Szabo and Abe Timmerman talked about the broken pages on
   db.test-smoke.org.

     In November
     http://xrl.us/tqfb

     and in December
     http://xrl.us/tqfc

   Yves Orton had some unhappy results with "Time::HiRes". After an
   incredibly long thread, everything was sorted out.

     http://xrl.us/tqfd

   Johan Vromans was worried that adding "Module::Load::Conditional" to
   the core would be like unleashing Pandora's box.

     http://xrl.us/tqfe

   Yves Orton wanted to know whether 'twas better to merge bugs or set up
   dependencies.

     Whatever lowers the count most
     http://xrl.us/tqff

   Jarkko tried in vain to sort out the UTF-8 -related failures that have
   been occurring in the smokes since August. Nicholas Clark explained
   the difficulties.

     http://xrl.us/tqfg

   Jarkko added some things to think about in "perltodo".

     Nick's movie
     http://xrl.us/tqfh

   Jim Cromie noticed that the "install" target misses
   "Compress/IO/{Base,Zlib}". The ideal solution involves using a more
   orthodox directory layout for the modules: "Configure" gets too
   confused otherwise.

     http://xrl.us/tqfi

   Rob "Sisyphus" puzzled over integer exponentiation under
   "-Duse64bitint" on Cygwin, wondering why the same result could be
   stored as an NV (a float) or an IV (an integer). Nicholas Clark could
   no longer remember why, only that it was a conscious design decision
   not to do integer exponentiation.

     http://xrl.us/tqfj

About this summary

   This summary was written by David Landgren.

   Weekly summaries are published on http://use.perl.org/ and posted on a
   mailing list, (subscription: [email protected]). The
   archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
   and comments are welcome.

   If you found this summary useful, please consider contributing to the
   Perl Foundation to help support the development of Perl.

-- 
"It's overkill of course, but you can never have too much overkill."