This Week on perl5-porters (10-16 July 2006)

This Week on perl5-porters - 10-16 July 2006

   "Kindly avoid inflicting upon this one the same fate shared by so
   much other documentation produced by a patchwork of well-meaning but
   disparate contributors seeing more the trees than the forest as they
   continually apply peephole optimizations that inadvertently compromise
   the overall integrity of the work, where the presentation eventually
   more resembles an illegible patchwork quilt of no especial application
   or intended purpose than it does a dedicated garment of one cloth
   woven." -- Tom Christiansen, at the height of his form.

Topics of Interest

Adding examples to the core documentation

   Gabor Szabo continued his quest to add examples to the core
   documentation, based on feedback he had received giving training
   lessons. First up, he wanted to rewrite "perlfunc" to use the modern
   approach for opening files (using lexical file handles and the three
   argument form).

   While no-one could dispute that this was a good idea, Nicholas
   objected to the fact that it meant that new stuff was being added but
   no old stuff was being pruned away.

   Other changes that Gabor wished to make involved adding more examples
   to the documentation. Again, Nicholas wondered why people wouldn't
   have the reflex to toss off a few one-liners to see how things worked.

   H.Merijn Brand suggested creating an examples directory. Gabor liked
   the idea, but regretted that the usual installer tools ("CPAN" and
   "CPANPLUS") do not install the "eg/" directories, making them all but
   invisible to mere mortals.

     Opening gambit
     http://xrl.us/o7qy

   Gabor then continued with examples for "index"/"rindex". Given that
   the debate about "open" was not resolved, this was not applied.

     http://xrl.us/o7qz

   He then added a brief discussion of "use strict" in "perlintro" and
   replacep5psummd the use of $a and $b by some less-special variables. 
H.Merijn
   liked it enough to apply it.

     http://xrl.us/o7q2

   "perlintro" also received another two batches of changes, here:

     http://xrl.us/o7q3

   and here:

     http://xrl.us/o7q4

   Gabor was briefly confused as to whether he should be patching "blead"
   or "maint". H.Merijn Brand and Yves Orton explained that he should
   patch "blead", and that the "maint" pumpking (currently Nicholas
   Clark, but will be Dave Mitchell in the future) will backport patches
   to "maint" as appropriate.

     http://xrl.us/o7q5

   So Gabor patched "perlhack" in order to point future porters to the
   right thing to hack.

     X marks the spot
     http://xrl.us/o7q6

   Gabor then gave perlopentut.pod a work-over to begin recommending the
   use of lexical file handles rather than typeglobs. H.Merijn Brand had
   a number of issues with the proposed changes, and the thread continued
   on for quite some time. People seemed to be in agreement that some
   sort of improvement was needed, but the hard question was what was
   truly essential for a tutorial, and in what order to present it.

     Opening night
     http://xrl.us/o7q7

   After digesting the suggestions that everyone put forward, Gabor
   produced a second cut. After an initial false start, Tom Christiansen
   explained that from a tutorial perspective, omitting parentheses and
   preferring "or" over "||" is incorrect. Having taught thousands of
   people Perl, Tom has come to the conclusion that parens and
   low-precedence "||" *helps* people come to grips with the language.
   Omitting needless parentheses comes further up the curve of Perl
   mastery.

     http://xrl.us/o7q8

On the correct use of "my_strlcpy"

   Steve Peters had tried to replace some uses of "strcpy", (which is
   deemed "unsafe" in that it can copy without limit), with the safer
   "my_strlcpy". Except that they don't return the same thing, which
   caused Steve Hay's smoke machine to have a fit. The first Steve was
   rather embarrassed and apologised.

   On the plus side, this revealed a place where more tests are needed.

     Copy right
     http://xrl.us/o7q9

"Hash::Util::FieldHash" *versus* global destruction

   Joshua ben Jore wanted to know whether "Hash::Util::FieldHash" makes
   inside-out objects safe in the face of object that defines a "DESTROY"
   method. Anno Siegel explained that everything should be fine during
   the normal course of events, but wasn't too sure whether global
   destruction (what happens when the interpreter is being taken down
   just before the perl process expires) changes the rules so much that
   this no longer holds true.

   Joshua spelt out some of the potential problem scenarios he envisaged,
   and how best to deal with them. Anno thought that the proposed
   work-arounds were all rather bletcherous, and promised to think about
   it to see whether he could come up with a good solution.

   The underlying problem is that inside-out objects reveal an aspect of
   global destruction that was never apparent before, and if you're
   really unlucky, it could bite your code.

     Unintended consequences
     http://xrl.us/o7ra

Confusing "SvCUR" and "SvLEN" documentation

   Yves Orton read the documentation concerning these two functions and
   had difficulty in reconciling it with what "Devel::Peek" was telling
   him. Jan thought it made sense to him, and so he wondered what it was
   that Yves didn't understand. There ensued a long conversation between
   Jan and Yves that discussed the matter in more detail.

   Yves finally understood the issue, and suggested that Jan write up
   some documentation to clarify the matter. Jan hoped that someone else
   could summarise the issue instead (who, me?) and Yves said that he
   would take a stab at it, and allow Jan to make fun of it until it all
   made sense.

     String theory
     http://xrl.us/o7rb

CPAN contributor feedback

   Responding to the itchy thread from last week, David Muir Sharnoff
   explained that his number one gripe with the core was its less than
   complete support for tied arrays, magic on tied variables, and the
   lack of a "GETREF" method for ties. All of which would no doubt make
   his life easier.

     Not to mention closing a bug or two
     http://xrl.us/o7rc

Code that is only executed once

   David Nicol proposed adding a 'once' statement modifier, that, after
   having been executed, would then delve into the optree and excise the
   statement out of the optree by rethreading the prior opcode to the
   next opcode.

   Rafael agreed that Perl 6 has a similar construct, but that there were
   no plans to add it to 5.10. He was happy that "state" variables
   resolved the ugly "my $x if 0" hack, and thought that "UNITCHECK",
   from the "perltodo" list was more worthy of burning up some tuits.

     COBOL has an ALTER verb, after all
     http://xrl.us/o7rd

Removing the succeed path from the super-linear cache code

   This is an optimisation in the regular expression engine that, as far
   as Dave Mitchell can tell, is never exercised by the test suite, and,
   try as he might, was unable to concoct any example that could. So
   after talking things over with Hugo van der Sanden, he decided to fire
   up the chain-saw.

   So if your fancy patterns now starting taking longer than the
   scheduled moment of the heat death of the universe, you know who to
   blame. Hugo had hoped that some sort of instrumentation under a
   debugging build might be added to the engine to allow the measurements
   of different optimisation strategies. Dave wasn't planning to,
   although he figured it would probably be doable, even if he didn't
   know right now what shape or form it would take.

     Look for a patch tomorrow morning
     http://xrl.us/o7re

Comments on recent API changes to "threads"

   Artur Bergman, one of the original authors of the "threads" module,
   discovered all the work that had been done recently, and objected to
   the change that allowed "exit" to end a thread silently (and not the
   entire process), saying that it blurred the distinction between
   threads and processes.

   He also spotted a section in the code that dealt with passing signals
   to threads, noting that it was not thread-safe. Jerry wasn't sure
   whether this was a true, and asked for a patch to help him understand
   the situation.

   Jerry D. Hedden regretted that some of the design decisions that Artur
   mentioned, as a way of explaining why things were the way they were,
   were not mentioned in the documentation.

   Jan defended the design decision to allow a nested function to
   terminate the thread, and not have to percolate the desire back up the
   call stack to the main thread function.

   Liz Mattijsen voiced the opinion that it was important for a thread
   encountering some naughty library code, that contained evil "exit"
   somewhere, to not be able to take out the whole application.

   Artur explained that a technique already existed for dealing with this
   situation.

     Picking up the thread
     http://xrl.us/o7rf

Threads signalling issue

   Jerry D. Hedden was sufficiently worried by Artur's claim that the
   signal dispatching code was not safe, that he broke the discussion out
   into a separate thread. He stated that as far as he could see, the
   code *was* thread-safe. Jan Dubois confirmed that he was certain that
   it was fine, as well.

   Artur described the problem he envisaged in more detail, which allowed
   Jan to point out where the argument was flawed, since the code already
   took the situation Artur outlined into account.

   And so now Artur owes Jan a beer.

   Nicholas Clark pointed out that even though the above code is
   race-safe on a single-processor box, on an SMP machine it might be
   possible to have a race, and he showed where it could happen. The jury
   is still out on this matter.

     http://xrl.us/o7rg

"Storable.pm" has poor version compatibility

   Kari Pahula was shuffling binary images between two computers using
   "RPC::PlServer" and "RPC::PlClient", with "Storable" acting as a
   serialising mechanism, and shipping the result over the wire.

   Things started to fall apart when the two computers in question were
   running different versions of "Storable" (The 2.6 receiver *versus*
   the 2.7 sender), which rendered the received data essentially
   worthless. Kari wished that "Storable" had a mechanism to specify that
   version "N" should produce a data stream compatible with version
   "N-x", for some small value of "x". That is, Kari would quite happy to
   go back to his 2.7 server, and explicitly add a "make this 2.6
   compatible" flag, except that this is currently not possible.

     http://xrl.us/o7rh

Patches of Interest

Test scripts for "I18N::Langinfo" and "POSIX"

   Last week, Sébastien Aperghis-Tramoni wrote some tests to improve the
   coverage of these two modules. In doing so, he discovered a couple of
   bugs: fields documented as existing, in fact, don't. Steve Peters had
   problems with some of the test results, and applied a partial patch.
   Craig Berry looked at some of the skipped tests this week, and made a
   few more changes.

     http://xrl.us/o7ri

   Here are the changes, concerning missing POSIX functionality, that
   Craig realised was being dealt with appropriately.

     http://xrl.us/o7rj

   Sébastien came back later on with a way to improve
   "POSIX::localeconv()"'s usefulness, by defining all keys in the
   underlying hash, and making the undefined ones point to an empty
   string. This would make the test suite easier to manage, and would
   lead to less make-work code being required in client code.

     http://xrl.us/o7rk

"perlhack": reflections on portability

   Jarkko Hietaniemi is undergoing therapy to recover from doing combat
   for far too long with C compilers that conspire to make the perl
   codebase such a fun place to hang out.

     The tao of portability
     http://xrl.us/o7rm

   And another batch of things to watch out for.

     http://xrl.us/o7rn

Making sense of the z/OS patches

   Jarkko then started picking away at the z/OS patches that had recently
   been sent to the list from IBM, to see what he could salvage. First up
   was some low-hanging fruit in the shape of hints/os390.sh and
   "Makefile.SH".

     http://xrl.us/o7ro

   And then a number of EBCDIC fixes in the test suite.

     http://xrl.us/o7rp

   He carried on with some more library files and test. Stephen McCamant
   wondered whether there wasn't an elegant way to factorise the
   EBCDIC/ASCII differences so that less code wound up being duplicated.

     http://xrl.us/o7rq

   And something to do with "main" that H.Merijn was unable to apply,
   despite his best intentions.

     http://xrl.us/o7rr

   Taking that, and Stephen's, remarks into account, Jarkko brewed a
   fresh patch.

     http://xrl.us/o7rs

   At the end of the week, Jarkko was able to tease out a fix for
   "MIME::Base64"'s "quoted-print.t" test. He wasn't precisely sure how
   EBCDIC and "quoted-printable" are supposed to work together at the
   best of times, but sent the patch on its way.

     =E8=96=A4@=94=81=A8@=98=A4=96=A3=85@=94=85@=96=95@=A3=88=81=A3=
     http://xrl.us/o7rt

Various additions to cflags.SH and sundry files

   Jarkko discovered with delight that recent versions of "gcc" have a
   "-std" to specify to which C standard the source code should be held
   against. So he taught cflags.SH to specify that C89 is the only game
   in town.

     // is not a comment
     http://xrl.us/o7ru

   And in the interest of perfection, Jarkko also ensured that a number
   of recent additions to the code base were cited in cflags.SH.

     http://xrl.us/o7rv

   In a grab-bag of assorted fixes, Jarkko patched dump.c, pp_sys.c, sv.c
   and util.c and snuck in a few more additions to "perlhack".

     http://xrl.us/o7rw

   In a last glorious attack on "perlhack", Jarkko wrote about the
   "Configure -Dgccansipedantic" switch, that forces the "gcc" compiler
   to become extremely fussy about what it considers acceptable practices
   in code.

     Squeaky clean
     http://xrl.us/o7rx

Updated escapes and debug output improvements

   Yves Orton landed a fairly hefty patch of tweakery, in part to improve
   the escaping code, making it Unicode aware. The second part was to
   clean up the way debugging output is displayed by the regexp engine.
   The latter work was not yet complete, but Yves felt it was time to
   commit.

   Dave Mitchell applied the changes, and noted a number of compiler
   warnings about "char *"/"U8" differences, and mentioned that the debug
   output is currently a bit of mess, and the direction in which he
   wished to push it.

   Yves explained what he was tinkering with at the moment, and asked for
   guidance on dealing with the variables used in the code. He thought
   that the de-recursion work that Dave had put into the engine offered a
   suitable place for squirrelling away new variables, and thought that
   the current way of dealing with variables was weird.

   Dave Mitchell explained that the current state of affairs can be
   traced back to the heritage of Henry Spencer's original regular
   expression library, twisted beyond belief to work in a
   multi-interpreter, multi-threaded environment.

     *sounds of hammers, saws and arc-welders in the basement*
     http://xrl.us/o7ry

   Yves fixed up a number of warnings that the new code issued, in a
   subsequent patch.

     http://xrl.us/o7rz

Watching the smoke signals

Smoke [5.9.4] 28549 FAIL(F) linux 2.6.12-9-powerpc [debian] (ppc/1 cpu)

   This failure, Rafael Garcia-Suarez believed, was due to the recent
   work performed on dump.c whose aim was to improve the output by
   escaping "invisible" characters. Yves promised to take a look (which
   happened just above).

     http://xrl.us/o7r2

New and old bugs from RT

5.7.2 and "perlPod::Text::Overstrike" (#7959)

   Whatever the problem was, at some point in the past four years, Steve
   Peters discovered that it had been fixed.

     Counterstrike
     http://xrl.us/o7r3

localtime(3) calls tzset(3), but localtime_r(3) may not. (#26136)

   Jason Vas Dias posted an patch to fix a bug for Perl on Fedora,
   following on from a revival of the patch by Benjamin Holzman. Benjamin
   followed up on Jason's work and tweaked the patch to make it suitable
   for both "maint" and "blead".

     It's about time
     http://xrl.us/o7r4

"defined"-ness of substrings disappear over repeated calls (#39247)

   A few words of wisdom from Yitzchak Scott-Thoennes on the matter of XS
   code dealing with magic.

     http://xrl.us/o7r5

Bug in system calls when %ENV is very large (#39547)

   Steve Hay traced this problem down to the way perl gets compiled on
   the Win32 platform and thought that as things stand, the restriction
   appears to be an intrinsic limitation of the platform, and doubted it
   could be solved.

   Randy W. Sims forwarded the message to Microsoft, who confirmed that
   the environment block for a process is indeed limited to just shy of
   32 kilobytes.

     Their bug, not ours
     http://xrl.us/o7r6

gcc 3.3 has problems with "__attribute__((unused))" (#39634)

   Andy Dougherty tweaked the setting of the "HAS_ATTRIBUTE" define in a
   way that he felt was the clearest to understand. Rafael applied it.

     http://xrl.us/o7r7

filetest problem with STDIN/OUT on Windows (#39637)

   Steve Hay wrote some more C code to analyse this problem and found
   that indeed, the "-r" operator returns false for "STDIN". Trying it
   out with various C compilers led him to conclude that this is also
   some sort of Win32 limitation. Steve and Yves kicked around a few
   ideas, but at the end of the week it looks like it will wind up
   documented in "perlport" as something that doesn't work.

     The tailor's face and hands
     http://xrl.us/o7r8

"S_regmatch" produces a stack overflow (#39774)

   A pattern that results in a stack overflow was reported. Steve Peters
   explained that in "blead", the overflow no longer exists, although an
   error message described another sort of exhaustion is emitted instead.

   Yves Orton explained how the pattern was one of the sort that tend to
   give NFAs such as perl's regexp engine fits, as it forces backtracking
   information to be stashed away at every single point in the string. He
   went on to show how the pattern could be rewritten in a much more
   efficient manner.

   He then expressed hope that some day, the optimiser might be able to
   compose the transform all by itself, but reasoned that it might be
   difficult.

     When he says it's not easy
     http://xrl.us/o7r9

Cygwin Perl bug -- "pod2usage(-verbose => 0)" and "pod2usage(-verbose => 
1)" (#39775)

   David Christensen wanted to know what the status was on a patch that
   fixes a problem with "pod2usage".

     http://xrl.us/o7sa

"Switch" module bug (#39789)

   Artyom Tseitlin, poor chap, was bitten by too-clever-by-half magic of
   "Text::Balanced" (upon which "Switch" relies). Adriano Ferreira posted
   a semi-evil hack that would allow Artyom to continue to use "Switch".

     Divide and conquer
     http://xrl.us/o7sb

Unable to build Perl under Irix with "-Duseshrplib" (#39797)

   Philippe Schaffnit was going mad trying to discover what his mistake
   was in trying to build Perl on Irix. (no, *using Irix* is not an
   acceptable answer). The problem sounded familiar to Andy Dougherty,
   who suggested an approach to narrow things down. At the end of the
   week we still had not heard back from Philippe.

     http://xrl.us/o7sc

Mortality of objects (*e.g.* %$_) passed as args (#39800)

   Vaclav Ovsik ran afoul of the absence of refcounting to @_ and the
   perl stack.

     Don't do that then
     http://xrl.us/o7sd

"bleadperl -Dm -e1" segfaults on win32 (#39806)

   Rafael made an educated guess that a thread was trying to print
   something using a data structure that had already been freed...

     http://xrl.us/o7se

"eval" and hash access in subroutine (#39816)

   Thomas Ziehmer composed a clever little ditty that shows how he was
   forced to make a bogus "%hash = %hash" assignment to make things work
   correctly. I shall have commented on this next week, since nobody did
   this week.

     Which was last week, anyway
     http://xrl.us/o7sf

"PERL5SHELL" is not checked for tainted data (#39832)

   Paul Fenwick showed how an evil hacker could make Perl inadvertently
   launch Notepad through a "system" call, even in taint mode, since
   "PERL5SHELL" is not subject to the usual tainting strictures.

   Rafael took a stab at closing the hole, and asked for someone with
   access to a Windows machine to validate the fix. Yves confirmed that
   the problem was indeed resolved. Dr. Ruud suggested applying similar
   checks upon $ENV{SystemRoot} and $ENV{windir}.

   Rafael wanted to know what purpose those two variables served.

     Could have been worse
     http://xrl.us/o7sg

Update perlipc.pod TCP server example with respect to safe signals and 
"accept()" (#39835)

   Andy Wardley discovered that the introduction of Safe Signals, from
   5.7.3 onwards had rendered the TCP server examples in "perlipc"
   obsolete. So he updated the POD, in the hope that some kind soul would
   apply it to "blead" (and even "maint") so that others may avoid
   running into the same problem.

     http://xrl.us/o7sh

Script with threads hang when there are nested subthreads (#39839)

   Erland Sommarskog discovered that nested threads don't.

     http://xrl.us/o7si

   Rafael Garcia-Suarez noticed that a "Data::Bind" compilation failure
   could be fixed by diddling with how "Carp" was used, and therefore
   wondered if something hadn't been broken somewhere. The symptoms
   reminded David Landgren that he had encountered similar problems
   testing modules under "blead".

     http://xrl.us/o7sj

Perl5 Bug Summary

   An internal server error prevented the bug summary from being posted
   this week, but I assume we are somewhere around 1500.

     Check 'em out
     http://rt.perl.org/rt3/NoAuth/perl5/Overview.html

New Core Modules

   *   "Term::ANSIColor" version 1.11 released by Russ Allbery.
       Essentially a documentation and test suite enhancements release.

         http://xrl.us/o7sk

   *   "Test::Builder/More/Simple" version 0.64 was rushed to the
       delivery room in an emergency by Michael Schwern. A change in 0.63
       to "test_fail()" broke "Test::Exception" (and possibly many more
       modules as well).

         Author and module are in good health
         http://xrl.us/o7sm

In Brief

   Marcus Holland-Moritz commented on the "state" variables progress,
   saying that if the making iffy constructs results in a slowdown in the
   general case, then those constructs should simply be outlawed as
   errors.

     But the slowdown is only wafer thin
     http://xrl.us/o7sn

   David Nicol suggested that "INIT" blocks could be used to initialise
   "state" variables, and proposed a suitable error message to be used
   when indicating that a state variable was not going to be initialised
   as expected.

     http://xrl.us/o7so

   Ron Savage followed up on the followup from Jan Dubois regarding
   Oracle on ActiveState, noting that he was able to install
   "DBD::Oracle", but unable to do anything with it. Jan suggested a few
   things to try out, and outlined what ActiveState would have to do to
   get it to work in all cases.

     http://xrl.us/o7sp

   Steve Hay noted that a recent version of "ext/threads" doesn't build
   with Borland on Win32, due to differences in header files. Jerry D.
   Hedden found an appropriate "#define" upon which to hang the missing
   definitions.

     http://xrl.us/o7sq

   And Jerry delivered the patch, applied by Rafael.

     http://xrl.us/o7sr

   He then bumped up the version with a fix to stop signalling terminated
   threads from dumping core.

     http://xrl.us/o7ss

   Dr. Ruud wondered whether it was worth chasing down 0 and replacing it
   with "NULL", in pp.c to begin with.

     http://xrl.us/o7st

   statefulness is a property of the code, not the variable, sayeth Dave
   Mitchell.

     http://xrl.us/o7su

   Steve Hay encountered a strange problem with perly.h on Win32, that he
   traced tentatively back to the switch-over from "byacc" to "bison". A
   tweak to regen_perly.pl fixed the problem. So Steve applied the fix,
   but regretted not being able to understand why the error was
   occurring.

     Action and reaction
     http://xrl.us/o7sv

   Jarkko tweaked hv.c and sv.c for "-DPERL_GLOBAL_STRUCT_PRIVATE" and
   expressed disgust at the very existence of a global variable named
   "done_sanity_check". After having stepped back to look at his
   handiwork, he realised that recent work to bring DynaLoader.o into
   "libperl" has rendered the Symbian port of Perl unusable.

     http://xrl.us/o7sw

   Steve Hay discovered that "mod_perl" 1.x and "apache" 1.x do not like
   "blead", possibly due to the way constants have been rejigged.

     http://xrl.us/o7sx

   Sadahiro Tomoyuki produced a patch to update comments in "scan_const",
   since much of it remains as it did at change #780 in the repository,
   which, if the records are correct, is roughly about the same time that
   the porters stopped using punched cards.

     http://xrl.us/o7sy

   He also edited "perlop" to clarify the parsing of quoted constructs,
   so that people could apply some scientific method rather than just
   guessing to see how things behaved.

     http://xrl.us/o7sz

   Yves Orton silenced a "system" warning during tests, although was only
   able to do so for Win32, although he thought that it should be
   possible to tweak it to work in a cross-platform manner.

     http://xrl.us/o7s3

About this summary

   This summary was written by David Landgren.

   Last week's summary, here:

     http://xrl.us/o7s4

   elicited a response from H.Merijn who was concerned with some smoke
   tests throwing out black smoke that was not reproducible with a test
   run by hand.

   Yves Orton also mentioned that he had a documentation patch for the
   heredoc/escaping muddle that was in limbo, and was wondering whether
   it needed improvement or was just pushed down on the stack of things
   to apply.

   If you want a bookmarklet approach to viewing bugs and change reports,
   there are a couple of bookmarklets that you might find useful on my
   page of Perl stuff:

     http://www.landgren.net/perl/

   Weekly summaries are published on http://use.perl.org/ and posted on a
   mailing list, (subscription: [email protected]). The
   archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
   and comments are welcome.

   If you found this summary useful, please consider contributing to the
   Perl Foundation to help support the development of Perl.
-- 
"It's overkill of course, but you can never have too much overkill."