First draft of work published which removes the dependency on HTML::Parser from HTTP::Message

I now have working code published which allows HTTP::Message to work without
the dependency on HTML::Parser. This is useful because it's a step towards
splitting out some of the HTTP modules into their own distribution which does
not have this dependency, which in turn depends on a C compiler. So, this
project could help allow parts of LWP to be used in places where a C compiler
is not available, or when it would be more convenient to distribute one code
line that could be used directly on multiple architectures.
( But this is not the only the use of HTML::Parser by the distribution.
LWP::UserAgent makes use of HTML::HeadParser which in turn uses HTML::Parser. )

My code is here:

http://github.com/markstos/libwww-perl/tree/remove-html-parser-dependency

The solution passes the numerous existing tests for charset detection, as well
as a new one I added.

However, I'm not yet recommending that the work be merged because the approach
is not clean.

Essentially I have have embedded a fairly full-featured Pure Perl HTML parser
into HTTP::Message. :) The code was taken from my fork of the
"HTML::Parser::Simple" project and specialized some for this case:

 http://github.com/markstos/html--parser--simple

I think a cleaner approach would be to publish this Pure Perl HTML parser, and
then have an option to use it if HTML::Parser is not available.

A little history about HTML::Parser::Simple:

Ron Savage created the project based on the htmlparser.js JavaScript parser by
John Resig.  This branch was not trying to be particurly compatible with
anything. It defines a new API.  In particular, it bundles a parse tree
*consumer*, Tree::Simple, as well as parse tree producer.

I forked the project and made some incompatible changes to pursue a different
goal: Create a pure Perl HTML parser that is compatible with the HTML::Parser
API. Or specifically, I wanted emulate the HTML::Parser 2.x API enough so that
my parser could be used in place of it with HTML::FillInForm. My work met that
goal-- it can be used to pass all HTML::FillInForm tests with some minor
failures that I don't think matter.

This new case of parsing meta tags is another specialized use of the parser
that gives me another reason to publish the work.

Here's the problem: While I care about these specific goals for an HTML::Parser
that is "compatible enough", I'm not really interested in personally pursuing
the idea of a Pure Perl HTML::Parser that is 100% compatible with HTML::Parser
just for the sake of it.  In short, I'm sure there will be change requests
beyond what the uses I care about, and I'm not interested in maintaining the
module to extend it for other uses.

There's also the matter of what to name it, since a version of
HTML::Parser::Simple already exists. There's always HTML::Parser::PP or
HTML::Parser::PurePerl, but those names just invite the idea that the goal is
to be 100% compatible with HTML::Parser.

I'll discuss the matter further with Ron Savage to get his thoughts.

Feedback on the topic from other LWP users is welcome.

    Mark

-- 
http://mark.stosberg.com/