Re: Facing problem with HTML::Parser
[email protected] (Paul Bijnens) Tue, 16 Aug 2016 19:03:39 +0200
| Newsgroups | perl.libwww |
|---|---|
| Message-ID | <[email protected]> |
--------------6DCDAAD5E979BA590674B9AE
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
See below:
On 2016-08-11 07:44, Shivani Palle wrote:
> Hi,
>
>
> I am facing one issue while using HTML::Parser. Please help me.
>
> /*Issue:*/
>
> I am using HTML::Parser to parse all the HTML files through out the
> directories to get hard coded strings from the html files(text between
> the tags).
>
> the code is like this:
>
> #!/usr/bin/perl -w
> package Example;
> require HTML::Parser;
> @Example::ISA = qw(HTML::Parser);
> use File::Find;
> use File::Basename;
>
> #my @files = glob("*.thtml");
> find({ wanted => \&process_file, no_chdir => 1 },
> "/mnt/src/xxx/git/xxx-ive-rdv/");
>
> #foreach $file (@files){
> sub process_file {
> if (-f $_) {
> if ($_ =~ m/(.thtml)$/i) {
> #my($file, $dir, $ext) = fileparse($_);
> my $file = $_;
> #step1: Parsing the html file and storing the parsed content in
> another file
> my $parser = Example->new;
> $parser->ignore_elements(qw(script)); #ignoring script elements
> $parser->parse_file($file);
> print $parser->{TEXT};
>
> sub text
> {
> my ($self,$text) = @_;
> $self->{TEXT} .= $text."\n";
> }
> open(my $fh, '>', 'parserOutput.txt');
> print $fh $parser->{TEXT};
> close $fh;
> }
> }
> }
>
>
>
> */Failing case/*:
>
> It is breaking some lines in to two lines.
> For example, I have the following line.
>
> *Before Parsing:*
> <label for="chkInstallAgent">Install Agent for this role</label>
>
> *After Parsing*:
> Install Agent for this
> role
>
> There is no tag in "Install Agent for this role". But still it is
> breaking in to two lines.
> Can you please help me with it.
>
There is a configuration option the HTML::Parser to avoid the breaking:
From the manual page of HTML::Parser:
$p->unbroken_text
$p->unbroken_text( $bool )
By default, blocks of text are given to the text handler as soon as
possible (but the parser takes care always to break text at a
boundary
between whitespace and non-whitespace so single words and
entities can
always be decoded safely). This might create breaks that make
it hard
to do transformations on the text. When this attribute is enabled,
blocks of text are always reported in one piece. This will
delay the
text event until the following (non-text) event has been
recognized by
the parser.
(And most other comments e.g. from Shlomi Fish apply as well, to create
a much cleaner program, of course.
--------------6DCDAAD5E979BA590674B9AE
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>See below:<br>
</p>
<br>
<div class="moz-cite-prefix">On 2016-08-11 07:44, Shivani Palle
wrote:<br>
</div>
<blockquote
cite="mid:CAH0Myt8GNrMXqHB4ge-FnB+qkpzL4K_5SAVx=eBcU=QqaN5hMA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>Hi,</div>
<div><br>
</div>
<div><br>
</div>
<div>I am facing one issue while using HTML::Parser. Please help
me.</div>
<div><br>
</div>
<div><i><b>Issue:</b></i></div>
<div><br>
</div>
<div>I am using HTML::Parser to parse all the HTML files through
out the directories to get hard coded strings from the html
files(text between the tags).</div>
<div><br>
</div>
<div>the code is like this:</div>
<div><br>
</div>
<div> #!/usr/bin/perl -w</div>
<div>package Example;</div>
<div>require HTML::Parser;</div>
<div>@Example::ISA = qw(HTML::Parser);</div>
<div>use File::Find;</div>
<div>use File::Basename;</div>
<div><br>
</div>
<div>#my @files = glob("*.thtml");</div>
<div>find({ wanted => \&process_file, no_chdir => 1 },
"/mnt/src/xxx/git/xxx-ive-rdv/");</div>
<div><br>
</div>
<div>#foreach $file (@files){</div>
<div>sub process_file {</div>
<div> if (-f $_) {</div>
<div> if ($_ =~ m/(.thtml)$/i) {</div>
<div> #my($file, $dir, $ext) = fileparse($_);</div>
<div> my $file = $_; </div>
<div> #step1: Parsing the html file and storing the parsed
content in another file</div>
<div> my $parser = Example->new;</div>
<div> $parser->ignore_elements(qw(script)); #ignoring
script elements</div>
<div> $parser->parse_file($file);</div>
<div> print $parser->{TEXT};</div>
<div><br>
</div>
<div> sub text</div>
<div> {</div>
<div> my ($self,$text) = @_;</div>
<div> $self->{TEXT} .= $text."\n";</div>
<div> }</div>
<div> open(my $fh, '>', 'parserOutput.txt');</div>
<div> print $fh $parser->{TEXT};</div>
<div> close $fh;</div>
<div> }</div>
<div> }</div>
<div>}</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><b><i>Failing case</i></b>:</div>
<div><br>
</div>
<div>It is breaking some lines in to two lines.</div>
<div>For example, I have the following line.</div>
<div><br>
</div>
<div><b>Before Parsing:</b></div>
<div><label for="chkInstallAgent">Install Agent for this
role</label></div>
<div><br>
</div>
<div><b>After Parsing</b>:</div>
<div>Install Agent for this</div>
<div>role</div>
<div><br>
</div>
<div>There is no tag in "Install Agent for this role". But still
it is breaking in to two lines.</div>
<div>Can you please help me with it.</div>
<div><br>
</div>
</div>
</blockquote>
<br>
<p>There is a configuration option the HTML::Parser to avoid the
breaking:<br>
</p>
<p>From the manual page of HTML::Parser:<br>
</p>
<p> $p->unbroken_text<br>
$p->unbroken_text( $bool )<br>
By default, blocks of text are given to the text handler
as soon as<br>
possible (but the parser takes care always to break text
at a boundary<br>
between whitespace and non-whitespace so single words and
entities can<br>
always be decoded safely). This might create breaks that
make it hard<br>
to do transformations on the text. When this attribute is
enabled,<br>
blocks of text are always reported in one piece. This will
delay the<br>
text event until the following (non-text) event has been
recognized by<br>
the parser.<br>
<br>
</p>
<p><br>
</p>
<p>(And most other comments e.g. from Shlomi Fish apply as well, to
create a much cleaner program, of course.<br>
</p>
<br>
<br>
<br>
</body>
</html>
--------------6DCDAAD5E979BA590674B9AE--