Re: Facing problem with HTML::Parser

--001a11433a666cc25e053a420cfe
Content-Type: text/plain; charset=UTF-8

Hi All,


Thank you very much for the extra help you gave me. It's working fine.
 I know how busy you are, so I really appreciated the time you spent for
helping me.

Thanks,
Shivani

On Tue, Aug 16, 2016 at 10:33 PM, Paul Bijnens <[email protected]>
wrote:

> See below:
>
> On 2016-08-11 07:44, Shivani Palle wrote:
>
> Hi,
>
>
> I am facing one issue while using HTML::Parser. Please help me.
>
> *Issue:*
>
> I am using HTML::Parser to parse all the HTML files through out the
> directories to get hard coded strings from the html files(text between the
> tags).
>
> the code is like this:
>
>  #!/usr/bin/perl -w
> package Example;
> require HTML::Parser;
> @Example::ISA = qw(HTML::Parser);
> use File::Find;
> use File::Basename;
>
> #my @files = glob("*.thtml");
> find({ wanted => \&process_file, no_chdir => 1 },
> "/mnt/src/xxx/git/xxx-ive-rdv/");
>
> #foreach $file (@files){
> sub process_file {
>    if (-f $_) {
>        if ($_ =~ m/(.thtml)$/i) {
>    #my($file, $dir, $ext) = fileparse($_);
>    my $file = $_;
>     #step1: Parsing the html file and storing the parsed content in
> another file
>     my $parser = Example->new;
>     $parser->ignore_elements(qw(script)); #ignoring script elements
>     $parser->parse_file($file);
>     print  $parser->{TEXT};
>
>     sub text
>     {
>         my ($self,$text) = @_;
>         $self->{TEXT} .= $text."\n";
>     }
>     open(my $fh, '>', 'parserOutput.txt');
>     print $fh  $parser->{TEXT};
>     close $fh;
>    }
>   }
> }
>
>
>
> *Failing case*:
>
> It is breaking some lines in to two lines.
> For example, I have the following line.
>
> *Before Parsing:*
> <label for="chkInstallAgent">Install Agent for this role</label>
>
> *After Parsing*:
> Install Agent for this
> role
>
> There is no tag in "Install Agent for this role". But still it is breaking
> in to two lines.
> Can you please help me with it.
>
>
> There is a configuration option the HTML::Parser to avoid the breaking:
>
> From the manual page of HTML::Parser:
>
>     $p->unbroken_text
>     $p->unbroken_text( $bool )
>         By default, blocks of text are given to the text handler as soon as
>         possible (but the parser takes care always to break text at a
> boundary
>         between whitespace and non-whitespace so single words and entities
> can
>         always be decoded safely). This might create breaks that make it
> hard
>         to do transformations on the text. When this attribute is enabled,
>         blocks of text are always reported in one piece. This will delay
> the
>         text event until the following (non-text) event has been
> recognized by
>         the parser.
>
>
> (And most other comments e.g. from Shlomi Fish apply as well, to create a
> much cleaner program, of course.
>
>
>
>

--001a11433a666cc25e053a420cfe
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi All,<div><br></div><div><br></div><div><span style=3D"c=
olor:rgb(51,51,51);font-family:&quot;Helvetica Neue&quot;,Helvetica,Arial,s=
ans-serif;font-size:14px;line-height:20px">Thank you very much for the extr=
a help you gave me. It&#39;s working fine.</span></div><div><span style=3D"=
color:rgb(51,51,51);font-family:&quot;Helvetica Neue&quot;,Helvetica,Arial,=
sans-serif;font-size:14px;line-height:20px">=C2=A0I know how busy you are, =
so I really appreciated the time you spent for helping me.</span></div><div=
><span style=3D"color:rgb(51,51,51);font-family:&quot;Helvetica Neue&quot;,=
Helvetica,Arial,sans-serif;font-size:14px;line-height:20px"><br></span></di=
v><div><span style=3D"color:rgb(51,51,51);font-family:&quot;Helvetica Neue&=
quot;,Helvetica,Arial,sans-serif;font-size:14px;line-height:20px">Thanks,</=
span></div><div><span style=3D"color:rgb(51,51,51);font-family:&quot;Helvet=
ica Neue&quot;,Helvetica,Arial,sans-serif;font-size:14px;line-height:20px">=
Shivani</span></div></div><div class=3D"gmail_extra"><br><div class=3D"gmai=
l_quote">On Tue, Aug 16, 2016 at 10:33 PM, Paul Bijnens <span dir=3D"ltr">&=
lt;<a href=3D"mailto:[email protected]" target=3D"_blank">paul.bi=
[email protected]</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex=
">
 =20
   =20
 =20
  <div bgcolor=3D"#FFFFFF" text=3D"#000000">
    <p>See below:<br>
    </p><div><div class=3D"h5">
    <br>
    <div>On 2016-08-11 07:44, Shivani Palle
      wrote:<br>
    </div>
    <blockquote type=3D"cite">
      <div dir=3D"ltr">
        <div>Hi,</div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div>I am facing one issue while using HTML::Parser. Please help
          me.</div>
        <div><br>
        </div>
        <div><i><b>Issue:</b></i></div>
        <div><br>
        </div>
        <div>I am using HTML::Parser to parse all the HTML files through
          out the directories to get hard coded strings from the html
          files(text between the tags).</div>
        <div><br>
        </div>
        <div>the code is like this:</div>
        <div><br>
        </div>
        <div>=C2=A0#!/usr/bin/perl -w</div>
        <div>package Example;</div>
        <div>require HTML::Parser;</div>
        <div>@Example::ISA =3D qw(HTML::Parser);</div>
        <div>use File::Find;</div>
        <div>use File::Basename;</div>
        <div><br>
        </div>
        <div>#my @files =3D glob(&quot;*.thtml&quot;);</div>
        <div>find({ wanted =3D&gt; \&amp;process_file, no_chdir =3D&gt; 1 }=
,
          &quot;/mnt/src/xxx/git/xxx-ive-rdv/<wbr>&quot;);</div>
        <div><br>
        </div>
        <div>#foreach $file (@files){</div>
        <div>sub process_file {</div>
        <div>=C2=A0 =C2=A0if (-f $_) {</div>
        <div>=C2=A0 =C2=A0 =C2=A0 =C2=A0if ($_ =3D~ m/(.thtml)$/i) {</div>
        <div>=C2=A0 =C2=A0#my($file, $dir, $ext) =3D fileparse($_);</div>
        <div>=C2=A0 =C2=A0my $file =3D $_;=C2=A0</div>
        <div>=C2=A0 =C2=A0 #step1: Parsing the html file and storing the pa=
rsed
          content in another file</div>
        <div>=C2=A0 =C2=A0 my $parser =3D Example-&gt;new;</div>
        <div>=C2=A0 =C2=A0 $parser-&gt;ignore_elements(qw(<wbr>script)); #i=
gnoring
          script elements</div>
        <div>=C2=A0 =C2=A0 $parser-&gt;parse_file($file);</div>
        <div>=C2=A0 =C2=A0 print =C2=A0$parser-&gt;{TEXT};</div>
        <div><br>
        </div>
        <div>=C2=A0 =C2=A0 sub text</div>
        <div>=C2=A0 =C2=A0 {</div>
        <div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 my ($self,$text) =3D @_;</div>
        <div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 $self-&gt;{TEXT} .=3D $text.&quot;=
\n&quot;;</div>
        <div>=C2=A0 =C2=A0 }</div>
        <div>=C2=A0 =C2=A0 open(my $fh, &#39;&gt;&#39;, &#39;parserOutput.t=
xt&#39;);</div>
        <div>=C2=A0 =C2=A0 print $fh =C2=A0$parser-&gt;{TEXT};</div>
        <div>=C2=A0 =C2=A0 close $fh;</div>
        <div>=C2=A0 =C2=A0}</div>
        <div>=C2=A0 }</div>
        <div>}</div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><b><i>Failing case</i></b>:</div>
        <div><br>
        </div>
        <div>It is breaking some lines in to two lines.</div>
        <div>For example, I have the following line.</div>
        <div><br>
        </div>
        <div><b>Before Parsing:</b></div>
        <div>&lt;label for=3D&quot;chkInstallAgent&quot;&gt;Install Agent f=
or this
          role&lt;/label&gt;</div>
        <div><br>
        </div>
        <div><b>After Parsing</b>:</div>
        <div>Install Agent for this</div>
        <div>role</div>
        <div><br>
        </div>
        <div>There is no tag in &quot;Install Agent for this role&quot;. Bu=
t still
          it is breaking in to two lines.</div>
        <div>Can you please help me with it.</div>
        <div><br>
        </div>
      </div>
    </blockquote>
    <br>
    </div></div><p>There is a configuration option the HTML::Parser to avoi=
d the
      breaking:<br>
    </p>
    <p>From the manual page of HTML::Parser:<br>
    </p>
    <p>=C2=A0=C2=A0=C2=A0 $p-&gt;unbroken_text<br>
      =C2=A0=C2=A0=C2=A0 $p-&gt;unbroken_text( $bool )<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 By default, blocks of text=
 are given to the text handler
      as soon as<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 possible (but the parser t=
akes care always to break text
      at a boundary<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 between whitespace and non=
-whitespace so single words and
      entities can<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 always be decoded safely).=
 This might create breaks that
      make it hard<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to do transformations on t=
he text. When this attribute is
      enabled,<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 blocks of text are always =
reported in one piece. This will
      delay the<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 text event until the follo=
wing (non-text) event has been
      recognized by<br>
      =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the parser.<br>
      <br>
    </p>
    <p><br>
    </p>
    <p>(And most other comments e.g. from Shlomi Fish apply as well, to
      create a much cleaner program, of course.<br>
    </p>
    <br>
    <br>
    <br>
  </div>

</blockquote></div><br></div>

--001a11433a666cc25e053a420cfe--