HTML::Parser not stripping out comments

Jay · Jun 14, 2004

I'm trying to get HTML:

arser to strip out the comments using some of the
sample code from the man page. I'm using the ignore_elements and I still
get comments in the dtext. Am I doing something wrong?

Tia,
Jay

CODE:
use HTML:

arser ();

# Create parser object
$p = HTML:

arser->new( api_version => 3,
start_h => [\&start, "tagname, attr"],
end_h => [\&end, "tagname"],
comment_h => [\&comment, "self,text"],
text_h => [\&dtext, "self,text"],
marked_sections => 1,
);

$p->ignore_elements( qw(script, comment, style) );
$p->strict_comment( [1] );
# Parse directly from file
$p->parse_file("0");

sub start {
my($self, $tagname, $attr, $attrseq, $origtext) = @_;
#...
}

sub end {
my($self, $tagname, $origtext) = @_;
#...
}

sub text {
my($self, $origtext, $is_cdata) = @_;
#...
}
sub comment{
#my($self, $origtext, $is_cdata) = @_;
#...
}

sub dtext {
my($self, $dtext ) = @_;
$dtext=~s/\s+/ /g;
print "DTEXT: $dtext\n";
}

Example of some of the output from parsing some web page:

DTEXT: 
DTEXT:
DTEXT:

Gisle Aas · Jun 15, 2004

Jay said:
I'm trying to get HTML:arser to strip out the comments using some of the
sample code from the man page. I'm using the ignore_elements and I still
get comments in the dtext. Am I doing something wrong?

Probably

CODE:
use HTML:arser ();

# Create parser object
$p = HTML:arser->new( api_version => 3,
start_h => [\&start, "tagname, attr"],
end_h => [\&end, "tagname"],
comment_h => [\&comment, "self,text"],
text_h => [\&dtext, "self,text"],
marked_sections => 1,

You really want marked_sections to be enabled?

);

$p->ignore_elements( qw(script, comment, style) );

You need to remove all the "," here. Otherwise they end up part of
the strings passed to ignore_elements. Also there is no <comment> tag
in HTML, so there is not comment element either.

The extra commas probably explain why you get the JavaScript comment
reported to your &dtext callback. Anything between <script> and

$p->strict_comment( [1] );

I don't think you actually want strict_comment enabled either. A
plain 1 is also a perfectly fine true boolean.

# Parse directly from file
$p->parse_file("0");

That's a strange file name.

Regards,
Gisle

Jay · Jun 15, 2004

Gisle Aas said:
Jay said:

I'm trying to get HTML:arser to strip out the comments using some of the
sample code from the man page. I'm using the ignore_elements and I still
get comments in the dtext. Am I doing something wrong?

Click to expand...

Probably

CODE:
use HTML:arser ();

# Create parser object
$p = HTML:arser->new( api_version => 3,
start_h => [\&start, "tagname, attr"],
end_h => [\&end, "tagname"],
comment_h => [\&comment, "self,text"],
text_h => [\&dtext, "self,text"],
marked_sections => 1,

Click to expand...

You really want marked_sections to be enabled?

);

$p->ignore_elements( qw(script, comment, style) );

Click to expand...

You need to remove all the "," here. Otherwise they end up part of
the strings passed to ignore_elements. Also there is no <comment> tag
in HTML, so there is not comment element either.

The extra commas probably explain why you get the JavaScript comment
reported to your &dtext callback. Anything between <script> and

$p->strict_comment( [1] );

Click to expand...

I don't think you actually want strict_comment enabled either. A
plain 1 is also a perfectly fine true boolean.

# Parse directly from file
$p->parse_file("0");

Click to expand...

That's a strange file name.

Regards,
Gisle

Thanks Gisle,
I will look at this some more with your reccomendations and post the
results.
yes, a strange filename.

Jay

Jay · Jun 15, 2004

Jay said:
I'm trying to get HTML:arser to strip out the comments using some of the
sample code from the man page. I'm using the ignore_elements and I still
get comments in the dtext. Am I doing something wrong?

Tia,
Jay

CODE:
use HTML:arser ();

# Create parser object
$p = HTML:arser->new( api_version => 3,
start_h => [\&start, "tagname, attr"],
end_h => [\&end, "tagname"],
comment_h => [\&comment, "self,text"],
text_h => [\&dtext, "self,text"],
marked_sections => 1,
);

$p->ignore_elements( qw(script, comment, style) );
$p->strict_comment( [1] );
# Parse directly from file
$p->parse_file("0");

sub start {
my($self, $tagname, $attr, $attrseq, $origtext) = @_;
#...
}

sub end {
my($self, $tagname, $origtext) = @_;
#...
}

sub text {
my($self, $origtext, $is_cdata) = @_;
#...
}
sub comment{
#my($self, $origtext, $is_cdata) = @_;
#...
}

sub dtext {
my($self, $dtext ) = @_;
$dtext=~s/\s+/ /g;
print "DTEXT: $dtext\n";
}

Example of some of the output from parsing some web page:

DTEXT: 
DTEXT:
DTEXT:

That did the trick, thanks alot.

Jay

Problem with body text extraction with HTML::Parser	1	Dec 13, 2005
HTML::Parser	10	Aug 26, 2004
Why is this WordPress comments form not submitting?	1	Jan 12, 2020
adding a data field to a subclass of HTML::Parser	3	Mar 12, 2007
oriented object programming	1	Jul 21, 2003
strip given HTML tags	1	Sep 24, 2003
PERL/HTML: extract repetitive information	2	Jul 11, 2007
parsing in wrong order?	1	Oct 12, 2004

HTML::Parser not stripping out comments

Jay

Gisle Aas

Jay

Jay

Ask a Question

Similar Threads

Members online

Forum statistics

Latest Threads