utf8 and HTML Entities

Nick Gerber · Sep 19, 2007

Hi

I'm lost :-(

I have a string encodet in utf8 with part HTML Entities and part
characters in utf-8.

How do I translate the HTML Entities into proper utf-8?

Thanks

Ben Bullock · Sep 19, 2007

I have a string encodet in utf8 with part HTML Entities and part
characters in utf-8.

How do I translate the HTML Entities into proper utf-8?

Since this must be a commonly encountered problem, my first guess would be
to try cpan to save myself the bother of writing it myself. I rapidly found:

http://search.cpan.org/~gaas/HTML-Parser-3.56/lib/HTML/Entities.pm

Please note that I can't vouch for this software since I have not tried it.

As far as utf8 goes you need to use the "Encode" module.

Nick Gerber · Sep 20, 2007

I tried HTML/Entities.pm, but it didn't do the trick for me. But, it was
me that could not make it to do the conversion for me. I'll try again.

Thanks

Helmut Wollmersdorfer · Sep 21, 2007

Nick said:
I tried HTML/Entities.pm, but it didn't do the trick for me. But, it was
me that could not make it to do the conversion for me. I'll try again.

That's my way which works for millions of HTML (or XML) files:

use HTML::Entities;

my $ENCODING = 'utf8'; # or iso-8859-7, CP1250 etc.

open (HTML, "<:encoding($ENCODING)", "$DIR/$file")
or die "Can't open: $1!";

my $data = <HTML>;

my $content = decode_entities($data);

binmode(STDOUT, ":utf8");

print "$content\n";

It is also save (in most cases) to use

my $content = decode_entities(decode_entities($data));

which decodes something like

&amp;

| $ perl -version
| This is perl, v5.8.8 built for i486-linux-gnu-thread-multi

Helmut Wollmersdorfer

Mumia W. · Sep 21, 2007

Hi

I'm lost :-(

I have a string encodet in utf8 with part HTML Entities and part
characters in utf-8.

How do I translate the HTML Entities into proper utf-8?

Thanks

Click to expand...

Should be enough here to get you going:

[ long program snipped ]

No, that's too much.

Mr. Gerber didn't post any code or data, and so he didn't get many
responses because no one knew exactly what he was talking about.

As Mr. Bullock said, HTML::Entities should do it. Here is an example:

#!/usr/bin/perl
use strict;
use warnings;
use HTML::Entities;

binmode(STDOUT, ':utf8');
local $/;
my $data = <DATA>;

$data = decode_entities($data);

print $data, "\n";

__DATA__
膄膅膆
á é í ó ú
ä ë ï ö ü

Nick Gerber · Sep 25, 2007

Thanks all.

Nick

Hi

I'm lost :-(

I have a string encodet in utf8 with part HTML Entities and part
characters in utf-8.

How do I translate the HTML Entities into proper utf-8?

Thanks

Click to expand...

Should be enough here to get you going:

[ long program snipped ]

Click to expand...

No, that's too much.

Mr. Gerber didn't post any code or data, and so he didn't get many
responses because no one knew exactly what he was talking about.

As Mr. Bullock said, HTML::Entities should do it. Here is an example:

#!/usr/bin/perl
use strict;
use warnings;
use HTML::Entities;

binmode(STDOUT, ':utf8');
local $/;
my $data = <DATA>;

$data = decode_entities($data);

print $data, "\n";

__DATA__
膄膅膆
á é í ó ú
ä ë ï ö ü

HTML::Entities & UTF8	1	Nov 10, 2008
Batch Convert HTML to UTF-8 Files	2	Oct 2, 2023
Cyrillic text from file - set utf8 in cmd, unknown characters output anyway	0	Nov 11, 2022
Problem with a login script, SESSION user rights and put this together so it works with the other pages and MySQL. Code examples.	2	May 5, 2023
Is the pod of Encode::MIME::Header giving wrong advice?	5	Apr 23, 2014
DBMS bridge entity operations	2	Sep 19, 2022
utf8, length and syswrite are killing me	2	Feb 17, 2010
East european characters from LaTex to UTF8	4	Nov 30, 2007

utf8 and HTML Entities

Nick Gerber

Ben Bullock

Nick Gerber

Helmut Wollmersdorfer

Mumia W.

Nick Gerber

Ask a Question

Similar Threads

Members online

Forum statistics

Latest Threads