How to detect text charset (UTF-8 or Latin-1)

[Thomas Armstrong]

Hi.

I'm creating a Perl script extracting text from a webpage using LWP,
and want to check if text is UTF-8 or Latin-1 encoded?

Is there any known function? I don't know if "use utf8;" is enough

Thank you very much in advance.

 

[smallpond]

Hi.

I'm creating a Perl script extracting text from a webpage using LWP,
and want to check if text is UTF-8 or Latin-1 encoded?

Is there any known function? I don't know if "use utf8;" is enough

Thank you very much in advance.

Parse the Content-type header, for example:
content="text/html; charset=UTF-8"

Web pages that lie or omit the Content-type are not scarce,
unfortunately.

 

[Jürgen Exner]

Check the <META Charset='...'> tag.

jue

 

[Joost Diepenmaat]

Check the <META Charset='...'> tag.

jue

or the xml prolog (if there is one)

Joost.