identify the language of a web page

U

usgog

Suppose I need to classify 10000 web pages based on their languages.
What should I look for to determine the language of each web page? Any
advice is welcome.
 
A

Andreas Prilop

Suppose I need to classify 10000 web pages based on their languages.
What should I look for to determine the language of each web page?

The "lang" attribute in HTML; the "xml:lang" attribute in XHTML.
 
R

Richard Tobin

Suppose I need to classify 10000 web pages based on their languages.
What should I look for to determine the language of each web page? Any
advice is welcome.

Assuming you want to do this by inspection of the text (rather than
looking for xml:lang and the like), Google for language
identification. The first page lists several tools and a research
bibliography on the subject.

-- Richard
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top