Reading UTF-8 string from file with read() function.

[Sergei]

Hi,
I need to read a string from UTF-8 encoded text file.
I know at which byte position the string starts and its length (also
in byte units).
The problem is that read( FILEHANDLE,SCALAR,LENGTH) function takes
LENGTH in character units, not in bytes.
I've tried to open the file in binary mode instead of UTF-8, so I can
read the correct length, but then I can't process the string with
regular expressions correctly as Perl thinks it's in binary encoding,
not UTF-8.
Also, I've tried to read the string using getc() function, but it is
unacceptably slow.
Is there any solution ?
Thanks a lot,
--Sergei

 

[Brian McCauley]

I need to read a string from UTF-8 encoded text file.
I know at which byte position the string starts and its length (also
in byte units).
The problem is that read( FILEHANDLE,SCALAR,LENGTH) function takes
LENGTH in character units, not in bytes.
I've tried to open the file in binary mode instead of UTF-8, so I can
read the correct length, but then I can't process the string with
regular expressions correctly as Perl thinks it's in binary encoding,
not UTF-8.

Is there any solution ?

Read the string from file as binary and then utf8::decode() it.

 

[Sergei]

...
Read the string from file as binary and then utf8::decode() it.

Brian,
You are right. I did:
use Encode 'decode_utf8';
$Unicode = decode_utf8($bytes);
And it works !
Thanks a lot !
--Sergei

 

[nobull]

use Encode 'decode_utf8';
$Unicode = decode_utf8($bytes);
And it works !

Yes, you can use Encode::decode_utf8() instead of the builtin
utf8::decode() if you like. Note: when called with a single agument
Encode::decode_utf8() is simply a wrapper for utf8::decode().

 

[Sergei]

Yes, you can use Encode::decode_utf8() instead of the builtin
utf8::decode() if you like. Note: when called with a single agument
Encode::decode_utf8() is simply a wrapper for utf8::decode().

I didn't know I could use it like this.
This way it's even better.
Thanks!
(Other's messages were useful too. What an excellent thing this news
group! Thanks a lot everybody!)