Locale/UTF-8 file path with std::ifstream

[mpalomas]

Hi C++ folks,

I have trouble to open files whose path contains non-ascii characters
with std::ifstream.
For instance let's say i just have a file which has Japanese
characters either in the file path or the file name : ç–‘å•.dat . The
file itself does not contains unicode characters or whatever, it is a
binary file, but the file name, or path, contains non-ascii
characters, here it is Japanese but it could be anything else, i know
nothing about my customers languages.

On Linux, simply doing this :
std::ifstream ifs('ç–‘å•.dat"); just works, i can open the file.. Of
course in my app file path are not hard coded in the source code, the
user choose his file using a file dialog. The file dialog returns me a
QString (Trolltech Qt framework). If the returned QString is named
filepath, then std::ifstream ifs(filepath.toUtf8()) works fine too.

The problem is my application is cross-platform, and on Windows XP the
2 above pieces of code does not work at all ! The ifstream fails to
open the file ... I suspect this is a locale issue, i just know my
Linux distribution uses UTF-8 by default, this must be why it works,
whereas on Windows, it seems strange, ifstream.getloc().name() returns
just "C" , and if i create a default locale with std::locale
my_locale("") , then my_locale.name() returns French_France.1252 ...

I'm stuck with all this locale/encoding problems, it is not clear in
my mind, to solve a problem firstly you need to understand the
problem, and i think i don't I wonder if i have to change the
locale on Windows to a UTF-8 one (i didn't succeed), if i have to use
some conversion functions, or wifstream, or wstring, even after
searching on Google i didn't made any progress, internationalization
stuff does not seem to be trivial with C++ .

Any help, hint, or suggestion would be appreciated !

Regards,

Michaël

 

[Daniel T.]

I have trouble to open files whose path contains non-ascii characters
with std::ifstream.
For instance let's say i just have a file which has Japanese
characters either in the file path or the file name : ã^ñ’.dat . The
file itself does not contains unicode characters or whatever, it is a
binary file, but the file name, or path, contains non-ascii
characters, here it is Japanese but it could be anything else, i know
nothing about my customers languages.

On Linux, simply doing this :
std::ifstream ifs('ã^ñ’.dat"); just works, i can open the file. Of
course in my app file path are not hard coded in the source code, the
user choose his file using a file dialog. The file dialog returns me a
QString (Trolltech Qt framework). If the returned QString is named
filepath, then std::ifstream ifs(filepath.toUtf8()) works fine too.

The problem is my application is cross-platform, and on Windows XP the
2 above pieces of code does not work at all ! The ifstream fails to
open the file ...

Any help, hint, or suggestion would be appreciated !

Can you convert a QString into UTF-16LE? Try using that instead of converting to UTF-8 and see what happens.

 

[mpalomas]

Can you convert a QString into UTF-16LE? Try using that instead of converting to UTF-8 and see what happens.

Yes i can and it worked
I did :
const unsigned short* utf16=filepath.utf16();
And it seems fortunately that on Windows MS added an extension to the
ifstream constructor which takes a const unsigned short* , thus :
std::ifstream ifs(utf16) is fine on Windows, and now all my files with
foreign characters in their path can be loaded.

Thanks a lot Daniel !