Question about language setting

Discussion in 'Perl Misc' started by Dave Saville, Dec 23, 2013.

  1. Dave Saville

    Dave Saville Guest

    I have a perl script that checks a program's "data base" for
    consistancy.

    A German user said it would not run and threw a "Feature bundle 5.0.0"
    not supported from feature.pm. This was because I call Carp.

    He now says

    <qoute>
    the perl problem seems to be solved: it's not a problem with
    installation or version. The problem is the setting for "LANG": It has
    to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    for german!
    </quote>

    Surely (base) perl modules aren't dependent on language?

    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 23, 2013
    #1
    1. Advertising

  2. "Dave Saville" <> writes:
    > A German user said it would not run and threw a "Feature bundle 5.0.0"
    > not supported from feature.pm. This was because I call Carp.
    >
    > He now says
    >
    > <qoute>
    > the perl problem seems to be solved: it's not a problem with
    > installation or version. The problem is the setting for "LANG": It has
    > to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    > for german!
    > </quote>
    >
    > Surely (base) perl modules aren't dependent on language?


    According to a completely random web search result,

    https://www.illumos.org/issues/1203

    the issue is possibly that the 5.0 becomes 5,0 when formatted
    according to German conventions. The thread and related bug & bugfix has
    some more information on this (AFAIK, 'feature bundle 5.0.0' means
    'Perl5').
     
    Rainer Weikusat, Dec 23, 2013
    #2
    1. Advertising

  3. Dave Saville

    Dave Saville Guest

    On Mon, 23 Dec 2013 16:13:08 UTC, Rainer Weikusat
    <> wrote:

    > "Dave Saville" <> writes:
    > > A German user said it would not run and threw a "Feature bundle 5.0.0"
    > > not supported from feature.pm. This was because I call Carp.
    > >
    > > He now says
    > >
    > > <qoute>
    > > the perl problem seems to be solved: it's not a problem with
    > > installation or version. The problem is the setting for "LANG": It has
    > > to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    > > for german!
    > > </quote>
    > >
    > > Surely (base) perl modules aren't dependent on language?

    >
    > According to a completely random web search result,
    >
    > https://www.illumos.org/issues/1203
    >
    > the issue is possibly that the 5.0 becomes 5,0 when formatted
    > according to German conventions. The thread and related bug & bugfix has
    > some more information on this (AFAIK, 'feature bundle 5.0.0' means
    > 'Perl5').


    Well that seems to suggest that it is not perl but something else

    <qoute>
    root source found, the bug is not in perl, its in Illumos locales,
    </quote>

    Whatever that is. Surely perl must be used in non English speaking
    countries? :)
    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 23, 2013
    #3
  4. "Dave Saville" <> writes:
    > On Mon, 23 Dec 2013 16:13:08 UTC, Rainer Weikusat
    > <> wrote:
    >
    >> "Dave Saville" <> writes:
    >> > A German user said it would not run and threw a "Feature bundle 5.0.0"
    >> > not supported from feature.pm. This was because I call Carp.
    >> >
    >> > He now says
    >> >
    >> > <qoute>
    >> > the perl problem seems to be solved: it's not a problem with
    >> > installation or version. The problem is the setting for "LANG": It has
    >> > to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    >> > for german!
    >> > </quote>
    >> >
    >> > Surely (base) perl modules aren't dependent on language?

    >>
    >> According to a completely random web search result,
    >>
    >> https://www.illumos.org/issues/1203
    >>
    >> the issue is possibly that the 5.0 becomes 5,0 when formatted
    >> according to German conventions. The thread and related bug & bugfix has
    >> some more information on this (AFAIK, 'feature bundle 5.0.0' means
    >> 'Perl5').

    >
    > Well that seems to suggest that it is not perl but something else
    >
    > <qoute>
    > root source found, the bug is not in perl, its in Illumos locales,
    > </quote>
    >
    > Whatever that is. Surely perl must be used in non English speaking
    > countries? :)


    As I already wrote: The thread and the associated OpenSlowalrus bug &
    bug fix provide some more background information on this. In particular,
    the suggested workaround (set LC_NUMERIC to a 'decimal point' locale)
    should be sufficient to determine if this is really the same
    problem. Depending on how you feel about that, you might also just tell
    your user that "Locale support on your system is obviously broken, get
    of my lawn, sucker!" -- the open source way of dealing with
    'uninteresting' user problems ...
     
    Rainer Weikusat, Dec 23, 2013
    #4
  5. On 2013-12-23 15:39, Dave Saville <> wrote:
    > I have a perl script that checks a program's "data base" for
    > consistancy.
    >
    > A German user said it would not run and threw a "Feature bundle 5.0.0"
    > not supported from feature.pm. This was because I call Carp.


    It would be useful to know where this error occurs. Just "a perl script"
    is a bit vague.

    I get this error if I write something like

    use feature ':5.0.0';

    but that is independent of the locale setting. Feature.pm was introduced
    in 5.10 and there aren't any feature bundles less than :5.10.

    Also, details about the system in question (OS/distribution, perl
    version) would also be useful.

    > He now says
    >
    ><qoute>
    > the perl problem seems to be solved: it's not a problem with
    > installation or version. The problem is the setting for "LANG": It has
    > to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    > for german!
    ></quote>
    >
    > Surely (base) perl modules aren't dependent on language?
    >


    They shouldn't be. It might be a bug in perl, in the module or in a
    system library. Or it might be a bug in your script. It is impossible to
    tell with the information you have given.

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
     
    Peter J. Holzer, Dec 23, 2013
    #5
  6. "Peter J. Holzer" <> writes:
    > On 2013-12-23 15:39, Dave Saville <> wrote:


    [...]

    >> He now says
    >>
    >><qoute>
    >> the perl problem seems to be solved: it's not a problem with
    >> installation or version. The problem is the setting for "LANG": It has
    >> to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    >> for german!
    >></quote>
    >>
    >> Surely (base) perl modules aren't dependent on language?

    >
    > They shouldn't be. It might be a bug in perl, in the module or in a
    > system library. Or it might be a bug in your script. It is impossible to
    > tell with the information you have given.


    Considering the statement from 'the user', this must be something which
    causes the behaviour of some perl code to change because it is using a
    German locale. Considering what is in a locale definition, this makes
    LC_NUMERIC the most likely candidate, especially considering that this
    is the only locale-information used as part of version checking (by
    Perl_upg_version in util.c).
     
    Rainer Weikusat, Dec 23, 2013
    #6
  7. On 2013-12-23 22:50, Rainer Weikusat <> wrote:
    > "Peter J. Holzer" <> writes:
    >> On 2013-12-23 15:39, Dave Saville <> wrote:
    >>> He now says
    >>>
    >>><qoute>
    >>> the perl problem seems to be solved: it's not a problem with
    >>> installation or version. The problem is the setting for "LANG": It has
    >>> to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    >>> for german!
    >>></quote>
    >>>
    >>> Surely (base) perl modules aren't dependent on language?

    >>
    >> They shouldn't be. It might be a bug in perl, in the module or in a
    >> system library. Or it might be a bug in your script. It is impossible to
    >> tell with the information you have given.

    >
    > Considering the statement from 'the user', this must be something which
    > causes the behaviour of some perl code to change because it is using a
    > German locale. Considering what is in a locale definition, this makes
    > LC_NUMERIC the most likely candidate, especially considering that this
    > is the only locale-information used as part of version checking (by
    > Perl_upg_version in util.c).


    Yes, I know. But I couldn't reproduce it. And that's not very
    surprising: Many people use perl with a German locale, so it can't be
    something generic like "perl doesn't work with a German locale" or "use
    feature doesn't work with a German locale". It has to be something more
    specific, like a bug in the locale implementation of the user's system
    (but Dave hasn't told us what system it is) or the way a certain feature
    is used (but Dave hasn't told us anything about his script). But it is
    impossible to find out what it is unless Dave volunteers more
    information.

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
     
    Peter J. Holzer, Dec 23, 2013
    #7
  8. Dave Saville

    Dave Saville Guest

    On Mon, 23 Dec 2013 23:14:54 UTC, "Peter J. Holzer"
    <> wrote:

    > On 2013-12-23 22:50, Rainer Weikusat <> wrote:
    > > "Peter J. Holzer" <> writes:
    > >> On 2013-12-23 15:39, Dave Saville <> wrote:
    > >>> He now says
    > >>>
    > >>><qoute>
    > >>> the perl problem seems to be solved: it's not a problem with
    > >>> installation or version. The problem is the setting for "LANG": It has
    > >>> to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    > >>> for german!
    > >>></quote>
    > >>>
    > >>> Surely (base) perl modules aren't dependent on language?
    > >>
    > >> They shouldn't be. It might be a bug in perl, in the module or in a
    > >> system library. Or it might be a bug in your script. It is impossible to
    > >> tell with the information you have given.

    > >
    > > Considering the statement from 'the user', this must be something which
    > > causes the behaviour of some perl code to change because it is using a
    > > German locale. Considering what is in a locale definition, this makes
    > > LC_NUMERIC the most likely candidate, especially considering that this
    > > is the only locale-information used as part of version checking (by
    > > Perl_upg_version in util.c).

    >
    > Yes, I know. But I couldn't reproduce it. And that's not very
    > surprising: Many people use perl with a German locale, so it can't be
    > something generic like "perl doesn't work with a German locale" or "use
    > feature doesn't work with a German locale". It has to be something more
    > specific, like a bug in the locale implementation of the user's system
    > (but Dave hasn't told us what system it is) or the way a certain feature
    > is used (but Dave hasn't told us anything about his script). But it is
    > impossible to find out what it is unless Dave volunteers more
    > information.


    A little difficult to post extra info from the user as I only have a
    jpeg of the errors rather than something I can cut and paste :-(

    The script starts:

    use strict;
    use warnings;
    use Carp:
    use Encode;

    Compilation failed in require at .... line 3 - ie the "use Carp" line.

    System is OS/2 his perl is 5.10.0. (Yes I know :) )

    If you follow the links from the posted forum you end up with a C test
    program. I have built that on my system but as I am in the UK my
    system is defaulting to using a 'point' anyway. I have yet to persuade
    it to start up in a different locale. I know how to get a program to
    use a different language but that does not seem to help. I have posted
    this problem to an OS/2 tech group in the hope they may throw some
    light on the problem - assuming it is a similar one in that
    setlocale() is not working correctly as per the forum post.
    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 24, 2013
    #8
  9. "Dave Saville" <> writes:
    > On Mon, 23 Dec 2013 23:14:54 UTC, "Peter J. Holzer"
    > <> wrote:
    >> On 2013-12-23 22:50, Rainer Weikusat <> wrote:
    >> > "Peter J. Holzer" <> writes:
    >> >> On 2013-12-23 15:39, Dave Saville <> wrote:
    >> >>> He now says
    >> >>>
    >> >>><qoute>
    >> >>> the perl problem seems to be solved: it's not a problem with
    >> >>> installation or version. The problem is the setting for "LANG": It has
    >> >>> to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    >> >>> for german!
    >> >>></quote>
    >> >>>


    [...]

    >> > Considering the statement from 'the user', this must be something which
    >> > causes the behaviour of some perl code to change because it is using a
    >> > German locale. Considering what is in a locale definition, this makes
    >> > LC_NUMERIC the most likely candidate,


    [...]

    >> Yes, I know. But I couldn't reproduce it. And that's not very
    >> surprising: Many people use perl with a German locale, so it can't be
    >> something generic like "perl doesn't work with a German locale" or "use
    >> feature doesn't work with a German locale". It has to be something more
    >> specific, like a bug in the locale implementation of the user's system


    [...]

    > A little difficult to post extra info from the user as I only have a
    > jpeg of the errors rather than something I can cut and paste :-(
    >
    > The script starts:
    >
    > use strict;
    > use warnings;
    > use Carp:
    > use Encode;
    >
    > Compilation failed in require at .... line 3 - ie the "use Carp" line.
    >
    > System is OS/2 his perl is 5.10.0. (Yes I know :) )
    >
    > If you follow the links from the posted forum you end up with a C test
    > program. I have built that on my system but as I am in the UK my
    > system is defaulting to using a 'point' anyway. I have yet to persuade
    > it to start up in a different locale.


    According to some more searching around on the web, the OS/2 command to
    set a locale is (example)

    set lang=de_DE
    [http://www.borgendale.com/locale.htm]

    Results as to whether or not setlocale works on OS/2 are somewhat
    unclear. Assuming your system starts with 'English' language settings,
    you could use a modified test program,

    -----------
    #include <locale.h>
    #include <stdio.h>

    int main(void) {
    printf("%f\n",2.5);
    setlocale(LC_NUMERIC, "de_DE");
    printf("%f\n",2.5);
    setlocale(LC_NUMERIC, "C");
    printf("%f\n",2.5);

    return 0;
    }
    -----------

    this should print (possibly with variations in the number of trailing
    zeroes)

    2.500000
    2,500000
    2.500000

    If it doesn't, in particular, if there's a dot instead of a comma in the
    second line, setlocale doesn't work as it is supposed to and if you
    can't switch from English to German in this way, chances are very high
    that switching from German to English also won't work.
     
    Rainer Weikusat, Dec 27, 2013
    #9
  10. Dave Saville

    Dave Saville Guest

    On Fri, 27 Dec 2013 13:58:43 UTC, Rainer Weikusat
    <> wrote:

    >
    > If it doesn't, in particular, if there's a dot instead of a comma in the
    > second line, setlocale doesn't work as it is supposed to and if you
    > can't switch from English to German in this way, chances are very high
    > that switching from German to English also won't work.
    >


    It would appear we have the same or very similar problem with
    setlocale() as reported upthread. It is being looked into. FWIW I now
    have the original error in text form:

    Feature bundle "5.0.0" is not supported by Perl 5.10.0 at
    D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm line 207
    feature::croak('Feature bundle "5.0.0" is not supported by Perl
    5.10.0') called at D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm line 201
    feature::unknown_feature_bundle(5.0.0) called at
    D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm line 152
    feature::import('feature', ':5.0.0') called at
    D:/PROGRAMS/PERL/LIB/5.10.0/Exporter.pm line 3
    Exporter::BEGIN() called at D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm
    line 0
    eval {...} called at D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm line 0
    require Exporter.pm called at D:/PROGRAMS/PERL/LIB/5.10.0/Carp.pm
    line 14
    require Carp.pm called at c:\progr\pmmail3\bin\pmmail_folder_check.pl
    line 13
    main::BEGIN() called at D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm line 0
    eval {...} called at D:/PROGRAMS/PERL/LIB/5.10.0/feature.pm line 0
    BEGIN failed--compilation aborted.
    Compilation failed in require at D:/PROGRAMS/PERL/LIB/5.10.0/Carp.pm
    line 14.
    Compilation failed in require at
    c:\progr\pmmail3\bin\pmmail_folder_check.pl line 13.
    BEGIN failed--compilation aborted at
    c:\progr\pmmail3\bin\pmmail_folder_check.pl line 13.

    The only difference between this and the stripped down code I posted
    earlier is that this script starts with ten comment lines so 13 above
    is my test's 3. ie the use Carp; line.


    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 27, 2013
    #10
  11. Am 24.12.2013 11:39, schrieb Dave Saville:
    > On Mon, 23 Dec 2013 23:14:54 UTC, "Peter J. Holzer"
    > <> wrote:
    >
    >> On 2013-12-23 22:50, Rainer Weikusat <> wrote:
    >>> "Peter J. Holzer" <> writes:
    >>>> On 2013-12-23 15:39, Dave Saville <> wrote:
    >>>>> He now says
    >>>>>
    >>>>> <qoute>
    >>>>> the perl problem seems to be solved: it's not a problem with
    >>>>> installation or version. The problem is the setting for "LANG": It has
    >>>>> to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    >>>>> for german!
    >>>>> </quote>
    >>>>>
    >>>>> Surely (base) perl modules aren't dependent on language?
    >>>>
    >>>> They shouldn't be. It might be a bug in perl, in the module or in a
    >>>> system library. Or it might be a bug in your script. It is impossible to
    >>>> tell with the information you have given.
    >>>
    >>> Considering the statement from 'the user', this must be something which
    >>> causes the behaviour of some perl code to change because it is using a
    >>> German locale. Considering what is in a locale definition, this makes
    >>> LC_NUMERIC the most likely candidate, especially considering that this
    >>> is the only locale-information used as part of version checking (by
    >>> Perl_upg_version in util.c).

    >>
    >> Yes, I know. But I couldn't reproduce it. And that's not very
    >> surprising: Many people use perl with a German locale, so it can't be
    >> something generic like "perl doesn't work with a German locale" or "use
    >> feature doesn't work with a German locale". It has to be something more
    >> specific, like a bug in the locale implementation of the user's system
    >> (but Dave hasn't told us what system it is) or the way a certain feature
    >> is used (but Dave hasn't told us anything about his script). But it is
    >> impossible to find out what it is unless Dave volunteers more
    >> information.

    >
    > A little difficult to post extra info from the user as I only have a
    > jpeg of the errors rather than something I can cut and paste :-(
    >
    > The script starts:
    >
    > use strict;
    > use warnings;
    > use Carp:
    > use Encode;
    >
    > Compilation failed in require at .... line 3 - ie the "use Carp" line.
    >
    > System is OS/2 his perl is 5.10.0. (Yes I know :) )
    >
    > If you follow the links from the posted forum you end up with a C test
    > program. I have built that on my system but as I am in the UK my
    > system is defaulting to using a 'point' anyway. I have yet to persuade
    > it to start up in a different locale. I know how to get a program to
    > use a different language but that does not seem to help. I have posted
    > this problem to an OS/2 tech group in the hope they may throw some
    > light on the problem - assuming it is a similar one in that
    > setlocale() is not working correctly as per the forum post.
    >


    i have not read the whole tread,
    but how about using 'use Carp;' instead of 'use Carp:'?
    ^

    bye
    M
     
    Martin Gieretz, Dec 27, 2013
    #11
  12. Dave Saville

    Dave Saville Guest

    On Fri, 27 Dec 2013 14:59:11 UTC, Martin Gieretz
    <> wrote:

    <snip>
    >
    > i have not read the whole tread,
    > but how about using 'use Carp;' instead of 'use Carp:'?
    > ^


    Which would throw a syntax error not what we are seeing - My bad I
    just typed the three "use" lines into the mail, not copy/paste.

    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 27, 2013
    #12
  13. Dave Saville

    Dave Saville Guest

    On Fri, 27 Dec 2013 13:58:43 UTC, Rainer Weikusat
    <> wrote:

    > "Dave Saville" <> writes:
    > > On Mon, 23 Dec 2013 23:14:54 UTC, "Peter J. Holzer"
    > > <> wrote:
    > >> On 2013-12-23 22:50, Rainer Weikusat <> wrote:
    > >> > "Peter J. Holzer" <> writes:
    > >> >> On 2013-12-23 15:39, Dave Saville <> wrote:
    > >> >>> He now says
    > >> >>>
    > >> >>><qoute>
    > >> >>> the perl problem seems to be solved: it's not a problem with
    > >> >>> installation or version. The problem is the setting for "LANG": It has
    > >> >>> to be LANG=en_US.UTF-8 and not LANG=de_DE_EURO which is the standard
    > >> >>> for german!
    > >> >>></quote>
    > >> >>>

    >
    > [...]
    >
    > >> > Considering the statement from 'the user', this must be something which
    > >> > causes the behaviour of some perl code to change because it is using a
    > >> > German locale. Considering what is in a locale definition, this makes
    > >> > LC_NUMERIC the most likely candidate,

    >
    > [...]
    >
    > >> Yes, I know. But I couldn't reproduce it. And that's not very
    > >> surprising: Many people use perl with a German locale, so it can't be
    > >> something generic like "perl doesn't work with a German locale" or "use
    > >> feature doesn't work with a German locale". It has to be something more
    > >> specific, like a bug in the locale implementation of the user's system

    >
    > [...]
    >
    > > A little difficult to post extra info from the user as I only have a
    > > jpeg of the errors rather than something I can cut and paste :-(
    > >
    > > The script starts:
    > >
    > > use strict;
    > > use warnings;
    > > use Carp:
    > > use Encode;
    > >
    > > Compilation failed in require at .... line 3 - ie the "use Carp" line.
    > >
    > > System is OS/2 his perl is 5.10.0. (Yes I know :) )
    > >
    > > If you follow the links from the posted forum you end up with a C test
    > > program. I have built that on my system but as I am in the UK my
    > > system is defaulting to using a 'point' anyway. I have yet to persuade
    > > it to start up in a different locale.

    >
    > According to some more searching around on the web, the OS/2 command to
    > set a locale is (example)
    >
    > set lang=de_DE
    > [http://www.borgendale.com/locale.htm]
    >
    > Results as to whether or not setlocale works on OS/2 are somewhat
    > unclear. Assuming your system starts with 'English' language settings,
    > you could use a modified test program,
    >
    > -----------
    > #include <locale.h>
    > #include <stdio.h>
    >
    > int main(void) {
    > printf("%f\n",2.5);
    > setlocale(LC_NUMERIC, "de_DE");
    > printf("%f\n",2.5);
    > setlocale(LC_NUMERIC, "C");
    > printf("%f\n",2.5);
    >
    > return 0;
    > }
    > -----------
    >
    > this should print (possibly with variations in the number of trailing
    > zeroes)
    >
    > 2.500000
    > 2,500000
    > 2.500000
    >
    > If it doesn't, in particular, if there's a dot instead of a comma in the
    > second line, setlocale doesn't work as it is supposed to and if you
    > can't switch from English to German in this way, chances are very high
    > that switching from German to English also won't work.


    Interestingly:

    use strict;
    use warnings;
    use Carp;

    printf("%f\n", 2.5);

    Using perl 5.16.0

    [T:\tmp]try.pl
    Invalid version format (non-numeric data) at
    u:/perl5/lib/5.16.0/Carp.pm line 3.

    BEGIN failed--compilation aborted at u:/perl5/lib/5.16.0/Carp.pm line
    3.
    Compilation failed in require at try.pl line 3.
    BEGIN failed--compilation aborted at try.pl line 3.

    Carp line 3 says { use 5.006; }

    Using perl 5.8.2

    [T:\tmp]try.pl
    perl: warning: Setting locale failed.
    perl: warning: Please check that your locale settings:
    LC_ALL = (unset),
    LANG = "de_DE_EURO"
    are supported and installed on your system.
    perl: warning: Falling back to the standard locale ("C").
    2.500000

    Which is a darn site more useful IMHO.

    As the OS/2 setlocale() seems to suffer the same problem as the other
    OS above is there a perlish way around this one?

    The problem I have is that I am comparing strings from two sources.
    One where the string is in the local code page and the other in utf8.
    I solved this by using Encode and friends but that introduces the
    requirement for Carp and the above error :-(

    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 29, 2013
    #13
  14. On 2013-12-29 13:00, Dave Saville <> wrote:
    > Interestingly:
    >
    > use strict;
    > use warnings;
    > use Carp;
    >
    > printf("%f\n", 2.5);
    >
    > Using perl 5.16.0
    >
    > [T:\tmp]try.pl
    > Invalid version format (non-numeric data) at
    > u:/perl5/lib/5.16.0/Carp.pm line 3.
    >
    > BEGIN failed--compilation aborted at u:/perl5/lib/5.16.0/Carp.pm line
    > 3.
    > Compilation failed in require at try.pl line 3.
    > BEGIN failed--compilation aborted at try.pl line 3.
    >
    > Carp line 3 says { use 5.006; }


    Yup. I think somebody (Rainer?) had already pointed out this line as the
    likely culprit.

    > Using perl 5.8.2
    >
    > [T:\tmp]try.pl
    > perl: warning: Setting locale failed.
    > perl: warning: Please check that your locale settings:
    > LC_ALL = (unset),
    > LANG = "de_DE_EURO"
    > are supported and installed on your system.
    > perl: warning: Falling back to the standard locale ("C").
    > 2.500000
    >
    > Which is a darn site more useful IMHO.
    >
    > As the OS/2 setlocale() seems to suffer the same problem as the other
    > OS above is there a perlish way around this one?


    You could try

    BEGIN { $ENV{LANG} = 'C' }


    > The problem I have is that I am comparing strings from two sources.
    > One where the string is in the local code page and the other in utf8.
    > I solved this by using Encode


    Yes, Encode contains the necessary functions (But depending on the
    source it may be more convenient to use an I/O filter or soemthing
    similar instead of calling Encode::decode() explicitely).


    > and friends but that introduces the
    > requirement for Carp and the above error :-(


    I'm not sure why using Encode requires the use of Carp, but in any case

    { use 5.006; }

    is valid Perl and must compile successfully, regardless of any locale
    settings.

    Is OS/2 still officially supported by Perl? It isn't on
    http://www.cpan.org/ports/ anymore.

    A viable workaround might be to compile perl on OS/2 without locale
    support (since it doesn't seem to work correctly anyway). But that would
    also mean that your users need to use this locale-less perl binary,
    which may break some other scripts they use.

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
     
    Peter J. Holzer, Dec 29, 2013
    #14
  15. Dave Saville

    Dave Saville Guest

    On Mon, 30 Dec 2013 00:55:03 UTC, Ben Morrow <> wrote:

    Hi Ben

    > Since your broken locales are causing feature.pm to fail, you get weird
    > failures if it gets loaded, but not otherwise. Carp loads feature.pm
    > (unintentionally) on 5.16 or later, but not on earlier versions.
    >


    A clever man (ie not me) has found the problem.

    perl is built with locale.h
    libc is built with unidef.h

    The definitions of LC_NUMERIC and LC_MONETARY are reversed between
    the two. Correcting locale.h to match unidef.h results in the posted C
    testcase to work as expected. Our perl porter is rebuilding a version
    with this change.

    > It might be worth reporting a bug about this: broken locales are Not
    > Good, but they shouldn't cause perl to fail unless you explicitly try to
    > invoke locale functions.
    >


    Would you mind doing this please? I am not sure I can write a ticket
    that others could act on.

    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 30, 2013
    #15
  16. Dave Saville

    Dave Saville Guest

    Re: Question about language setting - 2nd question

    Now the first problem is solved, see elseehere in this tree, on to the
    next :)

    I have two input files the creation of which I have no control over.
    One has been written, from C, using whatever codepage the user has.
    The other file is encoded utf8.

    Both files contain a file name. One of the consistancy checks is to
    see if they match. I can read in the second file with open my $FILE2,
    '<:encoding(utf8)", .......

    So how to open the first?

    open my $FILE1, '<', .....
    open my $FILE1, <:raw', ....
    open my $FILE1, '<', .........
    binmode $FILE;

    All seem to do exactly the same thing - except for \r\n processing.

    or

    open my $FILE1, '<:encoding(whatever)', ......

    When perl converts to internal utf8 how does it decide to map code
    points > 127 when it is not told the cp? It does not default to
    current as I run with 850 and opening '<:encoding(cp850)' gives a
    different result for points > 127 than does open '<'. It's complicated
    by the fact that the field separator on the first file is hex DE that
    will get mangled in the former case. But my $sep = decode("cp$cp",
    "\xDE"); where $cp is the codepage seems to get around that one.

    What I need to be able to do is

    if ( $file1name eq $file2name ) where the names have accented
    characters.

    TIA
    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 30, 2013
    #16
  17. Dave Saville

    Dave Saville Guest

    Re: Question about language setting - 2nd question

    On Mon, 30 Dec 2013 14:36:22 UTC, Ben Morrow <> wrote:

    >
    > Quoth "Dave Saville" <>:
    > > Now the first problem is solved, see elseehere in this tree, on to the
    > > next :)
    > >
    > > I have two input files the creation of which I have no control over.
    > > One has been written, from C, using whatever codepage the user has.
    > > The other file is encoded utf8.
    > >
    > > Both files contain a file name. One of the consistancy checks is to
    > > see if they match. I can read in the second file with open my $FILE2,
    > > '<:encoding(utf8)", .......
    > >
    > > So how to open the first?
    > >
    > > open my $FILE1, '<', .....
    > > open my $FILE1, <:raw', ....
    > > open my $FILE1, '<', .........
    > > binmode $FILE;
    > >
    > > All seem to do exactly the same thing - except for \r\n processing.

    >
    > Yes.
    >
    > > or
    > >
    > > open my $FILE1, '<:encoding(whatever)', ......
    > >
    > > When perl converts to internal utf8 how does it decide to map code
    > > points > 127 when it is not told the cp?

    >
    > It reads bytes and maps them directly to Unicode codepoints; which,
    > because of the way Unicode is defined, means it's effectively using
    > ISO8859-1.
    >
    > > It does not default to
    > > current as I run with 850 and opening '<:encoding(cp850)' gives a
    > > different result for points > 127 than does open '<'.

    >
    > No. Perl doesn't know anything about OS/2 codepages. (It could, perhaps,
    > be taught; that is, it would be possible to write a :codepage layer
    > which looked up the current codepage and used that; but it would be
    > highly unportable to do something like that unless it was explicitly
    > requested.)
    >
    > > It's complicated
    > > by the fact that the field separator on the first file is hex DE that
    > > will get mangled in the former case. But my $sep = decode("cp$cp",
    > > "\xDE"); where $cp is the codepage seems to get around that one.

    >
    > When dealing with binary separators it's better to handle the data as
    > binary first, then decode to Unicode afterwards. That is, open the file
    > in :raw mode, split into fields on /\xDE/, and only then decode each
    > field from CP850 with decode().
    >
    > What you were doing was effectively treating the file as 'fields
    > separated by I-grave' rather than 'fields separated by binary 0xDE'.
    > Those things happen to come out looking the same in a CP850 file,
    > because CP850 is a 1-byte encoding and 0xDE is a byte with defined
    > semantics, but in general you can't assume binary separators can be
    > meaningfully decoded from any given encoding.


    Many thanks Ben. As I said I didn't "design" the 0xDE bit :) This
    code is checking files of a program that only runs on OS/2 so
    portability is not a question.

    --
    Regards
    Dave Saville
     
    Dave Saville, Dec 30, 2013
    #17
  18. Re: Question about language setting - 2nd question

    On 2013-12-30 14:36, Ben Morrow <> wrote:
    > Quoth "Dave Saville" <>:
    >> It does not default to current as I run with 850 and opening
    >> '<:encoding(cp850)' gives a different result for points > 127 than
    >> does open '<'.

    >
    > No. Perl doesn't know anything about OS/2 codepages. (It could, perhaps,
    > be taught; that is, it would be possible to write a :codepage layer
    > which looked up the current codepage and used that; but it would be
    > highly unportable to do something like that unless it was explicitly
    > requested.)


    There is the Encode::Locale module, which allows you to do something
    like
    open(my $fh, '<:encoding(locale)', $filename);

    I don't think it works on OS/2 (A colleague recently told me it doesn't
    even work on Windows), but I think it would be better to extend an
    existing module to work on more platforms than to add another
    platform-specific module.

    hp

    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
     
    Peter J. Holzer, Dec 30, 2013
    #18
  19. Re: Question about language setting - 2nd question

    * Peter J. Holzer wrote in comp.lang.perl.misc:
    >There is the Encode::Locale module, which allows you to do something
    >like
    > open(my $fh, '<:encoding(locale)', $filename);
    >
    >I don't think it works on OS/2 (A colleague recently told me it doesn't
    >even work on Windows), but I think it would be better to extend an
    >existing module to work on more platforms than to add another
    >platform-specific module.


    http://www.nntp.perl.org/group/perl.i18n/2008/07/msg214.html has some
    thoughts about Windows support in that area.
    --
    Björn Höhrmann · mailto: · http://bjoern.hoehrmann.de
    Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
    25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/
     
    Bjoern Hoehrmann, Dec 30, 2013
    #19
  20. Rainer Weikusat <> writes:
    [...]
    > Results as to whether or not setlocale works on OS/2 are somewhat
    > unclear. Assuming your system starts with 'English' language settings,
    > you could use a modified test program,
    >
    > -----------
    > #include <locale.h>
    > #include <stdio.h>
    >
    > int main(void) {
    > printf("%f\n",2.5);
    > setlocale(LC_NUMERIC, "de_DE");
    > printf("%f\n",2.5);
    > setlocale(LC_NUMERIC, "C");
    > printf("%f\n",2.5);
    >
    > return 0;
    > }
    > -----------
    >
    > this should print (possibly with variations in the number of trailing
    > zeroes)
    >
    > 2.500000
    > 2,500000
    > 2.500000
    >
    > If it doesn't, in particular, if there's a dot instead of a comma in the
    > second line, setlocale doesn't work as it is supposed to and if you
    > can't switch from English to German in this way, chances are very high
    > that switching from German to English also won't work.


    setlocale() returns a char* that's either a pointer to a string (if it
    succeeded), or a null pointer (if it failed). It's probably worth
    adding code to check for this rather than just depending on the output
    of printf to change. (On my system, the first setlocale() call fails
    because I don't have any German locales installed.)

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Working, but not speaking, for JetHead Development, Inc.
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Dec 31, 2013
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ed
    Replies:
    24
    Views:
    1,028
    Dimitri Maziuk
    Mar 27, 2006
  2. DaveInSidney
    Replies:
    0
    Views:
    437
    DaveInSidney
    May 9, 2005
  3. pabbu
    Replies:
    8
    Views:
    745
    Marc Boyer
    Nov 7, 2005
  4. Shravani
    Replies:
    8
    Views:
    816
    Bartc
    Mar 16, 2008
  5. Replies:
    28
    Views:
    1,195
Loading...

Share This Page