Multibyte and Gems

Discussion in 'Ruby' started by Martin Hess, Jun 23, 2009.

  1. Martin Hess

    Martin Hess Guest

    I've tracked down a problem with a Gem I am trying to use. It turns =20
    out that it has some non-ascii characters in it; for example the =20
    second quote in the regular expression below is not an ASCII character:

    parts =3D self.split( %r/( [:.;?!][ ] | (?:[ ]|^)["=93] )/x )

    It produces errors like this:

    :in `require': =
    /opt/local/lib/ruby1.9/gems/1.9.1/gems/webby-0.9.4/lib/=20
    webby/core_ext/string.rb:14: invalid multibyte char (US-ASCII) =20
    (SyntaxError)

    I fixed it by adding the following to the top of the offending file:

    # encoding: utf-8

    My questions:

    * Is this the preferred fix?

    * Is there a way to work around this problem without modifying the Gem?

    * Is there an easy way to see if gems have non-ascii source files but =20=

    haven't included an encoding comment? Some kind of Ruby warning for =20
    instance.
     
    Martin Hess, Jun 23, 2009
    #1
    1. Advertising

  2. Martin Hess

    Eric Hodel Guest

    On Jun 22, 2009, at 22:26, Martin Hess wrote:
    > I've tracked down a problem with a Gem I am trying to use. It turns =20=


    > out that it has some non-ascii characters in it; for example the =20
    > second quote in the regular expression below is not an ASCII =20
    > character:
    >
    > parts =3D self.split( %r/( [:.;?!][ ] | (?:[ ]|^)["=93] )/x )
    >
    > It produces errors like this:
    >
    > :in `require': =

    /opt/local/lib/ruby1.9/gems/1.9.1/gems/webby-0.9.4/=20
    > lib/webby/core_ext/string.rb:14: invalid multibyte char (US-ASCII) =20
    > (SyntaxError)
    >
    > I fixed it by adding the following to the top of the offending file:
    >
    > # encoding: utf-8
    >
    > My questions:
    >
    > * Is this the preferred fix?


    Yes.

    > * Is there a way to work around this problem without modifying the =20
    > Gem?


    File a bug with the author and have them release a new version, =20
    otherwise no.

    > * Is there an easy way to see if gems have non-ascii source files =20
    > but haven't included an encoding comment? Some kind of Ruby warning =20=


    > for instance.


    ruby -c will do this for you.
     
    Eric Hodel, Jun 23, 2009
    #2
    1. Advertising

  3. Martin Hess

    Martin Hess Guest

    So is it considered best practice to put an encoding comment at the =20
    begging of all your files now days? Such as:

    # encoding: utf-8

    or whatever encoding you like. is this what people are doing or are =20
    they doing it one off for the files that have non-ascii characters?

    It seems to me that if you have a modern editor it isn't too hard to =20
    accidentally slip in some non-ascii characters resulting in some pain =20=

    down the road.


    On Jun 23, 2009, at 12:25 AM, Eric Hodel wrote:

    > On Jun 22, 2009, at 22:26, Martin Hess wrote:
    >> I've tracked down a problem with a Gem I am trying to use. It turns =20=


    >> out that it has some non-ascii characters in it; for example the =20
    >> second quote in the regular expression below is not an ASCII =20
    >> character:
    >>
    >> parts =3D self.split( %r/( [:.;?!][ ] | (?:[ ]|^)["=93] )/x )
    >>
    >> It produces errors like this:
    >>
    >> :in `require': =

    /opt/local/lib/ruby1.9/gems/1.9.1/gems/webby-0.9.4/=20
    >> lib/webby/core_ext/string.rb:14: invalid multibyte char (US-ASCII) =20=


    >> (SyntaxError)
    >>
    >> I fixed it by adding the following to the top of the offending file:
    >>
    >> # encoding: utf-8
    >>
    >> My questions:
    >>
    >> * Is this the preferred fix?

    >
    > Yes.
    >
    >> * Is there a way to work around this problem without modifying the =20=


    >> Gem?

    >
    > File a bug with the author and have them release a new version, =20
    > otherwise no.
    >
    >> * Is there an easy way to see if gems have non-ascii source files =20
    >> but haven't included an encoding comment? Some kind of Ruby warning =20=


    >> for instance.

    >
    > ruby -c will do this for you.
    >
    >
     
    Martin Hess, Jun 23, 2009
    #3
  4. On Wed, Jun 24, 2009 at 1:43 AM, Martin Hess<> wrote:
    > So is it considered best practice to put an encoding comment at the beggi=

    ng
    > of all your files now days? Such as:
    >
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 # encoding: utf-8
    >
    > or whatever encoding you like. is this what people are doing or are they
    > doing it one off for the files that have non-ascii characters?
    >
    > It seems to me that if you have a modern editor it isn't too hard to
    > accidentally slip in some non-ascii characters resulting in some pain dow=

    n
    > the road.


    It isn't hard to mess up any code in a lot of ways, so as usual, try
    to run/test it before you release/deploy :)
    That also means that using Ruby 1.9.1 for your daily coding might be a
    better choice, otherwise you'll have to use multiruby.

    >
    > On Jun 23, 2009, at 12:25 AM, Eric Hodel wrote:
    >
    >> On Jun 22, 2009, at 22:26, Martin Hess wrote:
    >>>
    >>> I've tracked down a problem with a Gem I am trying to use. It turns out
    >>> that it has some non-ascii characters in it; for example the second quo=

    te in
    >>> the regular expression below is not an ASCII character:
    >>>
    >>> =C2=A0parts =3D self.split( %r/( [:.;?!][ ] | (?:[ ]|^)["=E2=80=9C] )/x=

    )
    >>>
    >>> It produces errors like this:
    >>>
    >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0:in `require':
    >>> /opt/local/lib/ruby1.9/gems/1.9.1/gems/webby-0.9.4/lib/webby/core_ext/s=

    tring.rb:14:
    >>> invalid multibyte char (US-ASCII) (SyntaxError)
    >>>
    >>> I fixed it by adding the following to the top of the offending file:
    >>>
    >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0# encoding: utf-8
    >>>
    >>> My questions:
    >>>
    >>> * Is this the preferred fix?

    >>
    >> Yes.
    >>
    >>> * Is there a way to work around this problem without modifying the Gem?

    >>
    >> File a bug with the author and have them release a new version, otherwis=

    e
    >> no.
    >>
    >>> * Is there an easy way to see if gems have non-ascii source files but
    >>> haven't included an encoding comment? Some kind of Ruby warning for
    >>> instance.

    >>
    >> ruby -c will do this for you.
    >>
    >>

    >
    >
    >




    --=20
    Michael Fellinger
    CTO, The Rubyists, LLC
    972-996-5199
     
    Michael Fellinger, Jun 25, 2009
    #4
  5. Martin Hess

    Eric Hodel Guest

    On Jun 24, 2009, at 19:08, Michael Fellinger wrote:
    > On Wed, Jun 24, 2009 at 1:43 AM, Martin Hess<> wrote:
    >> So is it considered best practice to put an encoding comment at the
    >> begging
    >> of all your files now days? Such as:
    >>
    >> # encoding: utf-8
    >>
    >> or whatever encoding you like. is this what people are doing or are
    >> they
    >> doing it one off for the files that have non-ascii characters?
    >>
    >> It seems to me that if you have a modern editor it isn't too hard to
    >> accidentally slip in some non-ascii characters resulting in some
    >> pain down
    >> the road.

    >
    > It isn't hard to mess up any code in a lot of ways, so as usual, try
    > to run/test it before you release/deploy :)
    > That also means that using Ruby 1.9.1 for your daily coding might be a
    > better choice, otherwise you'll have to use multiruby.


    With hoe, it's as easy as:

    multiruby_setup the_usual # only once
    rake multi
     
    Eric Hodel, Jun 25, 2009
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tomás

    Gems -- #include <gems.hpp>

    Tomás, Mar 4, 2006, in forum: C++
    Replies:
    7
    Views:
    447
    Tomás
    Mar 5, 2006
  2. Lloyd Zusman
    Replies:
    3
    Views:
    203
    Lloyd Zusman
    Jun 21, 2005
  3. Dany Cayouette

    'private' gems/gems hierarchy

    Dany Cayouette, Nov 25, 2005, in forum: Ruby
    Replies:
    3
    Views:
    174
    Jim Weirich
    Nov 25, 2005
  4. Thaddeus L Olczyk

    Problem getting gems/listing gems. EINVAL

    Thaddeus L Olczyk, Aug 15, 2006, in forum: Ruby
    Replies:
    0
    Views:
    203
    Thaddeus L Olczyk
    Aug 15, 2006
  5. Patrick Hurley

    Creating binary gems, from source gems

    Patrick Hurley, Mar 4, 2007, in forum: Ruby
    Replies:
    0
    Views:
    149
    Patrick Hurley
    Mar 4, 2007
Loading...

Share This Page