How to change stdin separator (perl vs ruby)

Discussion in 'Ruby' started by Sak .., Feb 10, 2010.

  1. Sak ..

    Sak .. Guest

    I have to process very very big text files and not line per line, I have
    to access to a group of lines seperated for a "%". Then I tryed to
    access line per line and detect the separator for each line saving the
    lines in a buffer, obviously it works but it's very slow (text files are
    bigger than 1GB).

    Perl have an awesome feature that is to change the separator for
    iterating files (by default in all languages is "\n"), this simple and
    extremely efficent code do what I need:

    $/ = "%";
    while ($article = <STDIN>) {
    #Iteration
    }


    I can write a equivalent code in ruby but not (by far) as efficient as
    perl. Someone knows how can I change the separator for iterate the stdin
    in ruby?

    Thanks!
    --
    Posted via http://www.ruby-forum.com/.
     
    Sak .., Feb 10, 2010
    #1
    1. Advertising

  2. Sak ..

    Stefan Lang Guest

    2010/2/10 Sak .. <>:
    > I have to process very very big text files and not line per line, I have
    > to access to a group of lines seperated for a "%". Then I tryed to
    > access line per line and detect the separator for each line saving the
    > lines in a buffer, obviously it works but it's very slow (text files are
    > bigger than 1GB).
    >
    > Perl have an awesome feature that is to change the separator for
    > iterating files (by default in all languages is "\n"), this simple and
    > extremely efficent code do what I need:
    >
    > $/ =3D "%";
    > while ($article =3D <STDIN>) {
    > =A0#Iteration
    > }
    >
    >
    > I can write a equivalent code in ruby but not (by far) as efficient as
    > perl. Someone knows how can I change the separator for iterate the stdin
    > in ruby?


    STDIN.each_line("%") { |line|
    # iteration
    }

    Stefan
     
    Stefan Lang, Feb 10, 2010
    #2
    1. Advertising

  3. Sak ..

    Xavier Noria Guest

    On Wed, Feb 10, 2010 at 4:23 PM, Sak .. <> wrote:

    > I can write a equivalent code in ruby but not (by far) as efficient as
    > perl. Someone knows how can I change the separator for iterate the stdin
    > in ruby?


    Ruby has also $/ among its global variables. The methods #gets and
    #readline take it into account. Those methods and #each also accept a
    separator as an optional argument.

    In Ruby 1.9 they additionally accept an integer, that's the same as
    setting $/ to a reference to an integer in Perl.
     
    Xavier Noria, Feb 10, 2010
    #3
  4. Sak ..

    Daniel Bye Guest

    On Thu, Feb 11, 2010 at 12:23:55AM +0900, Sak .. wrote:
    > I have to process very very big text files and not line per line, I have
    > to access to a group of lines seperated for a "%". Then I tryed to
    > access line per line and detect the separator for each line saving the
    > lines in a buffer, obviously it works but it's very slow (text files are
    > bigger than 1GB).
    >
    > Perl have an awesome feature that is to change the separator for
    > iterating files (by default in all languages is "\n"), this simple and
    > extremely efficent code do what I need:
    >
    > $/ = "%";
    > while ($article = <STDIN>) {
    > #Iteration
    > }
    >


    If your delimiter only occurs as a delimiter, and not in the data as well:

    File.open(path_to_file, 'r') do |f|
    f.each('%') do |record|
    # Do stuff.
    end
    end

    >
    > I can write a equivalent code in ruby but not (by far) as efficient as
    > perl. Someone knows how can I change the separator for iterate the stdin
    > in ruby?
    >
    > Thanks!
    > --
    > Posted via http://www.ruby-forum.com/.
    >
    >
    > !DSPAM:4b72d13c186061354869328!
    >
    >


    --
    Daniel Bye
    _
    ASCII ribbon campaign ( )
    - against HTML, vCards and X
    - proprietary attachments in e-mail / \
     
    Daniel Bye, Feb 10, 2010
    #4
  5. 2010/2/10 Daniel Bye <>:
    > On Thu, Feb 11, 2010 at 12:23:55AM +0900, Sak .. wrote:
    >> I have to process very very big text files and not line per line, I have
    >> to access to a group of lines seperated for a "%". Then I tryed to
    >> access line per line and detect the separator for each line saving the
    >> lines in a buffer, obviously it works but it's very slow (text files are
    >> bigger than 1GB).
    >>
    >> Perl have an awesome feature that is to change the separator for
    >> iterating files (by default in all languages is "\n"), this simple and
    >> extremely efficent code do what I need:
    >>
    >> $/ =3D "%";
    >> while ($article =3D <STDIN>) {
    >> =A0 #Iteration
    >> }
    >>

    >
    > If your delimiter only occurs as a delimiter, and not in the data as well=

    :

    Note that this statement is true for \n as well!

    > File.open(path_to_file, 'r') do |f|
    > =A0f.each('%') do |record|
    > =A0 =A0# Do stuff.
    > =A0end
    > end


    You can have it simpler:

    File.foreach path, '%' do |record|
    # so something with record
    end

    Kind regards

    robert

    --=20
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Feb 10, 2010
    #5
  6. Sak ..

    Sak .. Guest

    Stefan Lang wrote:
    > 2010/2/10 Sak .. <>:
    >> $/ = "%";
    >> while ($article = <STDIN>) {
    >> �#Iteration
    >> }
    >>
    >>
    >> I can write a equivalent code in ruby but not (by far) as efficient as
    >> perl. Someone knows how can I change the separator for iterate the stdin
    >> in ruby?

    >
    > STDIN.each_line("%") { |line|
    > # iteration
    > }
    >
    > Stefan


    Thanks Stefan for the answer, that's exactly what I need, thanks Xavier
    explanation is perfect. Thanks Daniel, good observation.


    --
    Posted via http://www.ruby-forum.com/.
     
    Sak .., Feb 10, 2010
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Johnathan Doe

    peek at stdin, flush stdin

    Johnathan Doe, May 15, 2004, in forum: C Programming
    Replies:
    5
    Views:
    25,443
    Chatoyer
    May 17, 2013
  2. Charlie Zender

    Reading stdin once confuses second stdin read

    Charlie Zender, Jun 19, 2004, in forum: C Programming
    Replies:
    6
    Views:
    822
    Dan Pop
    Jun 21, 2004
  3. Ben
    Replies:
    2
    Views:
    1,398
    jacob navia
    Aug 29, 2009
  4. Terry Cooper
    Replies:
    7
    Views:
    449
    Janos Sebok
    Jun 9, 2009
  5. Stefano Sabatini
    Replies:
    6
    Views:
    311
    Stefano Sabatini
    Jul 29, 2007
Loading...

Share This Page