unix zcat with ruby?

Discussion in 'Ruby' started by music, Apr 23, 2007.

  1. music

    music Guest

    I have to read in many files.
    I prefer to concat those files and reading only one large file.
    There is a way like unix zcat?
    music, Apr 23, 2007
    #1
    1. Advertising

  2. music

    Robert Dober Guest

    On 4/23/07, music <> wrote:
    > I have to read in many files.
    > I prefer to concat those files and reading only one large file.
    > There is a way like unix zcat?
    >
    >

    Sure is
    ARGF
    HTH
    R.

    --
    You see things; and you say Why?
    But I dream things that never were; and I say Why not?
    -- George Bernard Shaw
    Robert Dober, Apr 23, 2007
    #2
    1. Advertising

  3. music

    music Guest

    Robert Dober wrote:
    > On 4/23/07, music <> wrote:
    >> I have to read in many files.
    >> I prefer to concat those files and reading only one large file.
    >> There is a way like unix zcat?
    >>
    >>

    > Sure is
    > ARGF


    ARGF ok, but how can I pass ARGF to a method?
    music, Apr 24, 2007
    #3
  4. music

    Robert Dober Guest

    On 4/24/07, music <> wrote:
    > Robert Dober wrote:
    > > On 4/23/07, music <> wrote:
    > >> I have to read in many files.
    > >> I prefer to concat those files and reading only one large file.
    > >> There is a way like unix zcat?
    > >>
    > >>

    > > Sure is
    > > ARGF

    >
    > ARGF ok, but how can I pass ARGF to a method?
    >
    >

    As a global constant it is visible anywhere, you can thus simply do
    things like this

    def my_method
    ARGF.each_line do
    | line |
    whatever
    end
    end

    R.

    --
    You see things; and you say Why?
    But I dream things that never were; and I say Why not?
    -- George Bernard Shaw
    Robert Dober, Apr 24, 2007
    #4
  5. On 24.04.2007 09:04, music wrote:
    > Robert Dober wrote:
    >> On 4/23/07, music <> wrote:
    >>> I have to read in many files.
    >>> I prefer to concat those files and reading only one large file.
    >>> There is a way like unix zcat?
    >>>
    >>>

    >> Sure is
    >> ARGF

    >
    > ARGF ok, but how can I pass ARGF to a method?


    What???


    some_method(ARGF)

    robert
    Robert Klemme, Apr 24, 2007
    #5
  6. music

    music Guest

    Robert Dober wrote:
    > On 4/24/07, music <> wrote:
    >> Robert Dober wrote:
    >> > On 4/23/07, music <> wrote:
    >> >> I have to read in many files.
    >> >> I prefer to concat those files and reading only one large file.
    >> >> There is a way like unix zcat?
    >> >>
    >> >>
    >> > Sure is
    >> > ARGF

    >>
    >> ARGF ok, but how can I pass ARGF to a method?
    >>
    >>

    > As a global constant it is visible anywhere, you can thus simply do
    > things like this
    >
    > def my_method
    > ARGF.each_line do
    > | line |
    > whatever
    > end
    > end


    Great!!! and if input files are in gzip format? how can I read them?
    music, Apr 24, 2007
    #6
  7. music

    Robert Dober Guest

    On 4/24/07, music <> wrote:
    > Robert Dober wrote:
    > > On 4/24/07, music <> wrote:
    > >> Robert Dober wrote:
    > >> > On 4/23/07, music <> wrote:
    > >> >> I have to read in many files.
    > >> >> I prefer to concat those files and reading only one large file.
    > >> >> There is a way like unix zcat?
    > >> >>
    > >> >>
    > >> > Sure is
    > >> > ARGF
    > >>
    > >> ARGF ok, but how can I pass ARGF to a method?
    > >>
    > >>

    > > As a global constant it is visible anywhere, you can thus simply do
    > > things like this
    > >
    > > def my_method
    > > ARGF.each_line do
    > > | line |
    > > whatever
    > > end
    > > end

    >
    > Great!!! and if input files are in gzip format? how can I read them?
    >
    >

    http://www.ruby-doc.org/stdlib/libdoc/zlib/rdoc/index.html


    --
    You see things; and you say Why?
    But I dream things that never were; and I say Why not?
    -- George Bernard Shaw
    Robert Dober, Apr 24, 2007
    #7
  8. music

    music Guest

    Robert Dober wrote:
    > On 4/24/07, music <> wrote:
    >> Robert Dober wrote:
    >> > On 4/24/07, music <> wrote:
    >> >> Robert Dober wrote:
    >> >> > On 4/23/07, music <> wrote:
    >> >> >> I have to read in many files.
    >> >> >> I prefer to concat those files and reading only one large file.
    >> >> >> There is a way like unix zcat?
    >> >> >>
    >> >> >>
    >> >> > Sure is
    >> >> > ARGF
    >> >>
    >> >> ARGF ok, but how can I pass ARGF to a method?
    >> >>
    >> >>
    >> > As a global constant it is visible anywhere, you can thus simply do
    >> > things like this
    >> >
    >> > def my_method
    >> > ARGF.each_line do
    >> > | line |
    >> > whatever
    >> > end
    >> > end

    >>
    >> Great!!! and if input files are in gzip format? how can I read them?
    >>
    >>

    > http://www.ruby-doc.org/stdlib/libdoc/zlib/rdoc/index.html
    >


    Yes but can I use ARGF with zlib?
    music, Apr 24, 2007
    #8
  9. music

    music Guest

    music wrote:
    > Robert Dober wrote:
    >> On 4/24/07, music <> wrote:
    >>> Robert Dober wrote:
    >>> > On 4/24/07, music <> wrote:
    >>> >> Robert Dober wrote:
    >>> >> > On 4/23/07, music <> wrote:
    >>> >> >> I have to read in many files.
    >>> >> >> I prefer to concat those files and reading only one large file.
    >>> >> >> There is a way like unix zcat?
    >>> >> >>
    >>> >> >>
    >>> >> > Sure is
    >>> >> > ARGF
    >>> >>
    >>> >> ARGF ok, but how can I pass ARGF to a method?
    >>> >>
    >>> >>
    >>> > As a global constant it is visible anywhere, you can thus simply do
    >>> > things like this
    >>> >
    >>> > def my_method
    >>> > ARGF.each_line do
    >>> > | line |
    >>> > whatever
    >>> > end
    >>> > end
    >>>
    >>> Great!!! and if input files are in gzip format? how can I read them?
    >>>
    >>>

    >> http://www.ruby-doc.org/stdlib/libdoc/zlib/rdoc/index.html
    >>

    >
    > Yes but can I use ARGF with zlib?


    Any help? I'm searching in Zlib::GzipReader class but I can't undestand
    how I can use with ARGF.
    music, Apr 24, 2007
    #9
  10. music

    Robert Dober Guest

    On 4/24/07, music <> wrote:
    > music wrote:
    > > Robert Dober wrote:
    > >> On 4/24/07, music <> wrote:
    > >>> Robert Dober wrote:
    > >>> > On 4/24/07, music <> wrote:
    > >>> >> Robert Dober wrote:
    > >>> >> > On 4/23/07, music <> wrote:
    > >>> >> >> I have to read in many files.
    > >>> >> >> I prefer to concat those files and reading only one large file.
    > >>> >> >> There is a way like unix zcat?
    > >>> >> >>
    > >>> >> >>
    > >>> >> > Sure is
    > >>> >> > ARGF
    > >>> >>
    > >>> >> ARGF ok, but how can I pass ARGF to a method?
    > >>> >>
    > >>> >>
    > >>> > As a global constant it is visible anywhere, you can thus simply do
    > >>> > things like this
    > >>> >
    > >>> > def my_method
    > >>> > ARGF.each_line do
    > >>> > | line |
    > >>> > whatever
    > >>> > end
    > >>> > end
    > >>>
    > >>> Great!!! and if input files are in gzip format? how can I read them?
    > >>>
    > >>>
    > >> http://www.ruby-doc.org/stdlib/libdoc/zlib/rdoc/index.html
    > >>

    > >
    > > Yes but can I use ARGF with zlib?

    >
    > Any help? I'm searching in Zlib::GzipReader class but I can't undestand
    > how I can use with ARGF.
    >
    >

    Something like this
    ruby -rzlib -e 'reader = Zlib::GzipReader.new(ARGF);reader.each_line
    do |x| puts x end' xxx.gz
    I discovered however that
    Zlib::GzipReader.new(ARGF)
    seems to ignore all but the first file in the argument list - unless I
    did something stupid - that complicates matters slightly, you still
    can do things like

    ARGV.each do
    | filename |
    Zlib::GeipReader.open( filename) do
    |gz|
    gz.each_line ....
    ...
    end
    end

    of course but this behavior surprises me a little bit, maybe you
    should mine the Zlib doc for this.

    Robert

    --
    You see things; and you say Why?
    But I dream things that never were; and I say Why not?
    -- George Bernard Shaw
    Robert Dober, Apr 24, 2007
    #10
  11. music

    Robert Dober Guest

    >
    > of course but this behavior surprises me a little bit,

    forget what I said, I just switched my brain on
    the concatenation of two gzipped streams is of course not a gzipped stream,
    Zlib does well to inflate the first one, maybe a warning would be nice.
    R.
    Robert Dober, Apr 24, 2007
    #11
  12. music

    music Guest

    Robert Dober wrote:
    >>
    >> of course but this behavior surprises me a little bit,

    > forget what I said, I just switched my brain on
    > the concatenation of two gzipped streams is of course not a gzipped stream,
    > Zlib does well to inflate the first one, maybe a warning would be nice.
    > R.
    >


    Sorry for my newbie condition but I don't understand.
    music, Apr 24, 2007
    #12
  13. On Tue, Apr 24, 2007 at 07:52:41PM +0900, Robert Dober wrote:
    > >of course but this behavior surprises me a little bit,

    > forget what I said, I just switched my brain on
    > the concatenation of two gzipped streams is of course not a gzipped stream,


    But the standalone gzip program handles this:

    $ (echo "foo" | gzip -c; echo "bar" | gzip -c) | gzip -dc
    foo
    bar
    $

    It enables you to grow a gzip file without having to entirely decompress and
    recompress what's already there.
    Brian Candler, Apr 24, 2007
    #13
  14. music

    Robert Dober Guest

    On 4/24/07, Brian Candler <> wrote:
    > On Tue, Apr 24, 2007 at 07:52:41PM +0900, Robert Dober wrote:
    > > >of course but this behavior surprises me a little bit,

    > > forget what I said, I just switched my brain on
    > > the concatenation of two gzipped streams is of course not a gzipped stream,

    >
    > But the standalone gzip program handles this:

    zcat does too, which is nice of course.
    But this is an extra feature and I did not want to criticize zlib for
    not implementing this extra feature, maybe someone will provide a
    patch to ruby-core ;)
    >
    > $ (echo "foo" | gzip -c; echo "bar" | gzip -c) | gzip -dc
    > foo
    > bar
    > $


    >
    > It enables you to grow a gzip file without having to entirely decompress and
    > recompress what's already there.

    Yeah I just tested this, as I use gzip only with tar.
    Hmm does that mean that I was wrong that the concatenation of two
    gzipped streams is not a gzipped stream?
    Nahh, that surely is just an underlying feature of gzip (and bzip2 for
    that matter).
    Gotta look that after...
    >
    >

    Cheers
    Robert

    --
    You see things; and you say Why?
    But I dream things that never were; and I say Why not?
    -- George Bernard Shaw
    Robert Dober, Apr 24, 2007
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Me

    zcat output

    Me, Dec 28, 2004, in forum: Perl
    Replies:
    1
    Views:
    2,148
    Jim Gibson
    Dec 28, 2004
  2. Replies:
    12
    Views:
    1,615
    Dave Thompson
    Jan 10, 2005
  3. Replies:
    18
    Views:
    608
    Dave Thompson
    Jan 10, 2005
  4. David G. Andersen

    Speed gap between zcat and zlib's GzipReader

    David G. Andersen, Oct 19, 2004, in forum: Ruby
    Replies:
    3
    Views:
    443
    Yukihiro Matsumoto
    Oct 26, 2004
  5. Robert Wallace

    my own perl "dos->unix"/"unix->dos"

    Robert Wallace, Jan 21, 2004, in forum: Perl Misc
    Replies:
    7
    Views:
    266
    Michele Dondi
    Jan 22, 2004
Loading...

Share This Page