Info regarding Zlib::GzipReader

Discussion in 'Ruby' started by J-H Johansen, Jun 15, 2007.

  1. J-H Johansen

    J-H Johansen Guest

    Hi,

    I'm trying to parse through a gzip'ed proxy access log with
    Zlib::GzipReader and I'm having some difficulties.

    f = File.open(file, "r")
    gz = Zlib::GzipReader.new(f)
    gz.readlines.each do |block|
    puts block
    end

    What this piece of code will do is to read the first 6 lines of the
    proxy log before it reaches (what it believes to be) the end of the
    file. These few lines happens to be the info header which contains:

    #Software: ......
    #Version: ......
    #Start-date: ......
    #Date: ......
    #Fields: ....................
    #Remark: ........

    The access log contains a wee bit more than that though (980796 lines).
    By just using File.open(file) it seems I can read the whole file.

    I'm speculating here but I think that maybe the gzip file may have
    been buffered. I.e. first 6 lines has been gzip'ed and then the rest
    of the file has been gzip'ed and appended to it afterwards.

    One way of fixing the problem is to gunzip the file and then gzip the
    output into a new file. Problem solved (sort of).

    Do any of you know of any other way to do this without actually
    modifying the access logs ?

    I'm thinking of something along the lines of breaking up the file into
    smaller file handles which in turn can be used by GzipReader, but I
    don't know how this is done.

    Anyone know how this can be done or if there is any better ways of doing it ?


    Thanks

    --
    Jens-H Johansen
    --
    There are 10 kinds of people in the world: Those who understand binary and
    those who don't...
     
    J-H Johansen, Jun 15, 2007
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Matthew Brett
    Replies:
    4
    Views:
    1,169
    Matthew Brett
    May 9, 2010
  2. David G. Andersen

    Speed gap between zcat and zlib's GzipReader

    David G. Andersen, Oct 19, 2004, in forum: Ruby
    Replies:
    3
    Views:
    501
    Yukihiro Matsumoto
    Oct 26, 2004
  3. Replies:
    2
    Views:
    519
  4. Jos Backus
    Replies:
    10
    Views:
    520
    Jeremy Bopp
    Feb 4, 2011
  5. Thomas Wolf
    Replies:
    5
    Views:
    1,231
    Simon Krahnke
    Apr 26, 2012
Loading...

Share This Page