How to uncompress HTTP GZIP data

Discussion in 'Perl Misc' started by Mark Smith, Dec 19, 2005.

  1. Mark Smith

    Mark Smith Guest

    Hi,

    Following is the response header from IIS:

    HTTP/1.1 200 OK
    Cache-Control: no-cache
    Connection: close
    Date: Mon, 19 Dec 2005 21:00:17 GMT
    Pragma: no-cache
    Content-Type: text/html
    Expires: Wed, 01 Jan 1997 12:00:00 GMT
    Server: Microsoft-IIS/6.0
    P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
    Content-Encoding: gzip
    Vary: Accept-Encoding
    Transfer-Encoding: chunked

    The http data was captured from all of the TCP packets in sequence, and
    saved to a .Z file. The file will not uncompress using GUNZIP utility.
    I opened the .z file in a hex editor, and was able to see that the
    header does not begin with the magic header for gzip files. I fixed
    that, but it still doesn't work.

    HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?

    thanks much for all your help!
     
    Mark Smith, Dec 19, 2005
    #1
    1. Advertising

  2. Mark Smith wrote:
    > Following is the response header from IIS:
    >
    > HTTP/1.1 200 OK
    > Cache-Control: no-cache
    > Connection: close
    > Date: Mon, 19 Dec 2005 21:00:17 GMT
    > Pragma: no-cache
    > Content-Type: text/html
    > Expires: Wed, 01 Jan 1997 12:00:00 GMT
    > Server: Microsoft-IIS/6.0
    > P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
    > Content-Encoding: gzip
    > Vary: Accept-Encoding
    > Transfer-Encoding: chunked
    >
    > The http data was captured from all of the TCP packets in sequence, and
    > saved to a .Z file. The file will not uncompress using GUNZIP utility.
    > I opened the .z file in a hex editor, and was able to see that the
    > header does not begin with the magic header for gzip files. I fixed
    > that, but it still doesn't work.
    >
    > HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?


    You may find this thread useful:
    http://groups.google.com/group/comp.lang.perl.modules/browse_frm/thread/fbcfa9d737888425

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
     
    Gunnar Hjalmarsson, Dec 19, 2005
    #2
    1. Advertising

  3. "Mark Smith" <> wrote in message
    news:...
    > Hi,
    >
    > Following is the response header from IIS:
    >
    > HTTP/1.1 200 OK
    > Cache-Control: no-cache
    > Connection: close
    > Date: Mon, 19 Dec 2005 21:00:17 GMT
    > Pragma: no-cache
    > Content-Type: text/html
    > Expires: Wed, 01 Jan 1997 12:00:00 GMT
    > Server: Microsoft-IIS/6.0
    > P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
    > Content-Encoding: gzip
    > Vary: Accept-Encoding
    > Transfer-Encoding: chunked
    >
    > The http data was captured from all of the TCP packets in sequence, and
    > saved to a .Z file. The file will not uncompress using GUNZIP utility.
    > I opened the .z file in a hex editor, and was able to see that the
    > header does not begin with the magic header for gzip files. I fixed
    > that, but it still doesn't work.
    >
    > HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?
    >
    > thanks much for all your help!


    The HTTP header states that as well as having a content encoding of "gzip"
    applied, it also uses chunked transfer encoding. If that is the case then
    you will have to un-chunk the data before you can uncompress it.

    Post a hex dump of the start of file you have saved so we can see if it
    actually is chunked.

    Paul
     
    Paul Marquess, Dec 20, 2005
    #3
  4. Mark Smith <> wrote:

    > Following is the response header from IIS:
    >
    > HTTP/1.1 200 OK
    > Cache-Control: no-cache
    > Connection: close
    > Date: Mon, 19 Dec 2005 21:00:17 GMT
    > Pragma: no-cache
    > Content-Type: text/html
    > Expires: Wed, 01 Jan 1997 12:00:00 GMT
    > Server: Microsoft-IIS/6.0
    > P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
    > Content-Encoding: gzip
    > Vary: Accept-Encoding
    > Transfer-Encoding: chunked
    >
    > The http data was captured from all of the TCP packets in sequence, and
    > saved to a .Z file. The file will not uncompress using GUNZIP utility.
    > I opened the .z file in a hex editor, and was able to see that the
    > header does not begin with the magic header for gzip files. I fixed
    > that, but it still doesn't work.
    >
    > HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?



    It is not necessary to shout at us like that.


    > thanks much for all your help!



    Did you have a Perl question?


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Dec 20, 2005
    #4
  5. Mark Smith

    Mark Smith Guest

    Hi Paul

    Here are first few bytes of captured data. From what I could tell, it
    is chunked. And you can also see the 'magic header' of gzip (1F8B)
    starting at location 6....

    610D0A1F8B08000000000004000D0A3763300D0AEC5C7973E2C6B6FF7B5CF5BE

    thanks very much!

    data was from IIS 6
     
    Mark Smith, Dec 20, 2005
    #5
  6. Mark Smith

    Mark Smith Guest

    Sorry sir, I wasn't shouting. Just a bad typing style to make the
    question stand out.
     
    Mark Smith, Dec 20, 2005
    #6
  7. Mark Smith wrote:

    > Hi Paul
    >
    > Here are first few bytes of captured data. From what I could tell, it
    > is chunked. And you can also see the 'magic header' of gzip (1F8B)
    > starting at location 6....
    >
    > 610D0A1F8B08000000000004000D0A3763300D0AEC5C7973E2C6B6FF7B5CF5BE


    Yep, that's chunked and gzipped.

    Try filtering it through something like this to unchunk it, then see if
    gunzip can deal with it.

    undef $/;
    $_ = <>;

    while (length $_)
    {
    last if /^\r\n0+\r\n\r\n/;

    die "error dechunking\n"
    unless s/^(?:\r\n)?([0-9A-F]+)\r\n//i ;

    my $chunkSize = hex $1 ;

    print substr($_, 0, $chunkSize) ;
    substr($_, 0, $chunkSize) = '';
    }


    Paul
     
    Paul Marquess, Dec 20, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. chattycow
    Replies:
    4
    Views:
    3,802
    Chris Uppal
    May 16, 2006
  2. chattycow
    Replies:
    11
    Views:
    3,909
    Dale King
    May 22, 2006
  3. =?Utf-8?B?SGFycnkgS2Vjaw==?=

    Compress data on server and uncompress on client?

    =?Utf-8?B?SGFycnkgS2Vjaw==?=, May 22, 2006, in forum: ASP .Net
    Replies:
    4
    Views:
    489
  4. Replies:
    3
    Views:
    286
    Graeme Glass
    May 23, 2007
  5. Niels Egberts

    uncompress base64-gzipped string

    Niels Egberts, Jun 12, 2009, in forum: Python
    Replies:
    0
    Views:
    562
    Niels Egberts
    Jun 12, 2009
Loading...

Share This Page