java.util.zip Limitations

G

Gerry Wheeler

I have some ZIP files that apparently can't be opened by
java.util.zip.ZipFile, although they work fine with various Linux zip
utilities. ZipFile throws a ZipException in <init>, but the message in the
exception doesn't say exactly why. (And, of course, I've closed the window
where it used to be. I can get it again if needed.)

A) Does anyone have experience with various flavors of zip files, and any
hints?

B) What utilities might I use to get more information about my zip files
to see if they are, in fact, incompatible with java.util.zip? (I'm playing
on Linux, but I have Windows available if needed.)
 
A

Andrew Thompson

I have some ZIP files that apparently can't be opened by
java.util.zip.ZipFile, although they work fine with various Linux zip
utilities. ZipFile throws a ZipException in <init>, but the message in the
exception doesn't say exactly why. (And, of course, I've closed the window
where it used to be. I can get it again if needed.)
Click to expand...

Well ..D'uh!
A) Does anyone have experience with various flavors of zip files, and any
hints?
Click to expand...

Yes. Ask smart qns. quoting the exact error..

Oh, what? You mean tips w/ zips?

1 or 2
B) What utilities might I use to get more information about my zip files
to see if they are, in fact, incompatible with java.util.zip? (I'm playing
on Linux, but I have Windows available if needed.)
Click to expand...

I am unfamiliar w/ Linux (I'm sure one
of the many Linux folks will wander by
in a moment) but would recommend you
have a look at the zip file in WinZip
on the Windows PC to check it is made
with standard compression.

Also check the compression level, I
know Java cannot write in anything but
default compression level, but am
not sure whether it can read higher levels.

If the file is either, make a new one
with standard compression type/level
and try that.

HTH
 
G

Gerry Wheeler

If the file is either, make a new one with standard compression type/level
and try that.
Click to expand...

I can't change the zip file. It is one of a series of files distributed by
the National Weather Service via a data stream, and I'm hoping to be able
to unzip it on the fly as I receive the data stream. I have to handle
whatever they send.
 
A

Andrew Thompson

I can't change the zip file. It is one of a series of files distributed by
the National Weather Service via a data stream, and I'm hoping to be able
to unzip it on the fly as I receive the data stream. I have to handle
whatever they send.
Click to expand...

O...K..... but what about the information..

(A.T. previously)
[ ....would recommend you
have a look at the zip file in WinZip
on the Windows PC to check it is made
with standard compression.

Also check the compression level, I
know Java cannot write in anything but
default compression level, but am
not sure whether it can read higher levels. ]

So.. what are the compression level/types
of the supplied Zips?

If the Zip files supplied are standard,
the problem is your code.

How big are the Zips? If you can find one
under 50Kb I'll have a look at it myself..
 
G

Gerry Wheeler

O...K..... but what about the information..
Click to expand...

Two things: a) I recompiled my program with the zip code in place so I
could recreate the exception, and b) I found the Linux zipinfo program
which gives oodles of information about the zip file.

So, the exception is:

java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method) at
java.util.zip.ZipFile.<init>(ZipFile.java:112) at
java.util.zip.ZipFile.<init>(ZipFile.java:128) at
kg4nbb.emwin.EmwinPacketProcessor.decompressProduct(EmwinPacketProcessor.java:258)
at
kg4nbb.emwin.EmwinPacketProcessor.processPacket(EmwinPacketProcessor.java:120)
at kg4nbb.emwin.EmwinIngestor.ingest(EmwinIngestor.java:188) at
kg4nbb.emwin.EmwinIngestor.main(EmwinIngestor.java:74)

[That might wrap a little, but I think it's readable.] As I mentioned
previously, it's not particularly informative. I thought at first it might
have to do with permissions, or looking in the wrong directory or
something. But the ZipFile object is instantiated with a File object, and
tests on that indicate the file really does exist.



Here's the information about one of the offending zip files, produced by
zipinfo:



Archive: AFMMHXNC.ZIS 3072 bytes 1 file

End-of-central-directory record:
-------------------------------

Actual offset of end-of-central-dir record: 2477 (000009ADh)
Expected offset of end-of-central-dir record: 2477 (000009ADh)
(based on the length of the central directory and its expected offset)

This zipfile constitutes the sole disk of a single-part archive; its
central directory contains 1 entry. The central directory is 58
(0000003Ah) bytes long, and its (expected) offset in bytes from the
beginning of the zipfile is 2419 (00000973h).

There is no zipfile comment.

Central directory entry #1:
---------------------------

AFMMHXNC.TXT

offset of local header from start of archive: 0 (00000000h) bytes
file system or operating system of origin: MS-DOS, OS/2 or NT FAT
version of encoding software: 2.0
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 2.0
compression method: deflated
compression sub-type (deflation): normal
file security status: not encrypted
extended local header: no
file last modified on (DOS date/time): 2004 Apr 22 00:38:20
32-bit CRC value (hex): 75e0f21d
compressed size: 2377 bytes
uncompressed size: 15643 bytes
length of filename: 12 characters
length of extra field: 0 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: text
non-MSDOS external file attributes: 000000 hex
MS-DOS file attributes (20 hex): arc

There is no file comment.



(Yes, for whatever reason, the National Weather Service uses the extension
ZIS on their zip files; I dunno why.) From what I understand about zip
files, it all looks pretty standard. I'm not sure, though, about the need
for version 2.0 software. This is on Linux, so I must presume they're
referring to the version of zip, unzip, etc. Do the Java classes meet that
requirement?
 
G

Gerry Wheeler

Here's the information about one of the offending zip files, produced by
zipinfo:
Click to expand...

Hmmm, the rewrapping on that (in my newsreader) makes it rather hard to
read. I can repost or send the output directly if needed.
 
A

Andrew Thompson

So, the exception is:
Click to expand...

NOW we are cooking with gas!
java.util.zip.ZipException: error in opening zip file .....
kg4nbb.emwin.EmwinPacketProcessor.decompressProduct(EmwinPacketProcessor.java:258)
at
Click to expand...

OK. Did you write 'EmwinPacketProcessor'?

What's at line 258?
[That might wrap a little, but I think it's readable.]
Click to expand...

It wraps, but it is fine.
..As I mentioned
previously, it's not particularly informative.
Click to expand...

On the contrary, you just have to figure how
to read them and work out what to do with the
information gained.
..I thought at first it might
have to do with permissions,
Click to expand...

Almost certainly not. Those generally
mention SecurityAccessExceptions,
SecurityManagers and such, rather than
'..ZipException: error in opening zip file'
..or looking in the wrong directory or
something. But the ZipFile object is instantiated with a File object, and
tests on that indicate the file really does exist.
Click to expand...

Tests inside your code?

Are they before/after line 258?
Here's the information about one of the offending zip files, produced by
zipinfo: ....
Archive: AFMMHXNC.ZIS 3072 bytes 1 file ......
compression method: deflated
compression sub-type (deflation): normal
file security status: not encrypted
Click to expand...

That is all looking pretty standard..

.....
(Yes, for whatever reason, the National Weather Service uses the extension
ZIS on their zip files; I dunno why.)
Click to expand...

"We are individuals"? ;-)
...From what I understand about zip
files, it all looks pretty standard.
Agreed.

..I'm not sure, though, about the need
for version 2.0 software.
Click to expand...

I do not think that is relevant. But am not sure.

Next step.

Try this with a Java program that
_can_ read Zip files.
<http://www.physci.org/applet/ziplet.jsp>

Drop the 3/4Kb zip to 'physci' at the main domain,
and I'll give it a go with Ziplet (I'll put it
up at my site if you like)
 
R

Roedy Green

Not so. I have several applets that read
data from zip files created with WinZip.
Click to expand...

It depends. If you are lucky, Winzip and Pkzip-created files will use
only the compression algorithms that Java supports. IN GENERAL that
will not be so. However, you will often luck out, especially if you
have control of creating the zip and did not let it use anything
fancy.

I went through all this with The Replicator and finally decided my
best bet was to create and access zips only with Java to avoid WinZip
or PkZip getting tricky on me every once in a while.

See http://mindprod.com/products.html#REPLICATOR
 
A

Andrew Thompson

On Sat, 24 Apr 2004 05:36:34 GMT, Roedy Green wrote:

I got completely sidetracked by this,
but just thought I would add..

It does seem like the OP's Zip (or
rather .sip) files are standard algorithm/
default compression. AFAIU Java should
be able to read them.

_Now_ to the sidetrack...

(problems with zips)
I went through all this with The Replicator
Click to expand...

[ ;-) ]
Yes. I have played with Zip's a bit
and fallen prey to most of the traps
you mention.

Fortunately I usually make the Zip files
I need, so I can ensure they are JAva
compatible whether I roll a class to do
it in Java, use the jar tool, ..or get
lazy and drag'n'drop the files in my
(standard compression/algorithm default)
version of WinZip.
...and finally decided my
best bet was to create and access zips only with Java to avoid WinZip
or PkZip getting tricky on me every once in a while.
Click to expand...

I still like to get stung occasionally
to warn me of problems I might face,
...for example, if I offered a signed
applet that advertised "Drag'n'Drop
Zip/UnZip".

But I can appreciate that if you have
a wealth of knowledge of a topic, that
would be of more hindrance than help.
 
C

Chris Uppal

Gerry said:
Central directory entry #1:
---------------------------

AFMMHXNC.TXT

offset of local header from start of archive: 0 (00000000h) bytes
file system or operating system of origin: MS-DOS, OS/2 or NT FAT
version of encoding software: 2.0
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 2.0
compression method: deflated
compression sub-type (deflation): normal
file security status: not encrypted
extended local header: no
file last modified on (DOS date/time): 2004 Apr 22 00:38:20
32-bit CRC value (hex): 75e0f21d
compressed size: 2377 bytes
uncompressed size: 15643 bytes
length of filename: 12 characters
length of extra field: 0 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: text
non-MSDOS external file attributes: 000000 hex
MS-DOS file attributes (20 hex): arc

There is no file comment.
Click to expand...

This all looks normal to me (on the assumption that zipinfo converts the "disk
number" field to 1-based indexing for human consumption, I'd expect the value
in the file to be 0).

As Roedy has said, there are several other compression schemes (4 by my count,
although they come is several sub-flavours) that "real" Zip utilities from
PKWare can use (actually I've never seen any of them in the wild), but this
file advertises itself to use "deflate" compression (type 8) which *everyone*
can read.

The version field is also apparently OK. I have routinely used Java's Zip
stuff to read Zip files with versions of 2.0 and 2.3.

All of which makes:
java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method) at
java.util.zip.ZipFile.<init>(ZipFile.java:112) at
java.util.zip.ZipFile.<init>(ZipFile.java:128) at
Click to expand...

look extremely odd. Can you use Java to unzip *anything* on that machine ?
Maybe there's a problem with the installation of the native library. Also
worth testing to see if you can unzip the same file using Java on a Windows box
(most people have one they can reach, even if they are not willing to admit it
in public ;-)

If you like, and if you are allowed to, you can email the offending ZIS file to
me (note the spam blocker). I'm currently working on Zip file encoding and
decoding (in a different language) so I'm pretty well placed to say if there is
*really* anything wrong with the file. (And another test case never hurts ;-)

(Yes, for whatever reason, the National Weather Service uses the extension
ZIS on their zip files; I dunno why.)
Click to expand...

Possibly to avoid under-developed email filters that refuse *all* attachments
with a .zip extension.

-- chris
 
G

Gerry Wheeler

OK. Did you write 'EmwinPacketProcessor'? What's at line 258?
Click to expand...

Yes, that's my code. Line 258 is where I instantiate the ZipFile, like
this:

File f = <get a File for the data received over the stream>; if
(f.exists()) {
ZipFile z = new ZipFile(f); // line 258 // other stuff to
deal with the ZipEntries
}
}
On the contrary, you just have to figure how to read them and work out
what to do with the information gained.
Click to expand...

Yeah, but the message is "error in opening zip file". Well, yeah, thanks a
lot. :) A little more detail would have been useful.
Drop the 3/4Kb zip to 'physci' at the main domain, and I'll give it a go
with Ziplet (I'll put it up at my site if you like)
Click to expand...

OK, I'll send one over. Thanks.
 
G

Gerry Wheeler

OK, I'll send one over. Thanks.
Click to expand...

As I was looking for a good sample to send to Andrew, I remembered
something. I don't think this will (or should) affect this, but you never
know...

The NWS data stream is sent as packets of information. But the people who
wrote it were not good at packet stuff, so all the packets are exactly
1024 bytes, padded with 0's if necessary. And there's no indicator in the
packet header of the length of the actual data. So, the last packet of the
file (including zip files) has a bunch of extra 0's on the end. My code
removes the 0's if it recognizes (from the file name) that the data is a
text file. But I didn't want to start messing with the various binary file
types, so I leave the 0's in place for all of them.

If I understand zip files correctly, this should be OK because there's an
offset given to the zip directory, so the program should be able to find
all the parts and ignore the extra junk. But, if the Java code is being
extra picky, perhaps it's rejecting the file because of this. I'll whip up
a test version of the program to remove all the trailing 0's from zip
files to see what happens. I'll keep it dumb and hope I don't remove part
of the file's data.
 
G

Gerry Wheeler

I'll whip up
a test version of the program to remove all the trailing 0's from zip
files to see what happens. I'll keep it dumb and hope I don't remove part
of the file's data.
Click to expand...

Result: it doesn't work. Not only doesn't it fix my original problem, it
renders the zip file unreadable by other utilities as well. Apparently a
zip file has some information at the end that may be zero, so my quicky
test removed that from the file. Oh well.
 
G

Gerry Wheeler

Java reads it just fine, I made no
changes to the file whatsoever.
Click to expand...

OK, that's good to know. (And the sample I sent you had the extra 0's on
the end, so I know I don't need to worry about them.) So now I need to
look at my code. I'll mess with it a bit and let you know what I get.

Thanks!
 
G

Gerry Wheeler

Strip out the _all_ stuff you can
before it stops failing*, then post that.
<http://www.physci.org/codes/sscce.jsp>

* You sohuld be able to do it
in under 40 lines.
Click to expand...

Well, it's more than 40 lines, but I did put in some comments and white
space. :)

On my system, this test program throws the same exception as my real
program. Andrew -- you already have the sample zip file. Others who may be
following along can try this on any random zip file, or I can send you one
of my samples for testing. I'm going to take this with me to work today to
try it out on a Windows computer to see if there's a difference.



----------- sample code here --------------

package kg4nbb.emwin;

import java.io.File;
import java.util.Enumeration;
import java.util.zip.*;


/**
* This is a test program to attempt to unzip a file from the
* National Weather Service's EMWIN data stream.
*
* Under normal use, the file would arrive in packets within
* the EMWIN stream. This simplified test assumes the file has
* already been saved and is ready to be unzipped.
*
* @author gwheeler
*/
public class TestZipFile {

/** Creates a new instance of TestZipFile */
public TestZipFile() {
}

/**
* The main code, where everything starts.
*
* @param args[0] the name of the zip file
*/
public static void main(String[] args) {
if (args.length == 1) {
new TestZipFile().test(args[0]);
}
else {
System.err.println("usage: TestZipFile <filename>");
}
}


/**
* Attempts to unzip the specified file.
*/
private void test(String filename) {
try {
File f = new File(filename);
if (f.exists()) {
// Here's where the problem usually occurs.

System.out.println("attempting to open " + f);
ZipFile z = new ZipFile(f);
System.out.println("zip file " + f + " opened");

// Just show what's in it.

Enumeration entries = z.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = (ZipEntry)entries.nextElement();
System.out.println(" found entry " + entry.getName());
}

}
else {
System.err.println("can't find " + f);
}
}
catch (Exception e) {
// For testing purposes, just catch and report all exceptions.

e.printStackTrace();
}
}
}



----------- sample code ends --------------
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

invalid archives with java.util.zip 3
creating identical zip archives with java and zip tools 9
java.util.zip - problem opening some legitimite me zip files 1
java.util.zip on hpux 8
ftplib limitations? 9
Limitations and workarounds to expressions defining size of an array 44
Opening a zip causes OutOfMemoryError 3
I need help in understanding these files on my phone, Could someone help me understand these files? Urgent help needed. Please help. 1

Members online

No members online now.
Total: 28 (members: 0, guests: 28)
Robots: 462

Forum statistics

Threads
473,770
Messages
2,569,583
Members
45,074
Latest member
StanleyFra

Latest Threads

Top