Re: zipimport limited to 65536 files?

Discussion in 'Python' started by Terry Reedy, May 2, 2014.

  1. Terry Reedy

    Terry Reedy Guest

    On 5/1/2014 3:49 PM, Tom Graves wrote:
    > Hello,
    >
    > I am trying to use python (2.6.6) to read a jar file that contains
    > python files. I'm simply setting
    > PYTHONPATH=spark-assembly-1.0.0-SNAPSHOT-hadoop2.4.0.jar.
    > Unfortunately it fails to read the python files from the jar file and
    > if run in verbose mode just shows:
    >
    > import zipimport # builtin
    > # installed zipimport hook
    > # zipimport: found 0 names in spark-assembly-1.0.0-SNAPSHOT-hadoop2.4.0.jar
    >
    > I was messing around and noticed that if I reduce the number of files
    > and directories in the jar to below 65536 then it works:
    >
    > import zipimport # builtin
    > # installed zipimport hook
    > # zipimport: found 65452 names in pyspark.jar
    >
    > Is this a known limitation


    It is definitely not documented, which it should be if intentional.

    > or is this perhaps fixed in newer version or


    Install 3.4 and try. Or go to http://bugs.python.org,
    click search, enter 'zipimport' in the title box, change 'open' in the
    status box to 'dont care', and look at 44 titles returned (I did not see
    anything that looked relevant).

    All I know is that in the discussion about a ziplib issue, someone
    mentioned using zips with 100000s of files. But zipimport could have an
    additional limitation.

    > is there a work around?


    Multiple archives?

    > Note, I'm not subscribed to the mailing list so please copy me in
    > response if possible.


    --
    Terry Jan Reedy
    Terry Reedy, May 2, 2014
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bryan Parkoff

    65536 Switch Cases

    Bryan Parkoff, Sep 16, 2004, in forum: C++
    Replies:
    5
    Views:
    454
    Ivan Vecerina
    Sep 17, 2004
  2. Bob Swerdlow
    Replies:
    1
    Views:
    359
  3. Gabriele *Darkbard* Farina

    zipimport

    Gabriele *Darkbard* Farina, May 4, 2005, in forum: Python
    Replies:
    1
    Views:
    377
    Jonathan Brady
    May 4, 2005
  4. Replies:
    2
    Views:
    269
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Jan 28, 2006
  5. Tom Graves

    zipimport limited to 65536 files?

    Tom Graves, May 1, 2014, in forum: Python
    Replies:
    0
    Views:
    61
    Tom Graves
    May 1, 2014
Loading...

Share This Page