RE: How big is python24.dll?

Discussion in 'Python' started by Delaney, Timothy C (Timothy), Jul 13, 2004.

  1. "Martin v. Löwis" wrote:

    > Chris wrote:
    >> The size of the v2.4 dll isn't really known right now. It's only
    >> the first alpha which means that:
    >>
    >> a: more stuff is going to be added
    >> b: debugging code and other misc cruft is still in there.
    >>
    >> We won't have a decent idea of the dll size until the first release
    >> candidate later this year or early next year.

    >
    > b) is actually not true: the DLL does not contain debug code. The
    > alpha is very similar to the release wrt. the build process.


    Ouch. Did changing to .Net 2003 add that much overhead? Do we know what it would be with VC6?

    Tim Delaney
     
    Delaney, Timothy C (Timothy), Jul 13, 2004
    #1
    1. Advertising

  2. Delaney, Timothy C (Timothy) wrote:
    > Ouch. Did changing to .Net 2003 add that much overhead? Do we know what it would be with VC6?


    No, and no. python24.dll incorporates many extension modules which were
    separate .pyd files before. In particular, it contains all .pyd files
    which don't require additional libraries (DLLs or external libraries).
    A number of new extension modules were added for 2.4 also, which got
    incorporated into python24.dll.

    If you want to know the precise numbers, create a list of DLLs for
    Python 2.3, and a similar list for Python 2.4, and see what files
    went away. Also, compare the list of builtin modules to find out
    what modules have been added since 2.3.

    If this kind of combination has any effect on the size, it rather
    causes to safe space than to waste it: that way, Python ships with
    less DLLs, which means fewer files, and less overhead for DLL
    data structures.

    Regards,
    Martin
     
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Jul 13, 2004
    #2
    1. Advertising

  3. Martin v. Löwis wrote:

    > Delaney, Timothy C (Timothy) wrote:
    >
    >> Ouch. Did changing to .Net 2003 add that much overhead? Do we know
    >> what it would be with VC6?

    >
    >
    > No, and no. python24.dll incorporates many extension modules which were
    > separate .pyd files before. In particular, it contains all .pyd files
    > which don't require additional libraries (DLLs or external libraries).
    > A number of new extension modules were added for 2.4 also, which got
    > incorporated into python24.dll.
    >
    > If you want to know the precise numbers, create a list of DLLs for
    > Python 2.3, and a similar list for Python 2.4, and see what files
    > went away. Also, compare the list of builtin modules to find out
    > what modules have been added since 2.3.
    >
    > If this kind of combination has any effect on the size, it rather
    > causes to safe space than to waste it: that way, Python ships with
    > less DLLs, which means fewer files, and less overhead for DLL
    > data structures.
    >



    Also might want to mention: only those parts of a DLL in use are mapped
    into memory at run time. So using the larger DLL will not cause your
    Python program to suddenly become a bigger memory hog if you don't use
    the extra extension modules.

    At least, I think that's the way it works.

    --
    Mike
     
    Mike Thompson, Jul 13, 2004
    #3
  4. Delaney, Timothy C (Timothy)

    David Fraser Guest

    Martin v. Löwis wrote:
    > Delaney, Timothy C (Timothy) wrote:
    >
    >> Ouch. Did changing to .Net 2003 add that much overhead? Do we know
    >> what it would be with VC6?

    >
    >
    > No, and no. python24.dll incorporates many extension modules which were
    > separate .pyd files before. In particular, it contains all .pyd files
    > which don't require additional libraries (DLLs or external libraries).
    > A number of new extension modules were added for 2.4 also, which got
    > incorporated into python24.dll.
    >
    > If you want to know the precise numbers, create a list of DLLs for
    > Python 2.3, and a similar list for Python 2.4, and see what files
    > went away. Also, compare the list of builtin modules to find out
    > what modules have been added since 2.3.
    >
    > If this kind of combination has any effect on the size, it rather
    > causes to safe space than to waste it: that way, Python ships with
    > less DLLs, which means fewer files, and less overhead for DLL
    > data structures.
    >


    It may save space for the standard Python build. But it will make it
    harder to save space on py2exe or frozen Python programs for example...

    David
     
    David Fraser, Jul 13, 2004
    #4
  5. David Fraser <> writes:

    > Martin v. Löwis wrote:
    >> Delaney, Timothy C (Timothy) wrote:
    >>
    >>> Ouch. Did changing to .Net 2003 add that much overhead? Do we know
    >>> what it would be with VC6?


    >> No, and no. python24.dll incorporates many extension modules which
    >> were separate .pyd files before. In particular, it contains all .pyd
    >> files which don't require additional libraries (DLLs or external
    >> libraries). A number of new extension modules were added for 2.4
    >> also, which got incorporated into python24.dll.


    >> If you want to know the precise numbers, create a list of DLLs for
    >> Python 2.3, and a similar list for Python 2.4, and see what files
    >> went away. Also, compare the list of builtin modules to find out what
    >> modules have been added since 2.3.


    >> If this kind of combination has any effect on the size, it rather
    >> causes to safe space than to waste it: that way, Python ships with
    >> less DLLs, which means fewer files, and less overhead for DLL data
    >> structures.
    >>

    >
    > It may save space for the standard Python build. But it will make it
    > harder to save space on py2exe or frozen Python programs for example...


    IIRC, Martin did this change because I requested it, with py2exe in
    mind.

    If you really want the smallest file size in the py2exe builds, you may
    as well compile your own Python.dll, including only those modules as
    buildins which you really need.

    Thomas
     
    Thomas Heller, Jul 13, 2004
    #5
  6. "Martin v. Löwis" <> writes:

    > Delaney, Timothy C (Timothy) wrote:
    > > Ouch. Did changing to .Net 2003 add that much overhead? Do we know what it would be with VC6?

    >
    > No, and no. python24.dll incorporates many extension modules which were
    > separate .pyd files before. In particular, it contains all .pyd files
    > which don't require additional libraries (DLLs or external libraries).
    > A number of new extension modules were added for 2.4 also, which got
    > incorporated into python24.dll.


    CJKCodecs must add a fair bit, I'd have thought.

    Cheers,
    mwh

    --
    It's a measure of how much I love Python that I moved to VA, where
    if things don't work out Guido will buy a plantation and put us to
    work harvesting peanuts instead. -- Tim Peters, comp.lang.python
     
    Michael Hudson, Jul 13, 2004
    #6
  7. Delaney, Timothy C (Timothy)

    David Fraser Guest

    Thomas Heller wrote:
    > David Fraser <> writes:
    >
    >
    >>Martin v. Löwis wrote:
    >>
    >>>Delaney, Timothy C (Timothy) wrote:
    >>>
    >>>
    >>>>Ouch. Did changing to .Net 2003 add that much overhead? Do we know
    >>>>what it would be with VC6?

    >
    >
    >>>No, and no. python24.dll incorporates many extension modules which
    >>>were separate .pyd files before. In particular, it contains all .pyd
    >>>files which don't require additional libraries (DLLs or external
    >>>libraries). A number of new extension modules were added for 2.4
    >>>also, which got incorporated into python24.dll.

    >
    >
    >>>If you want to know the precise numbers, create a list of DLLs for
    >>>Python 2.3, and a similar list for Python 2.4, and see what files
    >>>went away. Also, compare the list of builtin modules to find out what
    >>>modules have been added since 2.3.

    >
    >
    >>>If this kind of combination has any effect on the size, it rather
    >>>causes to safe space than to waste it: that way, Python ships with
    >>>less DLLs, which means fewer files, and less overhead for DLL data
    >>>structures.
    >>>

    >>
    >>It may save space for the standard Python build. But it will make it
    >>harder to save space on py2exe or frozen Python programs for example...

    >
    >
    > IIRC, Martin did this change because I requested it, with py2exe in
    > mind.
    >
    > If you really want the smallest file size in the py2exe builds, you may
    > as well compile your own Python.dll, including only those modules as
    > buildins which you really need.


    OK, that's fine. How does it help py2exe then? Does anyone have a list
    of the modules added?

    David
     
    David Fraser, Jul 13, 2004
    #7
  8. David Fraser <> writes:

    >>>>No, and no. python24.dll incorporates many extension modules which
    >>>>were separate .pyd files before. In particular, it contains all .pyd
    >>>>files which don't require additional libraries (DLLs or external
    >>>>libraries). A number of new extension modules were added for 2.4
    >>>>also, which got incorporated into python24.dll.

    >>
    >>>>If you want to know the precise numbers, create a list of DLLs for
    >>>>Python 2.3, and a similar list for Python 2.4, and see what files
    >>>>went away. Also, compare the list of builtin modules to find out what
    >>>>modules have been added since 2.3.

    >>
    >>>>If this kind of combination has any effect on the size, it rather
    >>>>causes to safe space than to waste it: that way, Python ships with
    >>>>less DLLs, which means fewer files, and less overhead for DLL data
    >>>>structures.
    >>>>
    >>>
    >>>It may save space for the standard Python build. But it will make it
    >>>harder to save space on py2exe or frozen Python programs for example...


    >> IIRC, Martin did this change because I requested it, with py2exe in
    >> mind. If you really want the smallest file size in the py2exe
    >> builds, you may as well compile your own Python.dll, including only
    >> those modules as buildins which you really need.

    >
    > OK, that's fine. How does it help py2exe then? Does anyone have a list
    > of the modules added?


    Well, you can print sys.builtin_module_names, which outputs this:

    ('__builtin__', '__main__', '_bisect', '_codecs', '_codecs_big5',
    '_codecs_cp932', '_codecs_cp949', '_codecs_cp950',
    '_codecs_euc_jisx0213', '_codecs_euc_jp', '_codecs_euc_kr',
    '_codecs_gb18030', '_codecs_gb2312', '_codecs_gbk', '_codecs_hz ',
    '_codecs_iso2022_jp', '_codecs_iso2022_jp_1', '_codecs_iso2022_jp_2',
    '_codecs_iso2022_jp_3', '_codecs_iso2022_jp_ext ', '_codecs_iso2022_kr',
    '_codecs_johab', '_codecs_mapdata_ja_JP', '_codecs_mapdata_ko_KR',
    '_codecs_mapdata_zh_CN', '_codecs_mapdata_zh_TW', '_codecs_shift_jis',
    '_codecs_shift_jisx0213', '_csv', '_heapq', '_hotshot', '_locale',
    '_multibytecodec', '_random', '_sre', '_symtable', '_weakref',
    '_winreg', 'array', 'audioop', 'binascii', 'cPickle', 'cStringIO',
    'cmath', 'collections', 'datetime', 'errno', 'exceptions', 'gc',
    'imageop', 'imp', 'itertools', 'marshal', 'math', 'md5' , 'mmap',
    'msvcrt', 'nt', 'operator', 'parser', 'regex', 'rgbimg', 'rotor', 'sha',
    'signal', 'strop', 'struct', 'sys', ' thread', 'time', 'xreadlines',
    'xxsubtype', 'zipimport')

    Maybe more interesting is which modules remain non-builtin, here's the
    list from the Python 2.4 DLLs directory:

    12.07.2004 21:27 73.728 bz2.pyd
    12.07.2004 21:25 135.168 pyexpat.pyd
    12.07.2004 21:26 8.192 select.pyd
    18.06.2004 16:53 566.272 tcl84.dll
    18.06.2004 16:53 5.120 tclpip84.dll
    18.06.2004 17:03 1.027.584 tk84.dll
    12.07.2004 21:26 405.504 unicodedata.pyd
    12.07.2004 21:23 6.656 winsound.pyd
    12.07.2004 21:24 65.536 zlib.pyd
    12.07.2004 21:29 692.224 _bsddb.pyd
    12.07.2004 21:29 49.152 _socket.pyd
    12.07.2004 21:29 479.232 _ssl.pyd
    12.07.2004 21:28 12.800 _testcapi.pyd
    12.07.2004 21:27 40.960 _tkinter.pyd

    So, in *some* cases, py2exe only creates three files:
    python24.dll, library.zip, and yourapp.exe.

    Maybe the CJK codecs should be moved into separate pyds again, although
    that may depend on the part of the world you (or the users of your
    program) live in. Myself, I don't care too much about the python24.dll size.

    Thomas
     
    Thomas Heller, Jul 13, 2004
    #8
  9. On Tue, 13 Jul 2004 11:49:26 GMT, Michael Hudson <> wrote:
    > CJKCodecs must add a fair bit, I'd have thought.
    >


    I'm +0 for splitting CJKCodecs out from python24.dll if you are
    insisting on that. CJKCodecs will get bigger because of importing
    of CNS-11643 and BIG5HKSCS soon (about +400KB). Then:

    libpython24.dll w CJKCodecs inside: 1700 KB
    libpython24.dll w/o CJKCodecs inside: 1144 KB

    After CJKCodecs 1.1's importing, over half of python24.dll will be
    filled with CJKCodecs. I undertand that this can make Western users
    uncomfortable. And, CJKCodecs 1.1 will be consisted of only 5
    modules (not 25+ modules as currently in python CVS).

    Hye-Shik
     
    Hye-Shik Chang, Jul 13, 2004
    #9
  10. Delaney, Timothy C (Timothy)

    Peter Hansen Guest

    Hye-Shik Chang wrote:

    > I'm +0 for splitting CJKCodecs out from python24.dll if you are
    > insisting on that. CJKCodecs will get bigger because of importing
    > of CNS-11643 and BIG5HKSCS soon (about +400KB). Then:
    >
    > libpython24.dll w CJKCodecs inside: 1700 KB
    > libpython24.dll w/o CJKCodecs inside: 1144 KB
    >
    > After CJKCodecs 1.1's importing, over half of python24.dll will be
    > filled with CJKCodecs. I undertand that this can make Western users
    > uncomfortable. And, CJKCodecs 1.1 will be consisted of only 5
    > modules (not 25+ modules as currently in python CVS).


    Why is anyone concerned about the number of modules, whether 5
    or 25 (for CJKCodecs) or any other number (for the various .pyd
    files etc)? Once you've got more than a small handful, like
    two or maybe three tops, nobody is supposed to do any manual
    installation by copying individual files around. Using distutils
    or a Windows installer you basically needn't even _know_ there
    are all those files, almost even if you are the programmer!

    -Peter
     
    Peter Hansen, Jul 13, 2004
    #10
  11. Peter Hansen wrote:
    > Why is anyone concerned about the number of modules, whether 5
    > or 25 (for CJKCodecs) or any other number (for the various .pyd
    > files etc)? Once you've got more than a small handful, like
    > two or maybe three tops, nobody is supposed to do any manual
    > installation by copying individual files around. Using distutils
    > or a Windows installer you basically needn't even _know_ there
    > are all those files, almost even if you are the programmer!


    Because it makes maintenance more difficult: separate visual
    studio project files have to be created for them, and changes
    to one of them have to be manually propagated to all others,
    in both debug and release builds.

    At least that is my rationale for why I like the things the
    way they are. People who like to see a change need to
    contribute project files, at a minimum.

    Regards,
    Martin
     
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Jul 13, 2004
    #11
  12. Delaney, Timothy C (Timothy)

    Fuzzyman Guest

    Thomas Heller <> wrote in message news:<>...
    > David Fraser <> writes:
    >
    > >>>>No, and no. python24.dll incorporates many extension modules which
    > >>>>were separate .pyd files before. In particular, it contains all .pyd
    > >>>>files which don't require additional libraries (DLLs or external
    > >>>>libraries). A number of new extension modules were added for 2.4
    > >>>>also, which got incorporated into python24.dll.

    >
    > >>>>If you want to know the precise numbers, create a list of DLLs for
    > >>>>Python 2.3, and a similar list for Python 2.4, and see what files
    > >>>>went away. Also, compare the list of builtin modules to find out what
    > >>>>modules have been added since 2.3.

    >
    > >>>>If this kind of combination has any effect on the size, it rather
    > >>>>causes to safe space than to waste it: that way, Python ships with
    > >>>>less DLLs, which means fewer files, and less overhead for DLL data
    > >>>>structures.
    > >>>>
    > >>>
    > >>>It may save space for the standard Python build. But it will make it
    > >>>harder to save space on py2exe or frozen Python programs for example...

    >
    > >> IIRC, Martin did this change because I requested it, with py2exe in
    > >> mind. If you really want the smallest file size in the py2exe
    > >> builds, you may as well compile your own Python.dll, including only
    > >> those modules as buildins which you really need.

    > >
    > > OK, that's fine. How does it help py2exe then? Does anyone have a list
    > > of the modules added?

    >
    > Well, you can print sys.builtin_module_names, which outputs this:
    >
    > ('__builtin__', '__main__', '_bisect', '_codecs', '_codecs_big5',
    > '_codecs_cp932', '_codecs_cp949', '_codecs_cp950',
    > '_codecs_euc_jisx0213', '_codecs_euc_jp', '_codecs_euc_kr',
    > '_codecs_gb18030', '_codecs_gb2312', '_codecs_gbk', '_codecs_hz ',
    > '_codecs_iso2022_jp', '_codecs_iso2022_jp_1', '_codecs_iso2022_jp_2',
    > '_codecs_iso2022_jp_3', '_codecs_iso2022_jp_ext ', '_codecs_iso2022_kr',
    > '_codecs_johab', '_codecs_mapdata_ja_JP', '_codecs_mapdata_ko_KR',
    > '_codecs_mapdata_zh_CN', '_codecs_mapdata_zh_TW', '_codecs_shift_jis',
    > '_codecs_shift_jisx0213', '_csv', '_heapq', '_hotshot', '_locale',
    > '_multibytecodec', '_random', '_sre', '_symtable', '_weakref',
    > '_winreg', 'array', 'audioop', 'binascii', 'cPickle', 'cStringIO',
    > 'cmath', 'collections', 'datetime', 'errno', 'exceptions', 'gc',
    > 'imageop', 'imp', 'itertools', 'marshal', 'math', 'md5' , 'mmap',
    > 'msvcrt', 'nt', 'operator', 'parser', 'regex', 'rgbimg', 'rotor', 'sha',
    > 'signal', 'strop', 'struct', 'sys', ' thread', 'time', 'xreadlines',
    > 'xxsubtype', 'zipimport')
    >
    > Maybe more interesting is which modules remain non-builtin, here's the
    > list from the Python 2.4 DLLs directory:
    >
    > 12.07.2004 21:27 73.728 bz2.pyd
    > 12.07.2004 21:25 135.168 pyexpat.pyd
    > 12.07.2004 21:26 8.192 select.pyd
    > 18.06.2004 16:53 566.272 tcl84.dll
    > 18.06.2004 16:53 5.120 tclpip84.dll
    > 18.06.2004 17:03 1.027.584 tk84.dll
    > 12.07.2004 21:26 405.504 unicodedata.pyd
    > 12.07.2004 21:23 6.656 winsound.pyd
    > 12.07.2004 21:24 65.536 zlib.pyd
    > 12.07.2004 21:29 692.224 _bsddb.pyd
    > 12.07.2004 21:29 49.152 _socket.pyd
    > 12.07.2004 21:29 479.232 _ssl.pyd
    > 12.07.2004 21:28 12.800 _testcapi.pyd
    > 12.07.2004 21:27 40.960 _tkinter.pyd
    >
    > So, in *some* cases, py2exe only creates three files:
    > python24.dll, library.zip, and yourapp.exe.
    >
    > Maybe the CJK codecs should be moved into separate pyds again, although
    > that may depend on the part of the world you (or the users of your
    > program) live in. Myself, I don't care too much about the python24.dll size.
    >
    > Thomas



    I sometimes write command line tools that I bundle with p2exe - in
    which case having a smaller rather than more massive dll is preffered
    - particularly when I offer them for download. For small tools - size
    can matter. Oh well - I for one vote for keeping the codecs bundle
    seperate.

    As a side issue David Kastan has just compiled Python 2.3.4 for the
    pocketPC. He's endeavoured to keep the dll down to a minimum and
    everything else in a zip - making minimum size distributions possible.

    More of an issue on a machine with 32mb ram (including storage).....

    Regards,

    Fuzzy

    http://www.voidspace.org.uk/atlantibots/pythonutils.html
     
    Fuzzyman, Jul 14, 2004
    #12
  13. Fuzzyman wrote:
    > As a side issue David Kastan has just compiled Python 2.3.4 for the
    > pocketPC. He's endeavoured to keep the dll down to a minimum and
    > everything else in a zip - making minimum size distributions possible.
    >
    > More of an issue on a machine with 32mb ram (including storage).....


    On such a system, removing unneeded modules is trivial: just don't
    compile them (and comment-out the relevant lines in config.c).

    For people who want super-small packages of Python2.4, it would
    be possible to create a separate python24.dll, which is plug-in
    compatible except that it has fewer modules built-in. If there is
    a real need for such a thing, somebody will provide regardless
    of whether the standard distribution provides it.

    Regards,
    Martin
     
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Jul 14, 2004
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. A. B., Khalid

    How big is python24.dll?

    A. B., Khalid, Jul 12, 2004, in forum: Python
    Replies:
    5
    Views:
    423
    Miki Tebeka
    Jul 13, 2004
  2. Jose Rivera
    Replies:
    0
    Views:
    285
    Jose Rivera
    Jan 26, 2005
  3. Jose Rivera
    Replies:
    2
    Views:
    331
    Erik Johnson
    Jan 27, 2005
  4. Warren Postma
    Replies:
    1
    Views:
    475
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Apr 12, 2005
  5. Stormbringer

    How to make python24.dll smaller ?

    Stormbringer, Sep 14, 2005, in forum: Python
    Replies:
    2
    Views:
    401
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Sep 14, 2005
Loading...

Share This Page