Re: Python DLL in Windows Folder

Discussion in 'Python' started by Markus Gritsch, Dec 24, 2007.

  1. On 24/12/2007, Chris Mellon <> wrote:
    > On Dec 23, 2007 12:27 PM, Markus Gritsch <> wrote:
    > > On 23/12/2007, Christian Heimes <> wrote:
    > > > Markus Gritsch wrote:
    > > > > why does the Python installer on Windows put the Python DLL into the
    > > > > Windows system32 folder? Wouldn't it be more clean to place it into
    > > > > the Python installation folder beside the python.exe file?
    > > >
    > > > It's the easiest and best way to expose Python for 3rd party
    > > > applications and COM. The DLL is removed by the Windows Installer when
    > > > its usage counter drops to 0. There is no need to worry ;)

    > >
    > > I am not worrying about an orphaned DLL. The Problem is that this way
    > > the Python DLL is being made available for 3rd party applications,
    > > which possibly need a version of Python which is compiled using
    > > another compiler. We are embedding Python into our application which
    > > gets compiled using MSVC 8.0. We like to link dynamically, so the
    > > Python interpreter is not statically linked into the program. The
    > > Python DLL from the Python installer in the Windows system32 folder is
    > > compiled using MSVC 7.1.
    > >

    >
    >
    > What the python installer is doing is the Right Thing for making the
    > standard python dll available to third party applications.


    But this forces the 3rd party application being compiled with the same
    compiler as the Python DLL which is MSVC 7.1 for Python 2.5 which is a
    quite old compiler nowadays.

    > Applications that want a specific version of a specific DLL should use
    > the mechanisms available for doing so, instead of relying on there
    > being a specific version of the python dll in the windows folder.


    We do not rely on having some specific version of the Python DLL in
    the Windows system 32 folder. However, the MSVC 7.1 compiled version
    gets in the way when using a different compiler.

    Kind regards,
    Markus
     
    Markus Gritsch, Dec 24, 2007
    #1
    1. Advertising

  2. Markus Gritsch

    Lie Guest

    On Dec 25, 3:04 am, "Markus Gritsch" <> wrote:
    > On 24/12/2007, Chris Mellon <> wrote:
    >
    >
    >
    > > On Dec 23, 2007 12:27 PM, Markus Gritsch <> wrote:
    > > > On 23/12/2007, Christian Heimes <> wrote:
    > > > > Markus Gritsch wrote:
    > > > > > why does the Python installer on Windows put the Python DLL into the
    > > > > > Windows system32 folder?  Wouldn't it be more clean to place it into
    > > > > > the Python installation folder beside the python.exe file?

    >
    > > > > It's the easiest and best way to expose Python for 3rd party
    > > > > applications and COM. The DLL is removed by the Windows Installer when
    > > > > its usage counter drops to 0. There is no need to worry ;)

    >
    > > > I am not worrying about an orphaned DLL.  The Problem is that this way
    > > > the Python DLL is being made available for 3rd party applications,
    > > > which possibly need a version of Python which is compiled using
    > > > another compiler.  We are embedding Python into our application which
    > > > gets compiled using MSVC 8.0.  We like to link dynamically, so the
    > > > Python interpreter is not statically linked into the program.  The
    > > > Python DLL from the Python installer in the Windows system32 folder is
    > > > compiled using MSVC 7.1.

    >
    > > What the python installer is doing is the Right Thing for making the
    > > standard python dll available to third party applications.

    >
    > But this forces the 3rd party application being compiled with the same
    > compiler as the Python DLL which is MSVC 7.1 for Python 2.5 which is a
    > quite old compiler nowadays.


    Unfortunately, Windows doesn't give a lot of support around preventing
    DLL duplication. It is regular to found multiple duplicate DLLs on a
    Windows system (especially for different version), a common example is
    msvcrtxx.dll, which is C's runtime, many program pack their own msvcrt
    into their installation package. Therefore it is important for a
    windows program to include all the required DLLs into the installer
    package and use their own, unless they're sure that they can use other
    version (good programs are not DLL implementation specific and good
    DLLs maintain a stable interface).

    > > Applications that want a specific version of a specific DLL should use
    > > the mechanisms available for doing so, instead of relying on there
    > > being a specific version of the python dll in the windows folder.

    >
    > We do not rely on having some specific version of the Python DLL in
    > the Windows system 32 folder.  However, the MSVC 7.1 compiled version
    > gets in the way when using a different compiler.


    If that's the case, just use the MSVC 7.1's compiled code, there won't
    be much difference. If you insist to use your own version (or don't
    have a choice), you could instead pack your own DLL, use the local
    DLL, and ignore the system's DLL. That's the Best Practice in Windows.

    > Kind regards,
    > Markus
     
    Lie, Dec 24, 2007
    #2
    1. Advertising

  3. On 24/12/2007, Lie <> wrote:
    >
    > (good programs are not DLL implementation specific


    Assume an application embedding Python being compiled with MSVC 8.0.
    It uses the runtime msvcr80.dll. If you now pass objects created in
    your application to the Python interpreter which is compiled with MSVC
    7.1 which uses msvcr71.dll, you are asking for serious trouble. It is
    not a matter of being a "good program".

    > If that's the case, just use the MSVC 7.1's compiled code, there won't
    > be much difference.


    See above.

    > If you insist to use your own version (or don't
    > have a choice), you could instead pack your own DLL, use the local
    > DLL, and ignore the system's DLL.


    Yes, that works fine when deploying the application. However, during
    development it is inconveninent to put our own Python DLL beside our
    executable every time. Our Python DLL (compiled with MSVC 8.0) is in
    the PATH. If the original Python DLL would be located in the Python
    installation folder, there would be no problem. But since it is
    placed in the system32 folder which is searched before the PATH
    (http://msdn2.microsoft.com/en-us/library/ms682586(VS.85).aspx), the
    original Python DLL (compiled with MSVC 7.1) is used instead of your
    own one.

    Kind regards,
    Markus
     
    Markus Gritsch, Dec 25, 2007
    #3
  4. On 26/12/2007, Lie Ryan <> wrote:
    >
    >
    > On Dec 25, 2007 4:43 PM, Markus Gritsch <> wrote:
    > > On 24/12/2007, Lie <> wrote:
    > > >
    > > > (good programs are not DLL implementation specific

    > >
    > > Assume an application embedding Python being compiled with MSVC 8.0.
    > > It uses the runtime msvcr80.dll. If you now pass objects created in
    > > your application to the Python interpreter which is compiled with MSVC
    > > 7.1 which uses msvcr71.dll, you are asking for serious trouble. It is
    > > not a matter of being a "good program".

    >
    > don't cut my word, good programs aren't DLL implementation specific and good
    > DLL have consistent interface (consistent interface implies consistent
    > behavior). Of course this is a bit "utopia", as most DLL would sadly have to
    > change its interfaces once or twice and it is sometimes impossible for
    > programs to be completely free from its DLLs implementation


    Instead of being upset about cutting your word (which was not my
    intention, sorry about that), it would be nice if you could make a
    statement concerning the problem I mentioned: Having an object being
    created by one MSVC runtime, msvcr80.dll and passing it to another
    one, msvcr71.dll.

    > > > If that's the case, just use the MSVC 7.1's compiled code, there won't
    > > > be much difference.

    > >
    > > See above.
    > >

    >
    > I think you don't get my point... if you don't modify the python interpreter
    > (from what I understand, you just recompile your Python DLL without
    > modification), there is no need to compile your own interpreter, it just
    > makes problem and the performance benefits is outweighed by the problems of
    > having to use 3rd party Python DLL (even if you don't modify the code and
    > just recompile it, it is not the official release, thus 3rd party).


    I do get your point, but your point is maintaining compatibility
    between two different *versions* of a DLL/Program. I am talking about
    having a DLL/Program combination which uses different MSVC runtimes.

    > > > If you insist to use your own version (or don't
    > > > have a choice), you could instead pack your own DLL, use the local
    > > > DLL, and ignore the system's DLL.

    > >
    > > Yes, that works fine when deploying the application. However, during
    > > development it is inconveninent to put our own Python DLL beside our
    > > executable every time. Our Python DLL (compiled with MSVC 8.0) is in
    > > the PATH. If the original Python DLL would be located in the Python
    > > installation folder, there would be no problem. But since it is
    > > placed in the system32 folder which is searched before the PATH
    > >

    > (http://msdn2.microsoft.com/en-us/library/ms682586(VS.85).aspx),
    > the
    > > original Python DLL (compiled with MSVC 7.1) is used instead of your
    > > own one.

    >
    > If that is the case, you could just simply substitute the system32's dll
    > with your in the test machine. If you don't want to dirty the system
    > (perhaps because you use the system for other things too), then you could
    > benefit from dual booting or virtualization technology.
    >
    > Dual Booting is putting two (almost) completely seperate OSes in a single
    > machine.
    > Virtualization is a technology that allows an OS to be run inside another
    > OS, an example would be VMWare. This result in degraded performance in the
    > client OS, but it's easier to manage than Dual Booting.
    >
    > You don't need any special program to set up a dual boot system, although
    > you might require special knowledge on how to do it. For virtualization
    > technology, VMWare and Microsoft have a free version of their virtual
    > machines. So basically, these two set ups costs nothing.


    Thank you for explaining that.
     
    Markus Gritsch, Dec 26, 2007
    #4
  5. Markus Gritsch

    Lie Guest

    On Dec 27, 12:30 am, "Markus Gritsch" <> wrote:
    > On 26/12/2007, Lie Ryan <> wrote:
    > > > On Dec 25, 2007 4:43 PM, Markus Gritsch <> wrote:
    > > > On 24/12/2007, Lie <> wrote:

    >
    > > > > (good programs are not DLL implementation specific

    >
    > > > Assume an application embedding Python being compiled with MSVC 8.0.
    > > > It uses the runtime msvcr80.dll.  If you now pass objects created in
    > > > your application to the Python interpreter which is compiled with MSVC
    > > > 7.1 which uses msvcr71.dll, you are asking for serious trouble.  It is
    > > > not a matter of being a "good program".

    >
    > > don't cut my word, good programs aren't DLL implementation specific and good
    > > DLL have consistent interface (consistent interface implies consistent
    > > behavior). Of course this is a bit "utopia", as most DLL would sadly have to
    > > change its interfaces once or twice and it is sometimes impossible for
    > > programs to be completely free from its DLLs implementation

    >
    > Instead of being upset about cutting your word (which was not my
    > intention, sorry about that), it would be nice if you could make a
    > statement concerning the problem I mentioned: Having an object being
    > created by one MSVC runtime, msvcr80.dll and passing it to another
    > one, msvcr71.dll.


    I'm not upset, but if my wording seems to be like an upset person I'm
    sorry, English is not my first language, and I'm very bad at choosing
    words.

    A simple solution (but possibly inefficient, if the object is complex
    and need to be exchanged thousands of times) for the problem mentioned
    is to convert the object to a common agreed format that both sides can
    understand when transferring the data such as a simple string (not a
    string object, just simple string with no strings attached: pun (not)
    intended).

    In your case, which is:
    > Assume an application embedding Python being compiled with MSVC 8.0.
    > It uses the runtime msvcr80.dll. If you now pass objects created in
    > your application to the Python interpreter which is compiled with MSVC
    > 7.1 which uses msvcr71.dll, you are asking for serious trouble. It is
    > not a matter of being a "good program".


    No, you're not asking for trouble, you're passing the object to the
    Python DLL, not to the msvcr71 behind the Python so it shouldn't be a
    problem in a good designed program and DLL. This isn't a case about
    msvcr71 talking with msvcr8, nor it is msvcr71 with your program, nor
    it is msvcr8 with Python DLL, this is a case of Your Program talking
    to Python DLL.

    One way to solve this is:
    Assuming you don't have access to Python internals (although since
    Python is open source, you do have access) you could build a wrapper
    around Python Interpreter. The wrapper has two functions. One: accepts
    a string input (string as simple string, not string object) that
    describes the original object and returns a file that Python can pick
    with pickle. Two: accepts a pickled object and returns a string that
    describes the pickled object that Your Application could understand.

    This functions could be built in into Your Application, but the
    easiest way is to build the wrapper with Python, as you won't need to
    know how to read and write pickled file.

    For a simple case, where the object that needs to be passed is
    predetermined your function could just pass any data that is required
    to reconstruct the original object to the wrapper (for example, in a
    vector class, just the values of x, y, z), a versatile way to pass
    them is by command-line arguments, like "wrapper 1 2 3 > output". The
    wrapper would add any other required infos (methods, operator
    overloads, etc) before pickling the object and call the Python DLL to
    unpickle the object. For the vice versa case, the wrapper unpickle the
    file returned from Python DLL, and convert it to a format Your Program
    could understand, then notifying your program.

    Of course there may be other easier, more efficient ways to solve this
    problem, but this is a proof-of-concept that it is possible to compile
    your application in VS8 while still using the official VS<unknown>
    Python.

    But I can't think of a need why Python DLL needs to be used like this
    though

    > > > > If that's the case, just use the MSVC 7.1's compiled code, there won't
    > > > > be much difference.

    >
    > > > See above.

    >
    > > I think you don't get my point... if you don't modify the python interpreter
    > > (from what I understand, you just recompile your Python DLL without
    > > modification), there is no need to compile your own interpreter, it just
    > > makes problem and the performance benefits is outweighed by the problems of
    > > having to use 3rd party Python DLL (even if you don't modify the code and
    > > just recompile it, it is not the official release, thus 3rd party).

    >
    > I do get your point, but your point is maintaining compatibility
    > between two different *versions* of a DLL/Program.  I am talking about
    > having a DLL/Program combination which uses different MSVC runtimes.


    It doesn't matter, your program is talking to the Python DLL not to
    the MSVC Runtimes behind it. If you pass an object to the DLL, a good
    program would use an intermediary format that is understood by both
    sides (Python DLL and Application), and not rely on the implementation
    behind the Pythond DLL to understand the object being traded.
     
    Lie, Dec 26, 2007
    #5
  6. Markus Gritsch

    Tim Roberts Guest

    "Markus Gritsch" <> wrote:
    >
    >Instead of being upset about cutting your word (which was not my
    >intention, sorry about that), it would be nice if you could make a
    >statement concerning the problem I mentioned: Having an object being
    >created by one MSVC runtime, msvcr80.dll and passing it to another
    >one, msvcr71.dll.


    The C runtime doesn't create objects. Python does use it to allocate
    memory, but in the Microsoft run-times, malloc and free call the Win32 heap
    APIs, no matter what the version.

    It's true that mallocing in one CRT DLL and freeing in another can cause
    problems, but in Python, I don't think that can happen. Proper Python
    add-ins call Python APIs to create and destroy objects, so only the Python
    runtime will manage the memory.
    --
    Tim Roberts,
    Providenza & Boekelheide, Inc.
     
    Tim Roberts, Dec 27, 2007
    #6
  7. >> Instead of being upset about cutting your word (which was not my
    >> intention, sorry about that), it would be nice if you could make a
    >> statement concerning the problem I mentioned: Having an object being
    >> created by one MSVC runtime, msvcr80.dll and passing it to another
    >> one, msvcr71.dll.

    >
    > The C runtime doesn't create objects.


    Depends on your definition of "object". fopen will create objects,
    and passing those to a different CRT will cause crashes.

    Regards,
    Martin
     
    Martin v. Löwis, Dec 27, 2007
    #7
  8. On 27 Dez., 05:49, Tim Roberts <> wrote:
    > It's true that mallocing in one CRT DLL and freeing in another can cause
    > problems, but in Python, I don't think that can happen.  Proper Python
    > add-ins call Python APIs to create and destroy objects, so only the Python
    > runtime will manage the memory.


    There is at least one actual problem I came across in an application
    linked with MSVCR 8.0, loading the Python DLL linked against MSVCR
    7.1. When running a Python file using one of the PyRun_File[1]
    functions, the passed file handle has to be created from the embedding
    application. This handle is incompatible with the C file handling
    functions used by the C runtime of the Python DLL, which causes it to
    crash when calling PyRun_File.

    regards,
    Michael Doppler

    [1] http://docs.python.org/api/veryhigh.html
     
    Michael Doppler, Dec 29, 2007
    #8
  9. > There is at least one actual problem I came across in an application
    > linked with MSVCR 8.0, loading the Python DLL linked against MSVCR
    > 7.1. When running a Python file using one of the PyRun_File[1]
    > functions, the passed file handle has to be created from the embedding
    > application. This handle is incompatible with the C file handling
    > functions used by the C runtime of the Python DLL, which causes it to
    > crash when calling PyRun_File.


    Indeed. That is one of the main reasons to remove that API in Py3k.

    Regards,
    Martin
     
    Martin v. Löwis, Dec 29, 2007
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Anushi
    Replies:
    5
    Views:
    9,001
  2. Markus Gritsch

    Python DLL in Windows Folder

    Markus Gritsch, Dec 23, 2007, in forum: Python
    Replies:
    7
    Views:
    844
    Markus Gritsch
    Dec 27, 2007
  3. Markus Gritsch

    Re: Python DLL in Windows Folder

    Markus Gritsch, Dec 23, 2007, in forum: Python
    Replies:
    1
    Views:
    510
    Ross Ridge
    Dec 23, 2007
  4. Chris Mellon

    Re: Python DLL in Windows Folder

    Chris Mellon, Dec 24, 2007, in forum: Python
    Replies:
    22
    Views:
    1,129
    Ross Ridge
    Dec 29, 2007
  5. H. Simpson
    Replies:
    4
    Views:
    343
    H. Simpson
    Aug 3, 2004
Loading...

Share This Page