How does one get from "ImportError: DLL load failed:..." to a culprit.dll and symbol?

Discussion in 'Python' started by Chris Cormie, Feb 23, 2009.

  1. Chris Cormie

    Chris Cormie Guest

    Hi,

    I've been Googling around on a moderately common Windows Python problem:
    a mismatch between the symbols a python extension thinks are available
    and the contents of the associated DLL. Python users running into this
    problem are greeted with:

    import <somemodule>
    "ImportError: DLL load failed: The specified procedure could not be found."

    That's it: no mention of which dll nor which symbol is causing the
    problem. Both these pieces of information are helpful/important in
    resolving this class of problem: how does one get Python to dump this
    debugging data?

    (>python -v -d
    doesn't help, BTW)

    Regards,
    Chris
    Chris Cormie, Feb 23, 2009
    #1
    1. Advertising

  2. Chris Cormie

    John Machin Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    On Feb 23, 2:56 pm, Chris Cormie <> wrote:
    > Hi,
    >
    > I've been Googling around on a moderately common Windows Python problem:
    > a mismatch between the symbols a python extension thinks are available
    > and the contents of the associated DLL. Python users running into this
    > problem are greeted with:
    >
    > import <somemodule>
    > "ImportError: DLL load failed: The specified procedure could not be found.."
    >
    > That's it: no mention of which dll nor which symbol is causing the
    > problem. Both these pieces of information are helpful/important in
    > resolving this class of problem: how does one get Python to dump this
    > debugging data?
    >
    > (>python -v -d
    > doesn't help, BTW)


    Use -vv ... you need to know exactly which of possibly many
    incarnations of somemodule.pyd Python is trying to import from.

    Then get a copy of the Dependency Walker from http://www.dependencywalker.com/

    Open c:\the\failing\somemodule.pyd with the DW. Look at grumble
    messages in the bottom pane. Note: you often get 1 or 2 warning
    messages with a pyd that's not having problems. Scroll through all the
    modules in the middle pane, looking for grumbles.

    If that not-very-technical description [all I've ever needed] doesn't
    help, you'll need to read the DW help file (HTFF1K) or wait till
    someone who knows what they are doing comes along :)

    By the way, I presume that you are aware that extensions on Windows
    are tied to a particular version of Python ... you would have got a
    different error message if you had a version mismatch (I think). If
    your problem is not obvious, come back here with:
    -- what version of Python you are running
    -- what DLLs you find in the middle pane whose names match
    (1) MSVC*.DLL
    (2) PYTHON??.DLL

    You may need to answer questions about the extension (e.g. compiler/
    linker options used) -- is it your extension or a 3rd party's?

    HTH,
    John
    John Machin, Feb 23, 2009
    #2
    1. Advertising

  3. Chris Cormie

    Chris Cormie Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?


    > If that not-very-technical description [all I've ever needed] doesn't
    > help, you'll need to read the DW help file (HTFF1K) or wait till
    > someone who knows what they are doing comes along :)


    LOL, I am that person :p
    Your technique works well and it does provide the information and it is
    a (roundabout) solution to this class of problems: thank you very much.
    It doesn't answer the original question so I'll restate it:

    How do you get *Python* to tell you the dll and the problem symbol? Not
    external tools, Python. Python at a low level is calling LoadLibrary and
    GetProcAddress to resolve symbols and the call fails. At that point it
    has the name of the dll and the problem symbol to hand and yet strangely
    only gives an opaque error message. How does one get Python to print out
    the faulty DLL and symbol?

    (Even if it means a debug build of Python and adding the printf
    yourself, I just want to know where we stand on this issue! :) )

    Best Regards,
    Chris
    Chris Cormie, Feb 23, 2009
    #3
  4. Chris Cormie

    John Machin Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    On Feb 23, 11:41 pm, Chris Cormie <> wrote:
    > > If that not-very-technical description [all I've ever needed] doesn't
    > > help, you'll need to read the DW help file (HTFF1K) or wait till
    > > someone who knows what they are doing comes along :)

    >
    > LOL, I am that person :p


    It wasn't apparent, and still isn't.

    > Your technique works well and it does provide the information and it is
    > a (roundabout) solution to this class of problems:  thank you very much..
    > It doesn't answer the original question so I'll restate it:
    >
    > How do you get *Python* to tell you the dll and the problem symbol? Not
    > external tools, Python. Python at a low level is calling LoadLibrary and
    > GetProcAddress to resolve symbols and the call fails. At that point it
    > has the name of the dll and the problem symbol to hand and yet strangely
    > only gives an opaque error message. How does one get Python to print out
    > the faulty DLL and symbol?


    If you know all that, then the answer is to submit a patch, isn't it??


    > (Even if it means a debug build of Python and adding the printf
    > yourself,


    printf? Maybe you shouldn't submit a patch after all.
    John Machin, Feb 23, 2009
    #4
  5. Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    En Mon, 23 Feb 2009 10:41:20 -0200, Chris Cormie
    <> escribió:

    >> If that not-very-technical description [all I've ever needed] doesn't
    >> help, you'll need to read the DW help file (HTFF1K) or wait till
    >> someone who knows what they are doing comes along :)

    >
    > LOL, I am that person :p
    > Your technique works well and it does provide the information and it is
    > a (roundabout) solution to this class of problems: thank you very much.
    > It doesn't answer the original question so I'll restate it:
    >
    > How do you get *Python* to tell you the dll and the problem symbol? Not
    > external tools, Python. Python at a low level is calling LoadLibrary and
    > GetProcAddress to resolve symbols and the call fails. At that point it
    > has the name of the dll and the problem symbol to hand and yet strangely
    > only gives an opaque error message. How does one get Python to print out
    > the faulty DLL and symbol?


    You can't, because it isn't Python who's trying to load the symbol - the
    *only* symbol that Python attempts to import itself is "initFOO" from
    FOO.pyd, and when that fails you get an explicit message.
    The error you see must come from the extension itself, and propagates into
    Python as an ImportError.

    --
    Gabriel Genellina
    Gabriel Genellina, Feb 23, 2009
    #5
  6. Chris Cormie

    Mark Hammond Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    On 23/02/2009 11:41 PM, Chris Cormie wrote:
    >
    >> If that not-very-technical description [all I've ever needed] doesn't
    >> help, you'll need to read the DW help file (HTFF1K) or wait till
    >> someone who knows what they are doing comes along :)

    >
    > LOL, I am that person :p


    LOL sounds right!

    > How do you get *Python* to tell you the dll and the problem symbol? Not
    > external tools, Python. Python at a low level is calling LoadLibrary and
    > GetProcAddress to resolve symbols and the call fails.


    It is the LoadLibrary that is failing; it should be obvious that if it
    was a simple GetProcAddress that was failing, Python would simply throw
    an exception rather than displaying the ugly dialog box you see.

    The problem is that some *other* DLL in the chain of dependencies is
    failing to load; please re-adjust your perceptions of your own knowledge
    and re-read John's initial response - if you follow his instructions
    that should all become quite obvious.

    Mark
    Mark Hammond, Feb 23, 2009
    #6
  7. Chris Cormie

    Chris Cormie Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    Mark Hammond wrote:
    > On 23/02/2009 11:41 PM, Chris Cormie wrote:
    >>
    >>> If that not-very-technical description [all I've ever needed] doesn't
    >>> help, you'll need to read the DW help file (HTFF1K) or wait till
    >>> someone who knows what they are doing comes along :)

    >>
    >> LOL, I am that person :p

    >
    > LOL sounds right!
    >
    >> How do you get *Python* to tell you the dll and the problem symbol? Not
    >> external tools, Python. Python at a low level is calling LoadLibrary and
    >> GetProcAddress to resolve symbols and the call fails.

    >
    > It is the LoadLibrary that is failing; it should be obvious that if it
    > was a simple GetProcAddress that was failing, Python would simply throw
    > an exception rather than displaying the ugly dialog box you see.


    I'm talking about the whole class of errors when loading an extension,
    the LoadLibary succeeds but the GetProcAddress fails. Not one specific
    error, a class of errors and *by definition* from the original question
    LoadLibrary succeeds. That's the point: at some point in this common
    scenario we have the path to the dll and a symbol we are resolving, and
    the symbol name, yet for whatever reason the GetProcAddress lookup
    fails, resulting in a surprisingly uninformative errors message.

    Regards,
    Chris.
    Chris Cormie, Feb 24, 2009
    #7
  8. Chris Cormie

    Chris Cormie Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    Gabriel Genellina wrote:
    > En Mon, 23 Feb 2009 10:41:20 -0200, Chris Cormie
    > <> escribió:
    >
    >>> If that not-very-technical description [all I've ever needed] doesn't
    >>> help, you'll need to read the DW help file (HTFF1K) or wait till
    >>> someone who knows what they are doing comes along :)

    >>
    >> LOL, I am that person :p
    >> Your technique works well and it does provide the information and it
    >> is a (roundabout) solution to this class of problems: thank you very
    >> much. It doesn't answer the original question so I'll restate it:
    >>
    >> How do you get *Python* to tell you the dll and the problem symbol?
    >> Not external tools, Python. Python at a low level is calling
    >> LoadLibrary and GetProcAddress to resolve symbols and the call fails.
    >> At that point it has the name of the dll and the problem symbol to
    >> hand and yet strangely only gives an opaque error message. How does
    >> one get Python to print out the faulty DLL and symbol?

    >
    > You can't, because it isn't Python who's trying to load the symbol - the
    > *only* symbol that Python attempts to import itself is "initFOO" from
    > FOO.pyd, and when that fails you get an explicit message.
    > The error you see must come from the extension itself, and propagates
    > into Python as an ImportError.


    Right -- now we are getting somewhere! That makes a lot of sense.

    What I was expecting to see next was an explicit GetProcAddress() call
    in the extension code that was failing in initFOO() or elsewhere in the
    extension code. But there are no such calls.

    I think what might have been making this confusing is that the root of
    my example problem is link time.

    Let me explain by giving the specific example case I have to hand.
    I orginally got on to this class of opaque runtime errors caused by
    missing symbols through building the pycURL extension on MinGW. As of
    Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
    maintains a set of .a files for system DLLs including the C Runtime.
    When building pycURL naturally I want to link against libmsvcr90.a so
    Python 2.6 and pycURL are using the same C Rutime. Problem was, MinGW's
    .a was wrong: it exports symbols that msvcr90.dll does not in fact
    possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
    error message: "ImportError: DLL load failed"

    Now I expected this meant an explict LoadLibrary() GetProcAddress()
    sequence was failing in the extension but perhaps that's not what
    happens at all?
    Chris
    Chris Cormie, Feb 24, 2009
    #8
  9. Chris Cormie

    Chris Cormie Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    John Machin wrote:
    > On Feb 23, 11:41 pm, Chris Cormie <> wrote:
    >>> If that not-very-technical description [all I've ever needed] doesn't
    >>> help, you'll need to read the DW help file (HTFF1K) or wait till
    >>> someone who knows what they are doing comes along :)

    >> LOL, I am that person :p

    >
    > It wasn't apparent, and still isn't.


    Yikes, enough with the rudeness people. I'm asking a reasonable question
    in a good faith.
    Chris Cormie, Feb 24, 2009
    #9
  10. Chris Cormie

    Chris Cormie Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    Mark Hammond wrote:
    > On 23/02/2009 11:41 PM, Chris Cormie wrote:
    >>
    >>> If that not-very-technical description [all I've ever needed] doesn't
    >>> help, you'll need to read the DW help file (HTFF1K) or wait till
    >>> someone who knows what they are doing comes along :)

    >>
    >> LOL, I am that person :p

    >
    > LOL sounds right!
    >
    >> How do you get *Python* to tell you the dll and the problem symbol? Not
    >> external tools, Python. Python at a low level is calling LoadLibrary and
    >> GetProcAddress to resolve symbols and the call fails.

    >
    > It is the LoadLibrary that is failing; it should be obvious that if it
    > was a simple GetProcAddress that was failing, Python would simply throw
    > an exception rather than displaying the ugly dialog box you see.
    >
    > The problem is that some *other* DLL in the chain of dependencies is
    > failing to load; please re-adjust your perceptions of your own knowledge
    > and re-read John's initial response - if you follow his instructions
    > that should all become quite obvious.
    >
    > Mark


    I think an example would help rather than me jabbering on about
    LoadLibary/GetProcAddress .

    It really is a case of Python loads module x depending on DLL y that is
    missing symbol z, yet the error message is just "load x failed" when in
    theory it could be "load x failed: y doesn't have z".

    However I think I am confusing people (and myself!) by assuming it's
    always as explicit as a LoadLibary/GetProcAddress sequence that is
    failing when people see this error.

    Let me explain by giving the specific example case I have to hand.
    I originally got on to this class of opaque runtime errors caused by
    missing symbols through building the pycURL extension on MinGW. As of
    Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
    maintains a set of .a files for system DLLs including the C Runtime.
    When building pycURL naturally I want to link against libmsvcr90.a so
    Python 2.6 and pycURL are using the same C Runtime. Problem was, MinGW's
    .a was wrong: it exports symbols that msvcr90.dll does not in fact
    possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
    error message: "ImportError: DLL load failed"

    I think Gabriel's got the part of the answer: she said "it isn't Python
    who's trying to load the symbol - the *only* symbol that Python attempts
    to import itself is "initFOO" from FOO.pyd"

    So my next question is: given the above missing-symbol problem, is the
    system able/willing to report the bad symbol reference in the .pyd when
    Python attempts to import initFOO?

    Regards,
    Chris.
    Chris Cormie, Feb 24, 2009
    #10
  11. Chris Cormie

    John Machin Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?

    On Feb 24, 2:54 pm, Chris Cormie <> wrote:
    > Mark Hammond wrote:
    > > On 23/02/2009 11:41 PM, Chris Cormie wrote:

    >
    > >>> If that not-very-technical description [all I've ever needed] doesn't
    > >>> help, you'll need to read the DW help file (HTFF1K) or wait till
    > >>> someone who knows what they are doing comes along :)

    >
    > >> LOL, I am that person :p

    >
    > > LOL sounds right!

    >
    > >> How do you get *Python* to tell you the dll and the problem symbol? Not
    > >> external tools, Python. Python at a low level is calling LoadLibrary and
    > >> GetProcAddress to resolve symbols and the call fails.

    >
    > > It is the LoadLibrary that is failing; it should be obvious that if it
    > > was a simple GetProcAddress that was failing, Python would simply throw
    > > an exception rather than displaying the ugly dialog box you see.

    >
    > > The problem is that some *other* DLL in the chain of dependencies is
    > > failing to load; please re-adjust your perceptions of your own knowledge
    > > and re-read John's initial response - if you follow his instructions
    > > that should all become quite obvious.

    >
    > > Mark

    >
    > I think an example would help rather than me jabbering on about
    > LoadLibary/GetProcAddress .
    >
    > It really is a case of Python loads module x depending on DLL y that is
    > missing symbol z, yet the error message is just "load x failed" when in
    > theory it could be "load x failed: y doesn't have z".
    >
    > However I think I am confusing people (and myself!) by assuming it's


    Don't assume; read the [open] source code; see below.

    > always as explicit as a LoadLibary/GetProcAddress sequence that is
    > failing when people see this error.
    >
    > Let me explain by giving the specific example case I have to hand.
    > I originally got on to this class of opaque runtime errors caused by
    > missing symbols through building the pycURL extension on MinGW. As of
    > Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
    > maintains a set of .a files for system DLLs including the C Runtime.
    > When building pycURL naturally I want to link against libmsvcr90.a so
    > Python 2.6 and pycURL are using the same C Runtime. Problem was, MinGW's
    >   .a was wrong: it exports symbols that msvcr90.dll does not in fact
    > possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
    > error message: "ImportError: DLL load failed"
    >
    > I think Gabriel's got the part of the answer: she said  "it isn't Python
    > who's trying to load the symbol - the *only* symbol that Python attempts
    > to import itself is "initFOO" from FOO.pyd"
    >
    > So my next question is: given the above missing-symbol problem, is the
    > system able/willing to report the bad symbol reference in the .pyd when
    > Python attempts to import initFOO?


    My reading of the Python 2.6.1 version of dynload_win.c tells me that:

    1. It uses LoadLibraryEx() to attempt to load the pyd given its full
    pathname

    2. If that fails, it gets the Windows error code using GetLastError()

    3. It gets the Windows message text using FormatMessage()

    4. It assembles the message as "DLL load failed: " plus the Windows
    message text (less any trailing "\r\n").

    Note that if the pyd load succeeds, it rummages in memory to find the
    pythonxx.dll used by the pyd so that it can be checked for
    consistency. Then and only then it uses GetProcAddress() to get the
    address of the initFOO function.

    Do you know a way of getting more info out of Windows than GetLastError
    ()?
    John Machin, Feb 24, 2009
    #11
  12. Chris Cormie

    Chris Cormie Guest

    Re: How does one get from "ImportError: DLL load failed:..." to aculprit .dll and symbol?


    > My reading of the Python 2.6.1 version of dynload_win.c tells me that:
    >
    > 1. It uses LoadLibraryEx() to attempt to load the pyd given its full
    > pathname
    >
    > 2. If that fails, it gets the Windows error code using GetLastError()
    >
    > 3. It gets the Windows message text using FormatMessage()
    >
    > 4. It assembles the message as "DLL load failed: " plus the Windows
    > message text (less any trailing "\r\n").
    >
    > Note that if the pyd load succeeds, it rummages in memory to find the
    > pythonxx.dll used by the pyd so that it can be checked for
    > consistency. Then and only then it uses GetProcAddress() to get the
    > address of the initFOO function.
    >
    > Do you know a way of getting more info out of Windows than GetLastError
    > ()?


    Thank you for your help, it is much appreciated:

    In summary:

    Q) How can one get Python to tell you which symbol is causing a problem
    when it loads an extension with a bad reference to a symbol in a DLL it
    uses?

    A) There is no such way: you *must* use external tools: When a symbol is
    missing form a DLL that a Python extension depends on, the failure will
    be picked up in Python immediately on attempting to load the extension:
    the extension code is never reached and there is no specific attempt
    made in Python to resolve the problem symbol. further, The normal
    platform APIs lack the ability to determine the cause of the load failure.

    > Do you know a way of getting more info out of Windows than GetLastError
    > ()?


    No, but presumably the ability to determine in code that a DLL has a bad
    reference and which symbol is bad exists because the depends program is
    able to do it. Of course it's a separate question as to whether Python
    wants to load itself down with complex platform specific code for a
    small corner case.

    Best Regards,
    Chris.
    Chris Cormie, Feb 24, 2009
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ajay

    ImportError: DLL load failed

    Ajay, Sep 26, 2004, in forum: Python
    Replies:
    0
    Views:
    2,187
  2. Andrew MacIntyre

    Re: ImportError: DLL load failed

    Andrew MacIntyre, Jun 28, 2008, in forum: Python
    Replies:
    0
    Views:
    586
    Andrew MacIntyre
    Jun 28, 2008
  3. Richard Whidden

    pythoncom -- ImportError: DLL load failed

    Richard Whidden, Apr 24, 2009, in forum: Python
    Replies:
    3
    Views:
    1,726
    Mark Hammond
    Apr 25, 2009
  4. André
    Replies:
    4
    Views:
    5,990
    thompjs
    Jan 24, 2011
  5. Albert-Jan Roskam
    Replies:
    2
    Views:
    74
    Chris Angelico
    Nov 25, 2013
Loading...

Share This Page