Dynamically loading a static C library (compiler suggestion?)

Discussion in 'C Programming' started by Nickolai Leschov, Dec 24, 2007.

  1. Hello all,

    I am programming an embedded controller that has a 'C' library for using
    its system functions (I/O, timers, all the specific devices). The
    supplied library has .LIB and .H files.

    How can I dynamically load a LIB file and access all its functions?
    Surely someone has solved similar task?

    My intention is to use a Forth system for programming the controller,
    calling library functions from it.

    Regards,
    Nickolai Leschov
     
    Nickolai Leschov, Dec 24, 2007
    #1
    1. Advertising

  2. Nickolai Leschov

    Frank Buss Guest

    Nickolai Leschov wrote:

    > I am programming an embedded controller that has a 'C' library for using
    > its system functions (I/O, timers, all the specific devices). The
    > supplied library has .LIB and .H files.
    >
    > How can I dynamically load a LIB file and access all its functions?
    > Surely someone has solved similar task?
    >
    > My intention is to use a Forth system for programming the controller,
    > calling library functions from it.


    Usually you can't load a lib file, it is used to link to C programs (or to
    make it easier to use DLLs for Windows). But you could use something like
    http://ficl.sourceforge.net/ and write C wrappers for all the library
    functions you need.

    --
    Frank Buss,
    http://www.frank-buss.de, http://www.it4-systems.de
     
    Frank Buss, Dec 24, 2007
    #2
    1. Advertising

  3. Frank Buss wrote:
    >
    > Usually you can't load a lib file, it is used to link to C programs (or to
    > make it easier to use DLLs for Windows).


    I see, loading a lib (or obj) file is something you normally not do when
    building a 'C' program. It is not the usual approach, but is it
    possible? How?

    > http://ficl.sourceforge.net/ and write C wrappers for all the library
    > functions you need.


    Thanks, I'll examine this closer.
    Will it compile to and run on i186 ?
     
    Nickolai Leschov, Dec 24, 2007
    #3
  4. Nickolai Leschov

    jacob navia Guest

    Nickolai Leschov wrote:
    > Hello all,
    >
    > I am programming an embedded controller that has a 'C' library for using
    > its system functions (I/O, timers, all the specific devices). The
    > supplied library has .LIB and .H files.
    >
    > How can I dynamically load a LIB file and access all its functions?
    > Surely someone has solved similar task?
    >
    > My intention is to use a Forth system for programming the controller,
    > calling library functions from it.
    >
    > Regards,
    > Nickolai Leschov


    It is easy:

    You extract all the object files, then look the relocations
    inside. If you can resolve all the symbols then you
    do the relocations within a malloced memory buffer and
    jump into it.

    lcc-win proposes a software that does that under the windows system.

    For your system it would need to be adapted a bit, depending on your
    object format


    --
    jacob navia
    jacob at jacob point remcomp point fr
    logiciels/informatique
    http://www.cs.virginia.edu/~lcc-win32
     
    jacob navia, Dec 24, 2007
    #4
  5. Nickolai Leschov

    Richard Bos Guest

    Nickolai Leschov <> wrote:

    > I am programming an embedded controller that has a 'C' library for using
    > its system functions (I/O, timers, all the specific devices). The
    > supplied library has .LIB and .H files.
    >
    > How can I dynamically load a LIB file and access all its functions?


    That depends (at least in C; don't know about Forth) completely on the
    system you're using.

    Richard
     
    Richard Bos, Dec 24, 2007
    #5
  6. Nickolai Leschov

    Richard Bos Guest

    jacob navia <> wrote:

    > Nickolai Leschov wrote:
    > > I am programming an embedded controller that has a 'C' library for using
    > > its system functions (I/O, timers, all the specific devices). The
    > > supplied library has .LIB and .H files.
    > >
    > > How can I dynamically load a LIB file and access all its functions?
    > > Surely someone has solved similar task?
    > >
    > > My intention is to use a Forth system for programming the controller,
    > > calling library functions from it.

    >
    > It is easy:


    *Snigger*

    > You extract all the object files, then look the relocations
    > inside. If you can resolve all the symbols then you
    > do the relocations within a malloced memory buffer and
    > jump into it.


    *Snigger*

    And how do you propose to do that...

    > lcc-win proposes a software that does that under the windows system.


    ....on an embedded controller?

    As usual, jacob, you don't realise how deep the waters are in which
    you're dangling your toes.

    Richard
     
    Richard Bos, Dec 24, 2007
    #6
  7. Nickolai Leschov

    jacob navia Guest

    Richard Bos wrote:
    > jacob navia <> wrote:
    >
    >> Nickolai Leschov wrote:
    >>> I am programming an embedded controller that has a 'C' library for using
    >>> its system functions (I/O, timers, all the specific devices). The
    >>> supplied library has .LIB and .H files.
    >>>
    >>> How can I dynamically load a LIB file and access all its functions?
    >>> Surely someone has solved similar task?
    >>>
    >>> My intention is to use a Forth system for programming the controller,
    >>> calling library functions from it.

    >> It is easy:

    >
    > *Snigger*
    >
    >> You extract all the object files, then look the relocations
    >> inside. If you can resolve all the symbols then you
    >> do the relocations within a malloced memory buffer and
    >> jump into it.

    >
    > *Snigger*
    >
    > And how do you propose to do that...
    >
    >> lcc-win proposes a software that does that under the windows system.

    >
    > ...on an embedded controller?
    >
    > As usual, jacob, you don't realise how deep the waters are in which
    > you're dangling your toes.
    >
    > Richard


    I have written that software... Of course since it is from me and I am
    by definition an ignorant, I must have written it in a dream.

    Most embedded controllers do not invent new object code formats but
    use an existing one. COFF is highly used, and object files under
    windows are COFF.

    You are of course not an ignorant. This is highly visible in the
    depth of your arguments

    * Snigger *

    All you manage to say :)


    --
    jacob navia
    jacob at jacob point remcomp point fr
    logiciels/informatique
    http://www.cs.virginia.edu/~lcc-win32
     
    jacob navia, Dec 24, 2007
    #7
  8. Nickolai Leschov

    Doug Guest

    On Dec 24, 11:06 am, (Richard Bos) wrote:
    > jacob navia <> wrote:
    > > Nickolai Leschov wrote:
    > > > I am programming an embedded controller that has a 'C' library for using
    > > > its system functions (I/O, timers, all the specific devices). The
    > > > supplied library has .LIB and .H files.

    >
    > > > How can I dynamically load a LIB file and access all its functions?
    > > > Surely someone has solved similar task?

    >
    > > > My intention is to use a Forth system for programming the controller,
    > > > calling library functions from it.

    >
    > > It is easy:

    >
    > *Snigger*
    >
    > > You extract all the object files, then look the relocations
    > > inside. If you can resolve all the symbols then you
    > > do the relocations within a malloced memory buffer and
    > > jump into it.

    >
    > *Snigger*
    >
    > And how do you propose to do that...
    >
    > > lcc-win proposes a software that does that under the windows system.

    >
    > ...on an embedded controller?


    Presumably using SDK tools that come with the controller. Or write
    your own.

    Most of the dev we do for embedded devices isn't done 'on the device'
    anyway. The images are built on standard development machines then
    transferred to the device. What Jacob describes is perfectly possible
    within this development environment.

    I don't see what embedded controllers have to do with this anyway.
    The OP didn't mention them. I assume you brought them up just to be
    difficult.

    >
    > As usual, jacob, you don't realise how deep the waters are in which
    > you're dangling your toes.


    No. As usual, we're seeing another pointless attack on Jacob. I
    suspect, as usual, his answer has gone over your head.

    I saw nothing but a useful answer from Jacob, I'm afraid. As usual.
    This time it was possibly even on-topic.
     
    Doug, Dec 24, 2007
    #8
  9. Nickolai Leschov

    Bart C Guest

    "Nickolai Leschov" <> wrote in message
    news:fkntm4$f9g$...
    > Hello all,
    >
    > I am programming an embedded controller that has a 'C' library for using
    > its system functions (I/O, timers, all the specific devices). The supplied
    > library has .LIB and .H files.
    >
    > How can I dynamically load a LIB file and access all its functions? Surely
    > someone has solved similar task?
    >
    > My intention is to use a Forth system for programming the controller,
    > calling library functions from it.


    Let me try and answer this.

    You have a microprocessor with integrated devices, for which you have a
    library of useful functions. This library is in .LIB format and designed for
    use with a C compiler for that processor together with a Linker that
    produces a binary image that can be transfered somehow into the processor's
    memory.

    So presumably you have such tools available and you are supposed to program
    it in C. But you want to use Forth.

    Well, I suppose you could write some code in C, linked to the LIB, and
    create a mechanism so that your Forth language can call it (some sort jump
    table in memory). Sounds fiddly though. Another possibility is just to
    program the devices directly in Forth, if that's not too much work. Just
    need the datasheet.

    But perhaps best to ask in comp.arch.embedded and specify the exact device
    number.

    Bart
     
    Bart C, Dec 24, 2007
    #9
  10. Nickolai Leschov

    astrobe Guest


    > Well, I suppose you could write some code in C, linked to the LIB, and
    > create a mechanism so that your Forth language can call it (some sort jump
    > table in memory). Sounds fiddly though. Another possibility is just to
    > program the devices directly in Forth, if that's not too much work. Just
    > need the datasheet.
    >


    Or, one can write the Forth system in assembly language, hence
    prodincing an object file, and statically link it with the C library.

    Nickolai, you should give us more information on your system: Is the
    Forth system written in C, assembly, or something else? Does the
    system run an OS?

    Amicalement,
    Astrobe
     
    astrobe, Dec 24, 2007
    #10
  11. Nickolai Leschov

    Brad Eckert Guest

    On Dec 24, 2:55 am, Frank Buss <> wrote:
    >
    > Usually you can't load a lib file, it is used to link to C programs (or to
    > make it easier to use DLLs for Windows). But you could use something likehttp://ficl.sourceforge.net/and write C wrappers for all the library
    > functions you need.
    >


    You could also take a look at F-: http://sourceforge.net/projects/fminus

    It's made for small embedded systems and uses C as its assembly
    language. Your Forth program gets converted into a C source file
    containing a VM and the Forth converted into bytecode executable by
    the VM. C can call Forth functions using VM(VM_FnName). Forth can call
    C by placing the C code in CODE words.

    Brad
     
    Brad Eckert, Dec 24, 2007
    #11
  12. Nickolai Leschov

    Jack Klein Guest

    On Mon, 24 Dec 2007 03:43:37 -0800 (PST), Doug
    <> wrote in comp.lang.c:

    > On Dec 24, 11:06 am, (Richard Bos) wrote:
    > > jacob navia <> wrote:
    > > > Nickolai Leschov wrote:
    > > > > I am programming an embedded controller that has a 'C' library for using


    [snip]

    > I don't see what embedded controllers have to do with this anyway.
    > The OP didn't mention them. I assume you brought them up just to be
    > difficult.


    You quoted the OP's mention of them.

    Besides, the mechanics of libraries and dynamic loading are completely
    off-topic in comp.lang.c, they are 100% implementation and 0%
    language.

    --
    Jack Klein
    Home: http://JK-Technology.Com
    FAQs for
    comp.lang.c http://c-faq.com/
    comp.lang.c++ http://www.parashift.com/c -faq-lite/
    alt.comp.lang.learn.c-c++
    http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html
     
    Jack Klein, Dec 24, 2007
    #12
  13. Nickolai Leschov

    Stephen Pelc Guest

    On Mon, 24 Dec 2007 12:21:03 +0300, Nickolai Leschov
    <> wrote:

    >I am programming an embedded controller that has a 'C' library for using
    >its system functions (I/O, timers, all the specific devices). The
    >supplied library has .LIB and .H files.
    >
    >How can I dynamically load a LIB file and access all its functions?
    >Surely someone has solved similar task?
    >
    >My intention is to use a Forth system for programming the controller,
    >calling library functions from it.


    Last time we did this, it was a bit of a nightmare! The most
    difficult part was making sure that the C library was properly
    initialised. My recommendations would be:

    C Construction:
    1) Make a C program whose main() returns.
    2) Make a function table of the functions you wish to use.
    Store the base address of the function table at a known
    location in RAM. This will form the core of your Forth
    interface.

    Forth construction:
    1) Produce a wrapper that provides the Forth interface
    to the C functions.
    2) Provide a Forth interface for the functions in the
    table.

    Run-time:
    2) Make the Forth call the C entry point and hope that main()
    returns to you. You may have to place games with the crt0.s
    to achieve this. The point is that much of the C initailisation
    occurs before main() is run, so you have to start at the entry
    point. All this stuff is compiler dependent.
    2) From the known location call into the C using the Forth
    wrapper.

    If you want dynamic loading, then you'll have to provide a
    suitable loader, but the priciples are the same.

    More generic solutions are based on the "don't do that" rule.
    1) Use a Forth cross compiler
    or
    2) Use a Forth written in C.

    Stephen


    --
    Stephen Pelc,
    MicroProcessor Engineering Ltd - More Real, Less Time
    133 Hill Lane, Southampton SO15 5AF, England
    tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
    web: http://www.mpeforth.com - free VFX Forth downloads
     
    Stephen Pelc, Dec 24, 2007
    #13
  14. Nickolai Leschov

    Frank Buss Guest

    Nickolai Leschov wrote:

    > I see, loading a lib (or obj) file is something you normally not do when
    > building a 'C' program. It is not the usual approach, but is it
    > possible? How?


    As others have written, it is possible, but difficult. You have to parse
    the object file format, do the initialization etc. But maybe it is possible
    with some other ideas mentioned in this thread, e.g. writing just a main
    functions in a C program, linking all library functions and passing the
    function pointers to the Forth system. Depends a bit on the platform and
    Forth system how easy this will be.

    > > http://ficl.sourceforge.net/ and write C wrappers for all the library
    > > functions you need.

    >
    > Thanks, I'll examine this closer.
    > Will it compile to and run on i186 ?


    Yes, it compiles on nearly everything with a standard C compiler and
    sufficient memory. I have just compiled it on a WindowsCE ARM embedded
    system with some minor fixes needed for Visual C++ .NET 2005. But it is not
    as fast as native Forth systems.

    --
    Frank Buss,
    http://www.frank-buss.de, http://www.it4-systems.de
     
    Frank Buss, Dec 24, 2007
    #14
  15. Re: Dynamically loading a static C library (Updated; also compilersuggestion?)

    Hello again,

    First, I want to thank all of you for rushing out to help me, though I
    supplied so little clues. I really wanted not to bother you with
    unneeded details that could possibly lead the discussion to the wrong
    path, but in fact said not enough to be understood.

    So, let me clarify the situation:

    I am programming a microcontroller, ICPDAS i-8411 to be exact.
    (http://www.icpdas.com/products/PAC/i-8000/i-8411-i-8811.htm)
    Microcontroller not in the sense of "Embedded CPU" but "a small box". A
    box has slots: one for CPU board and several more for I/O boards. From
    the programmer's point of view microcontroller is a PC. It has:
    - i186-compatible CPU; 512K RAM
    - 512K flash-based disk with simple filesystem
    - DOS-compatible OS in boot sector
    - some specific devices

    Manufacturer suggests using one of the following compilers:
    - Borland C++ 3.1~5.02
    - MSC 6.0
    - MSVC (before version 1.52)
    - TC 2.01 (*available for free)
    - TC++ 1.01 (*available for free)
    and supplies a system library for their controller for use with one of
    the above compilers. Library doesn't have source and comes as .lib and
    ..h files. I also have some of the above compilers .

    So, the proposed design cycle looks like this:
    I write my program in 'C' and link against the manufacturer's library
    with one of recommended compilers, on Windows PC (maybe even in a DOS
    box). Then I transfer binary file to microcontroller using
    manufacturer's terminal program. I run it. Repeat until satisfied with
    result.

    There are 2 major problems for me:

    1. Proposed compilers are all old cruft.
    I would like to use more up-to-date tools for programming my embedded
    PC. I want something that is at least either (a) non-proprietary or (b)
    modern, or both. The best I saw is Borland C++ 5.02 (in the modern
    department, but still it could use some improvement) And how do you
    obtain license for the compilers _that_ old? Will Borland or Microsoft
    grant me license for them tomorrow? (or even today) Will DOS compilers
    run on my computers tomorrow? (or even today)

    2. No debugging.
    It doesn't need explaining, I guess. Debugging is good.

    So I started to think about using a Forth system on this controller. I
    can choose a free as in freedom Forth for DOS, put it on controller's
    disk and if I find a way to use system library from it, I'll be fine.
    With a little modification, I hope, I will have a license-free
    development environment with debugging. I'll also have the bonus of the
    compiler on the target, just in case.

    I want to use the supplied library because as opposed to microcontroller
    as "Embedded CPU", I don't have the datasheet for all my hardware. I
    don't get schematics, it's not practiced in the case of microcontrollers
    in the sense of "small box". Instead, I get system library that should
    fit a lineup of controllers.

    So, I thought that maybe I should dynamically load the system library
    and gain acess to its functions. From what I gather LIB is a collection
    of OBJ files in the OMF format, developed by Microsoft, but later
    standardized. I found a bunch of information here, on the 'D' community
    forum: http://www.dsource.org/forums/viewtopic.php?t=959
    Seems that if no one else, developers of exotic languages confront this
    problem?

    Either that, or I could use a better development environment for 16-bit
    DOS. (But the debugging is out of the question, as I understand)

    Previously I wrote:

    > I am programming an embedded controller that has a 'C' library for using
    > its system functions (I/O, timers, all the specific devices). The
    > supplied library has .LIB and .H files.
    >
    > How can I dynamically load a LIB file and access all its functions?
    > Surely someone has solved similar task?
    >
    > My intention is to use a Forth system for programming the controller,
    > calling library functions from it.
     
    Nickolai Leschov, Dec 24, 2007
    #15
  16. Nickolai Leschov

    santosh Guest

    OT - Re: Dynamically loading a static C library (Updated; also compiler suggestion?)

    Nickolai Leschov wrote:

    > Hello again,
    >
    > First, I want to thank all of you for rushing out to help me, though I
    > supplied so little clues. I really wanted not to bother you with
    > unneeded details that could possibly lead the discussion to the wrong
    > path, but in fact said not enough to be understood.
    >
    > So, let me clarify the situation:
    >
    > I am programming a microcontroller, ICPDAS i-8411 to be exact.
    > (http://www.icpdas.com/products/PAC/i-8000/i-8411-i-8811.htm)
    > Microcontroller not in the sense of "Embedded CPU" but "a small box".
    > A box has slots: one for CPU board and several more for I/O boards.
    > From the programmer's point of view microcontroller is a PC. It has:
    > - i186-compatible CPU; 512K RAM
    > - 512K flash-based disk with simple filesystem
    > - DOS-compatible OS in boot sector
    > - some specific devices
    >
    > Manufacturer suggests using one of the following compilers:
    > - Borland C++ 3.1~5.02
    > - MSC 6.0
    > - MSVC (before version 1.52)
    > - TC 2.01 (*available for free)
    > - TC++ 1.01 (*available for free)
    > and supplies a system library for their controller for use with one of
    > the above compilers. Library doesn't have source and comes as .lib and
    > .h files. I also have some of the above compilers .
    >
    > So, the proposed design cycle looks like this:
    > I write my program in 'C' and link against the manufacturer's library
    > with one of recommended compilers, on Windows PC (maybe even in a DOS
    > box). Then I transfer binary file to microcontroller using
    > manufacturer's terminal program. I run it. Repeat until satisfied with
    > result.
    >
    > There are 2 major problems for me:
    >
    > 1. Proposed compilers are all old cruft.
    > I would like to use more up-to-date tools for programming my embedded
    > PC. I want something that is at least either (a) non-proprietary or
    > (b) modern, or both. The best I saw is Borland C++ 5.02 (in the modern
    > department, but still it could use some improvement) And how do you
    > obtain license for the compilers _that_ old? Will Borland or Microsoft
    > grant me license for them tomorrow? (or even today) Will DOS compilers
    > run on my computers tomorrow? (or even today)


    DOS programs will continue to run (with minor problems?) for some time
    yet.

    As for the compilers, there is nothing wrong with working software. Not
    all systems have the same unnaturally accelerated life cycle as the PC.
    If they work for your systems and the manufacturer recommends them,
    then I don't see the problem. In any case, have you tried something
    like GCC or the Green Hills compiler?

    > 2. No debugging.
    > It doesn't need explaining, I guess. Debugging is good.


    Debugging embedded programs is rather tricky. Does your microcontroller
    have an emulation program? Can you debug through a serial port?

    Regarding dynamically loading your library file, depending on it's exact
    format it might be difficult to impossible. Judging by the file name
    extension, the manufacturer apparently intends the library to be used
    statically linked. Try contacting your manufacturer for other options.
     
    santosh, Dec 25, 2007
    #16
  17. Nickolai Leschov

    cr88192 Guest

    "Nickolai Leschov" <> wrote in message
    news:fkntm4$f9g$...
    > Hello all,
    >
    > I am programming an embedded controller that has a 'C' library for using
    > its system functions (I/O, timers, all the specific devices). The supplied
    > library has .LIB and .H files.
    >
    > How can I dynamically load a LIB file and access all its functions? Surely
    > someone has solved similar task?
    >
    > My intention is to use a Forth system for programming the controller,
    > calling library functions from it.
    >


    a few misc comments (have read rest of the thread):
    yeah, you have a 'fun' situation...


    I have done dynamic loading of static libraries, of course, I wrote my own
    compiler framework, and do more than a few little hacks (try to get an app
    to start rewriting and relinking itself at runtime, and one will see a
    little of this).


    now, probably, any way you can link against this image "statically", it
    preferred (for example, if you can link the library with your forth
    framework/program, that is idea).


    also possible:
    at least a few DOS-era compilers had mechanisms for creating DLL-like
    libraries. this may be a possibility, but one will need to investigate this.

    but, assuming dynamic linking:
    now, the big fun in this case, is not the loading/relocation, but the MZ-EXE
    file format.
    for example, the format lacks a symbol table (I think, but I have hardly
    really investigated the MZ-EXE format).

    in good old PE-COFF (windows), we have a symbol table (symbols may be
    stripped, but we can optionally leave them in). I guess it depends a lot on
    the compiler/linker being used. it is likely that the compiler and linker
    may have some means of including debugging information in the compiled EXE.

    this may well be the best bet to look into.

    look into leaving debugging info in the EXE, look into locating and
    deciphering this info, and then look into writing code to use this data for
    linking the lib against the existing image.


    another possibility:
    one can write their own linker, and manually include the needed info in a
    usable form (possibly, going further, for example, including
    proxies/jump-tables and other features to aid dynamic linking).


    or, if one is "really" lucky, maybe they will have a compiler that does some
    of this for them.


    as for the other issue, the longest and hardest route is this:
    write your own compiler framework.


    > Regards,
    > Nickolai Leschov
     
    cr88192, Dec 25, 2007
    #17
  18. Re: OT - Re: Dynamically loading a static C library (Updated; alsocompiler suggestion?)

    santosh wrote:

    > DOS programs will continue to run (with minor problems?) for some time
    > yet.


    They will do for now. But I'm not satisfied with this situation, thank you.

    > If they work for your systems and the manufacturer recommends them,
    > then I don't see the problem.

    Programming-wise, I think manufacturer is not very competent. I believe
    that manufacturer can produce hardware and software that _works_ for a
    price that I'm happy with, but I don't trust them in anything that's
    related to programming. I use their library because it provides
    interface to their hardware, but I'm not very fond of it.
    > In any case, have you tried something
    > like GCC or the Green Hills compiler?

    I'll look at them, thanks.

    > Debugging embedded programs is rather tricky. Does your microcontroller
    > have an emulation program? Can you debug through a serial port?

    No, my microcontroller doesn't have an emulation program. Besides,
    emulation won't buy me much compared to real debugging. I can use DOS
    emulation, but it's of little use plus it's not integrated into IDE.

    Debugging through a serial port is also not available, as far as I know.
    This is what I wanted to accomplish with Forth.

    > Regarding dynamically loading your library file, depending on it's exact
    > format it might be difficult to impossible. Judging by the file name
    > extension, the manufacturer apparently intends the library to be used
    > statically linked. Try contacting your manufacturer for other options.

    As far as I know it's OMF object file format. Is OMF loading hard? Who
    did it?
     
    Nickolai Leschov, Dec 25, 2007
    #18
  19. On Dec 25, 3:20 am, "cr88192" <> wrote:

    > also possible:
    > at least a few DOS-era compilers had mechanisms for creating DLL-like
    > libraries. this may be a possibility, but one will need to investigate this.


    > but, assuming dynamic linking:
    > now, the big fun in this case, is not the loading/relocation, but the MZ-EXE
    > file format.
    > for example, the format lacks a symbol table (I think, but I have hardly
    > really investigated the MZ-EXE format).


    The .LIB format is a collection of relocatable object modules, which
    have a hashed symbol table and a name header in each module.

    Name header:
    vvv QUOTE vvv

    82H LHEADR--LIBRARY MODULE HEADER RECORD
    ========================================

    Description
    -----------

    This record is very similar to the THEADR record. It is used to
    indicate the name of a module within a library file (which has an
    internal organization different from that of an object module).

    History
    -------

    This record type was defined in the original Intel specification with
    the same format but with a different purpose, so its use for libraries
    should be considered a Microsoft extension.

    Record Format
    -------------

    1 2 1 <-String Length-> 1
    82 Record String Name String Checksum
    Length Length


    NOTE: In LINK, THEADR, and LHEADR records are handled identically.
    See Appendix 2 for a complete description of Microsoft's library
    file format.

    ^^^ UNQUOTE ^^^

    Symbol Table Hashing:

    vvv QUOTE vvv
    Dictionary
    ----------

    The remaining blocks in the library compose the dictionary. The number
    of blocks in the dictionary is given in the library header. Note that
    there should always be a prime number of blocks in the dictionary.

    The dictionary is a hashed index to the library. Public symbols are
    essentially hashed twice, though in fact, both hash indexes are
    produced simultaneously. The first hash index, the block index, is
    used to determine a block within the dictionary in which to place the
    symbol. The second hash index, the bucket index, is used to choose a
    bucket within the block for the symbol. Blocks always have 37 buckets;
    they are the first 37 bytes of each block. If a bucket is full, it
    contains a nonzero value that points to the text of the symbol. To
    actually find the symbol, take the bucket value, multiply it by two,
    and use the resulting number as a byte offset from the beginning of
    the block.

    Collisions (that is, two or more distinct symbols hashing to the same
    block and bucket in the dictionary) are resolved by a technique known
    as linear open addressing. At the same time the hash indexes are
    produced, two hash deltas are also produced. If a symbol collides with
    a symbol already installed in the dictionary, the librarian attempts
    to find an empty bucket for it by adding the bucket delta to the
    bucket index and using the result mod 37 as a new bucket index. If
    this new bucket index points to a bucket that is empty, the librarian
    will install the symbol in that bucket. If the bucket is not empty,
    the delta is applied repeatedly until an empty bucket is found or
    until it is determined that there are no empty buckets on the block.
    If the latter is the case, the block delta is added to the block
    index, and the result mod the number of blocks in the dictionary is
    used as a new block index. With the new block index and the original
    bucket index, the sequence is repeated until an empty bucket on some
    block is found.

    The number of blocks and the number of buckets are prime so that no
    matter what values of hash indexes and deltas are produced for a
    symbol, in the worst case, all possible block-bucket combinations will
    be tried. Once a free block-bucket pair has been found for a symbol,
    the pair and information concerning its place of definition must be
    installed. Since a bucket is a single byte pointing into a 512-byte
    block, the bucket can give at best a word offset within that block.
    Thus, symbol entries within a dictionary must start on word
    boundaries. Since bytes 0 through 36 of each dictionary block make up
    the hash table, the first symbol entry will begin at byte 38
    (decimal).

    ^^^ UNQUOTE ^^^

    .... but I would not be surprised if in reading the symbol table, it
    should be possible to read it and then sort it yourself.
     
    Bruce McFarling, Dec 25, 2007
    #19
  20. Nickolai Leschov

    cr88192 Guest

    "Bruce McFarling" <> wrote in message
    news:...
    > On Dec 25, 3:20 am, "cr88192" <> wrote:
    >
    >> also possible:
    >> at least a few DOS-era compilers had mechanisms for creating DLL-like
    >> libraries. this may be a possibility, but one will need to investigate
    >> this.

    >
    >> but, assuming dynamic linking:
    >> now, the big fun in this case, is not the loading/relocation, but the
    >> MZ-EXE
    >> file format.
    >> for example, the format lacks a symbol table (I think, but I have hardly
    >> really investigated the MZ-EXE format).

    >
    > The .LIB format is a collection of relocatable object modules, which
    > have a hashed symbol table and a name header in each module.
    >


    yes, you have the symbol table, for the LIB.
    but, here is what you don't have:
    the symbol table for the *EXE*.

    as such, you need the table for the EXE, to be able to properly link the LIB
    into the program.

    after all, what if the LIB should do something, like, say, call 'malloc'...
    LIBs are very rarely completely self-contained.


    so, that is why I didn't mention LIB here, after all, we allready have all
    the info we need to link this, we just don't know anything about what it is
    being linked to (the EXE), and we need this info...

    now, in PE-COFF (Windows), and ELF (Linux, ...) we have symbol tables by
    default (unless the developer strips them).

    but, MZ-EXE, does not. it has relocs, but no symbol table, and that is a
    problem (this is why we need either debugger info, or the linker to embed a
    symbol table in some possibly non-standard way). additionally, the symbol
    table could be a seperate file (the app reads this file, and figures how it
    maps to the loaded program, and uses this to link the lib to the program).

    but, here is another problem:
    linkers often like ommiting unused object files and libraries (no links to
    it exist, so it is left out of the executable image). this is a problem if
    the lib being loaded needs one of these such symbols. one then needs access
    to any libraries used in linking the app, so that potentially any unresolved
    symbols can be dealt with by also linking in the needed objects (one may
    need a customized linker, that hopefully does not link in multiple versions
    of the same object file, but otherwise includes all relevant objects).

    another possibility, is if one has a list of functions/variables that they
    feel need be included, they can include dummy references to all these
    functions/variables (I have done his before, when I was faced with
    similar...).

    now, combined with something like DOS, and the problem is harder (DOS has a
    cramped memory space, and it is debatable if sufficient space exists to link
    in the needed libraries...).


    <snip>
     
    cr88192, Dec 27, 2007
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    6
    Views:
    838
    red floyd
    May 10, 2005
  2. Replies:
    2
    Views:
    6,511
    Maett
    May 16, 2005
  3. iceColdFire

    Static library Vs. Dynamic library

    iceColdFire, May 17, 2005, in forum: C++
    Replies:
    3
    Views:
    17,052
  4. tropos
    Replies:
    3
    Views:
    464
  5. Even

    static library and dynamic library

    Even, Oct 18, 2007, in forum: C Programming
    Replies:
    6
    Views:
    649
    Mark McIntyre
    Oct 20, 2007
Loading...

Share This Page