binary data conversion

Discussion in 'C Programming' started by Drake, Apr 11, 2004.

  1. Drake

    Drake Guest

    Well, I'm stuck in legacy land and I need a helping hand.

    We're trying to give some modern value-added functionality to
    a circa-1985 fortran proggie.

    The program produces a binary file, by itself no problem...
    each record needs to be converted into std::vector<mystruct>
    I'm having a helluva time with the binary-->(pod)datatype conversion.

    Although I've referred to C++ containers, iostreams
    are not used... FILE* only. Insert/extract operators are not an option.

    Here's 3 records (broken into lines for easy reading)...

    D5840000428F0000140501000B0000000000
    993A000066C70000140501000F0000000000
    C0880000538D0100140501001D0000000000

    Here's (pod) mystruct...

    static struct IDX_RECORD /* length 18 bytes */
    {
    unsigned long respnum; // 4 bytes
    unsigned long datfile_offset; // 4 bytes
    unsigned char disposition; // 1 byte
    unsigned char status; // 1 byte
    unsigned char segment_svd; // 1 byte
    unsigned char not_used; // 1 byte
    unsigned short segment_dupr; // 2 bytes
    unsigned long next_segment_offset; // 4 bytes
    };

    Is this one of those times fscanf IS the "better" (whatever that means) way?

    Assuming the powers have big-time aversion to "scan" functions,
    what's an alternative?

    Thank you for your thoughts,
    Drake
    Drake, Apr 11, 2004
    #1
    1. Advertising

  2. Drake wrote:

    > Well, I'm stuck in legacy land and I need a helping hand.
    >
    > We're trying to give some modern value-added functionality to
    > a circa-1985 fortran proggie.
    >
    > The program produces a binary file, by itself no problem...
    > each record needs to be converted into std::vector<mystruct>


    Bummer. Ask C++ questions in a C++ newsgroup like news:comp.lang.c++
    Martin Ambuhl, Apr 11, 2004
    #2
    1. Advertising

  3. Drake

    Mike Wahler Guest

    "Drake" <> wrote in message
    news:Xns94C7AF6EC57AEnomailcom@207.217.125.204...
    > Well, I'm stuck in legacy land and I need a helping hand.
    >
    > We're trying to give some modern value-added functionality to
    > a circa-1985 fortran proggie.
    >
    > The program produces a binary file, by itself no problem...
    > each record needs to be converted into std::vector<mystruct>
    > I'm having a helluva time with the binary-->(pod)datatype conversion.
    >
    > Although I've referred to C++ containers, iostreams
    > are not used... FILE* only. Insert/extract operators are not an option.
    >
    > Here's 3 records (broken into lines for easy reading)...
    >
    > D5840000428F0000140501000B0000000000
    > 993A000066C70000140501000F0000000000
    > C0880000538D0100140501001D0000000000
    >
    > Here's (pod) mystruct...
    >
    > static struct IDX_RECORD /* length 18 bytes */
    > {
    > unsigned long respnum; // 4 bytes
    > unsigned long datfile_offset; // 4 bytes
    > unsigned char disposition; // 1 byte
    > unsigned char status; // 1 byte
    > unsigned char segment_svd; // 1 byte
    > unsigned char not_used; // 1 byte
    > unsigned short segment_dupr; // 2 bytes
    > unsigned long next_segment_offset; // 4 bytes
    > };


    I'll assume those byte sizes in your comments refer
    to your particular implementation. The language
    doesn't require those sizes (but it does require
    minimum sizes).

    >
    > Is this one of those times fscanf IS the "better" (whatever that means)

    way?
    >
    > Assuming the powers have big-time aversion to "scan" functions,
    > what's an alternative?


    fscanf() is not what you need. It reads textual data.

    Use fread(). And be sure to account for 'endianness'.

    -Mike
    Mike Wahler, Apr 11, 2004
    #3
  4. Drake

    Mac Guest

    On Sun, 11 Apr 2004 00:14:44 +0000, Drake wrote:

    > Well, I'm stuck in legacy land and I need a helping hand.
    >
    > We're trying to give some modern value-added functionality to
    > a circa-1985 fortran proggie.
    >
    > The program produces a binary file, by itself no problem...
    > each record needs to be converted into std::vector<mystruct>
    > I'm having a helluva time with the binary-->(pod)datatype conversion.
    >
    > Although I've referred to C++ containers, iostreams
    > are not used... FILE* only. Insert/extract operators are not an option.
    >
    > Here's 3 records (broken into lines for easy reading)...
    >
    > D5840000428F0000140501000B0000000000
    > 993A000066C70000140501000F0000000000
    > C0880000538D0100140501001D0000000000


    Is this a hex dump of the file, or is the file text... You said it was
    binary earlier.

    >
    > Here's (pod) mystruct...
    >
    > static struct IDX_RECORD /* length 18 bytes */
    > {
    > unsigned long respnum; // 4 bytes
    > unsigned long datfile_offset; // 4 bytes
    > unsigned char disposition; // 1 byte
    > unsigned char status; // 1 byte
    > unsigned char segment_svd; // 1 byte
    > unsigned char not_used; // 1 byte
    > unsigned short segment_dupr; // 2 bytes
    > unsigned long next_segment_offset; // 4 bytes
    > };
    >
    > Is this one of those times fscanf IS the "better" (whatever that means) way?
    >


    Well, if the file is text, I think fscanf() is probably the best way, but
    you can do whatever you want, including reading a character at a time with
    getc() and not calling any other library functions. Another function to
    consider is fgets() (again, assuming a text file.)

    If the file is binary, which is what you said initially, I definitely
    would not use fscanf(). I would use getc() or fread().

    > Assuming the powers have big-time aversion to "scan" functions,
    > what's an alternative?
    >


    What do you mean by "powers?" Is that short for "powers that be," meaning,
    loosely, whoever is in charge?

    > Thank you for your thoughts,
    > Drake



    I think you may want comp.lang.c++.

    If you want a c solution, I suggest you rephrase to make that more clear,
    removing all reference to c++, just to be safe, and repost.

    If you want a c++ solution, post in comp.lang.c++, but read their faq
    and/or lurk first to make sure you don't annoy anyone.

    --Mac
    Mac, Apr 11, 2004
    #4
  5. Drake

    Drake Guest

    Martin Ambuhl <> wrote in news:c5a47k$2m9rj0$3@ID-
    227552.news.uni-berlin.de:

    > Drake wrote:
    >
    >> Well, I'm stuck in legacy land and I need a helping hand.
    >>
    >> We're trying to give some modern value-added functionality to
    >> a circa-1985 fortran proggie.
    >>
    >> The program produces a binary file, by itself no problem...
    >> each record needs to be converted into std::vector<mystruct>

    >
    > Bummer. Ask C++ questions in a C++ newsgroup like news:comp.lang.c++



    A few lines later the post tried and obviously failed in avoiding this
    answer.

    << Although I've referred to C++ containers, iostreams
    << are not used... FILE* only. Insert/extract operators are not an
    << option.

    Anyway std::vector isn't the problem. It's the conversion.


    Drake
    Drake, Apr 11, 2004
    #5
  6. "Drake" <x@.y.net> wrote in message
    news:Ju3ec.4494$...
    > Martin Ambuhl <> wrote in news:c5a47k$2m9rj0$3@ID-
    > 227552.news.uni-berlin.de:
    >
    > > Drake wrote:
    > >
    > >> Well, I'm stuck in legacy land and I need a helping hand.
    > >>
    > >> We're trying to give some modern value-added functionality to
    > >> a circa-1985 fortran proggie.
    > >>
    > >> The program produces a binary file, by itself no problem...
    > >> each record needs to be converted into std::vector<mystruct>

    > >
    > > Bummer. Ask C++ questions in a C++ newsgroup like news:comp.lang.c++

    >
    >
    > A few lines later the post tried and obviously failed in avoiding this
    > answer.
    >
    > << Although I've referred to C++ containers, iostreams
    > << are not used... FILE* only. Insert/extract operators are not an
    > << option.
    >
    > Anyway std::vector isn't the problem. It's the conversion.
    >
    >
    > Drake


    Tastefully dispatched. ;-)
    Dilton McGowan II, Apr 11, 2004
    #6
  7. Drake

    Malcolm Guest

    "Drake" <> wrote in message
    >
    > Here's 3 records (broken into lines for easy reading)...
    >
    > D5840000428F0000140501000B0000000000
    > 993A000066C70000140501000F0000000000
    > C0880000538D0100140501001D0000000000
    >
    > Here's (pod) mystruct...
    >
    > static struct IDX_RECORD /* length 18 bytes */
    > {
    > unsigned long respnum; // 4 bytes
    > unsigned long datfile_offset; // 4 bytes
    > unsigned char disposition; // 1 byte
    > unsigned char status; // 1 byte
    > unsigned char segment_svd; // 1 byte
    > unsigned char not_used; // 1 byte
    > unsigned short segment_dupr; // 2 bytes
    > unsigned long next_segment_offset; // 4 bytes
    > };
    >
    > Assuming the powers have big-time aversion to "scan" functions,
    > what's an alternative?
    >

    As someone else has pointed out, you don't say whether your example is a
    hexdump or a text file. fscanf() is useful only for text files, but of
    limited utlity if numerics are not separated by delimiters.

    What you need are the functions

    unsigned long read32us(FILE *fp);
    unsigned short read16us(FILE *fp);

    If you want to use C++ you can be ugly and replace the FILE * with a stream,
    or be object-oriented and derive a class from iostream with the functions
    added.
    In either case, build the read32() function on top of fgetc() / character
    extraction.
    If you are using C++, throw exceptions in the case of premature EOF.
    Malcolm, Apr 11, 2004
    #7
  8. Drake wrote:

    > Well, I'm stuck in legacy land and I need a helping hand.
    >
    > We're trying to give some modern value-added functionality to
    > a circa-1985 fortran proggie.
    >
    > The program produces a binary file, by itself no problem...
    > each record needs to be converted into std::vector<mystruct>
    > I'm having a helluva time with the binary-->(pod)datatype conversion.
    >
    > Although I've referred to C++ containers, iostreams
    > are not used... FILE* only. Insert/extract operators are not an option.
    >
    > Here's 3 records (broken into lines for easy reading)...
    >
    > D5840000428F0000140501000B0000000000
    > 993A000066C70000140501000F0000000000
    > C0880000538D0100140501001D0000000000
    >
    > Here's (pod) mystruct...
    >
    > static struct IDX_RECORD /* length 18 bytes */
    > {
    > unsigned long respnum; // 4 bytes
    > unsigned long datfile_offset; // 4 bytes
    > unsigned char disposition; // 1 byte
    > unsigned char status; // 1 byte
    > unsigned char segment_svd; // 1 byte
    > unsigned char not_used; // 1 byte
    > unsigned short segment_dupr; // 2 bytes
    > unsigned long next_segment_offset; // 4 bytes
    > };
    >
    > Is this one of those times fscanf IS the "better" (whatever that means) way?
    >
    > Assuming the powers have big-time aversion to "scan" functions,
    > what's an alternative?
    >
    > Thank you for your thoughts,
    > Drake


    If your file is ASCII encoded hex file (a.k.a. Hex dump),
    then you may want to read each line using fgets(), then
    fill in your data structure by converting the ASCII values
    into native formats. This is probably the faster method
    than fscanf.

    If the file is actually binary, you may want to use the
    fread function and read a block into a buffer then load
    the fields of your structure from the buffer.

    In no case do you want to load your structure directly
    from the file or memory. The simple rule is that the
    size of a structure may not be equal to the size of its
    members. The compiler is allowed to add "padding" bytes
    between members. This is what kills most programs that
    attempt to map a structure directly to an input stream.


    --
    Thomas Matthews

    C++ newsgroup welcome message:
    http://www.slack.net/~shiva/welcome.txt
    C++ Faq: http://www.parashift.com/c -faq-lite
    C Faq: http://www.eskimo.com/~scs/c-faq/top.html
    alt.comp.lang.learn.c-c++ faq:
    http://www.raos.demon.uk/acllc-c /faq.html
    Other sites:
    http://www.josuttis.com -- C++ STL Library book
    Thomas Matthews, Apr 12, 2004
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Victor Hannak
    Replies:
    0
    Views:
    7,443
    Victor Hannak
    Feb 20, 2004
  2. Alexander Eisenhuth
    Replies:
    5
    Views:
    536
    Bob Gailer
    Jul 25, 2003
  3. Replies:
    2
    Views:
    583
  4. , India
    Replies:
    2
    Views:
    457
    Fraser Ross
    Sep 15, 2009
  5. TRB_NV
    Replies:
    0
    Views:
    111
    TRB_NV
    May 19, 2006
Loading...

Share This Page