Multiple files in one

Discussion in 'C++' started by fryme, Sep 12, 2011.

  1. fryme

    fryme Guest

    I'm trying to create my own file format. I want to store image file
    and some text description in that file.
    File format will be something like that:
    image_file_size
    image_data
    desctiption_file_size
    description_data

    But without '\n' symbols.
    For that purpose i'm using std::ios::binary. Here is some code,
    describes that process (it is sketch, not last variant):
    Writing my file.

    long long image_length, desctiption_length;

    std::fstream m_out(output_file_path, std::ios::eek:ut |
    std::ios::binary);
    std::ifstream input_image(m_image_file_path.toUtf8().data());

    input_image.seekg(0, std::ios::end);
    image_length = input_image.tellg();
    input_image.seekg(0, std::ios::beg);

    // writing image length to output file
    m_out.write( (const char *)&image_length, sizeof(long long) );

    char *buffer = new char[image_length];
    input.read(buffer, image_length);

    // writing image to file
    m_out.write(buffer, image_length);

    // writing description file the same way
    // ...

    Reading my file.

    std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );

    long long xml_length, image_length;

    m_in.seekg(0, std::ios::beg);
    m_in.read((char *)&image_length, sizeof(long long));
    m_in.seekg(sizeof(long long));

    char *buffer = new char[image_length];
    m_in.read(buffer, image_length );

    std::fstream fs("E:\\Temp\\out.jpg");
    fs.write(buffer, image_length);

    And now image (E:\Temp\out.jpg) is broken. I'm watching it in hex
    editor and there are some extra bits.

    Can somebody help me and tell what i'm doing wrong?


    --
    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    fryme, Sep 12, 2011
    #1
    1. Advertising

  2. fryme

    Asger-P Guest

    Hi fryme

    Arent You missing this on the infile ?
    > std::ios::binary);



    Best regards
    Asger-P
     
    Asger-P, Sep 13, 2011
    #2
    1. Advertising

  3. fryme wrote:
    > [...]
    > For that purpose i'm using std::ios::binary. Here is some code,
    > describes that process (it is sketch, not last variant):


    You expect us to find bugs in a "sketch"?

    > [...]
    > std::fstream m_out(output_file_path, std::ios::eek:ut |
    > std::ios::binary);
    > std::ifstream input_image(m_image_file_path.toUtf8().data());

    ^^^^^^^^^^^^^^^
    All right, you are using Qt.

    > input_image.seekg(0, std::ios::end);
    > image_length = input_image.tellg();
    > input_image.seekg(0, std::ios::beg);
    >
    > // writing image length to output file
    > m_out.write( (const char *)&image_length, sizeof(long long) );
    >
    > char *buffer = new char[image_length];
    > input.read(buffer, image_length);

    ^^^^^
    I guess you mean "input_image", not "input".

    > [...]
    > std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );


    There is no "std::ios::binary" here. Is this just a detail missing
    from the "sketch"?

    > m_in.seekg(0, std::ios::beg);
    > m_in.read((char *)&image_length, sizeof(long long));
    > m_in.seekg(sizeof(long long));


    As far as I know, the second argument of "seekg" has no default value.

    > [...]
    > std::fstream fs("E:\\Temp\\out.jpg");


    Again no "std::ios::binary" here.

    Please, create working example code.

    --
    host -t mx moderators.isc.org


    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    Alexander Bartolich, Sep 13, 2011
    #3
  4. fryme

    Paavo Helde Guest

    fryme <> wrote in news:a51089c8-6be0-4fa7-9237-
    :

    > I'm trying to create my own file format. I want to store image file
    > and some text description in that file.


    Strongly discouraged as this file can be read only by your own software,
    not a good idea. What's wrong with writing two files, or writing the
    description in the EXIF section of jpeg or whatever?

    > File format will be something like that:
    > image_file_size
    > image_data
    > desctiption_file_size
    > description_data
    >
    > But without '\n' symbols.
    > For that purpose i'm using std::ios::binary. Here is some code,
    > describes that process (it is sketch, not last variant):
    > Writing my file.
    >
    > long long image_length, desctiption_length;
    >
    > std::fstream m_out(output_file_path, std::ios::eek:ut |
    > std::ios::binary);
    > std::ifstream input_image(m_image_file_path.toUtf8().data());


    binary flag missing

    >
    > input_image.seekg(0, std::ios::end);
    > image_length = input_image.tellg();
    > input_image.seekg(0, std::ios::beg);
    >
    > // writing image length to output file
    > m_out.write( (const char *)&image_length, sizeof(long long) );


    sizeof(long long) is platform-dependent, you should be using something
    more fixed like std::uint64_t, for example. Even then the endianness
    would be platform-dependent.

    >
    > char *buffer = new char[image_length];
    > input.read(buffer, image_length);
    >
    > // writing image to file
    > m_out.write(buffer, image_length);
    >
    > // writing description file the same way
    > // ...
    >
    > Reading my file.
    >
    > std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );


    binary flag missing

    >
    > long long xml_length, image_length;
    >
    > m_in.seekg(0, std::ios::beg);
    > m_in.read((char *)&image_length, sizeof(long long));
    > m_in.seekg(sizeof(long long));
    >
    > char *buffer = new char[image_length];
    > m_in.read(buffer, image_length );
    >
    > std::fstream fs("E:\\Temp\\out.jpg");


    binary flag missing

    > fs.write(buffer, image_length);
    >
    > And now image (E:\Temp\out.jpg) is broken. I'm watching it in hex
    > editor and there are some extra bits.
    >
    > Can somebody help me and tell what i'm doing wrong?


    Using C++ streams does not help you here in any way if your file is
    binary. Also, AFAIK the stream read()/write() can change the data even if
    the stream is in the binary mode, the correct way would be to use the
    stream buffer read/write functionality IIRC. I would personally not
    bother with this and used fopen() and friends instead (and not forgetting
    binary flags!).

    hth
    Paavo


    --
    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    Paavo Helde, Sep 13, 2011
    #4
  5. fryme

    red floyd Guest

    On 9/12/2011 11:44 AM, fryme wrote:
    > [redacted]
    > long long image_length, desctiption_length;
    >
    > std::fstream m_out(output_file_path, std::ios::eek:ut |
    > std::ios::binary);
    > std::ifstream input_image(m_image_file_path.toUtf8().data());
    >
    > [redacted]
    > // ...
    >
    > Reading my file.
    >
    > std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );

    std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in |
    std::ios::binary);


    --
    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    red floyd, Sep 13, 2011
    #5
  6. Am 12.09.2011 20:44, schrieb fryme:
    > I'm trying to create my own file format. I want to store image file
    > and some text description in that file.


    [..]

    > For that purpose i'm using std::ios::binary. Here is some code,
    > describes that process (it is sketch, not last variant):
    > Writing my file.
    >
    > long long image_length, desctiption_length;
    >
    > std::fstream m_out(output_file_path, std::ios::eek:ut |
    > std::ios::binary);
    > std::ifstream input_image(m_image_file_path.toUtf8().data());


    [..]

    > Reading my file.
    >
    > std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );


    It seems to me that both input_image and m_in need to be opened as
    binary streams as well.

    HTH & Greetings from Bremen,

    Daniel Krügler


    --
    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    Daniel Krügler, Sep 13, 2011
    #6
  7. fryme

    Jorgen Grahn Guest

    ["Followup-To:" header set to comp.lang.c++.]
    On Mon, 2011-09-12, fryme wrote:
    > I'm trying to create my own file format. I want to store image file
    > and some text description in that file.


    Probably unwise, but that's not relevant to the question.

    > File format will be something like that:
    > image_file_size
    > image_data
    > desctiption_file_size
    > description_data
    >
    > But without '\n' symbols.
    > For that purpose i'm using std::ios::binary. Here is some code,
    > describes that process (it is sketch, not last variant):
    > Writing my file.
    >
    > long long image_length, desctiption_length;
    >
    > std::fstream m_out(output_file_path, std::ios::eek:ut |
    > std::ios::binary);
    > std::ifstream input_image(m_image_file_path.toUtf8().data());
    >
    > input_image.seekg(0, std::ios::end);
    > image_length = input_image.tellg();
    > input_image.seekg(0, std::ios::beg);
    >
    > // writing image length to output file
    > m_out.write( (const char *)&image_length, sizeof(long long) );


    Ugh! Please don't -- this introduces endianness bugs in your files.
    Instead specify in your file format definition exactly *what* the
    "image_file_size" field means and how its formatted, and the code will
    follow.

    For example:

    image_file_size: 6 octets, MSB first, containing the length (in
    octets) of the immediately following image file data. May not be
    zero.

    I didn't read the rest. Sorry.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .


    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    Jorgen Grahn, Sep 13, 2011
    #7
  8. fryme wrote:
    > I'm trying to create my own file format. I want to store image file
    > and some text description in that file.


    Take a look at the approach that many recent programs take, i.e. using a
    simple archive of files. See for example
    http://tanguy.ortolo.eu/blog/article24/repack-zip-container, where the
    approach is described a bit. Note that this allows use of standard tools for
    manipulation.


    > i'm using std::ios::binary.


    This flag does probably not do what you expect it to do. In order to output
    preformatted bytes or read bytes in order to parse them yourself, it is
    vital though.


    > std::fstream m_out(output_file_path, std::ios::eek:ut |
    > std::ios::binary);


    Use ofstream instead, then you don't have to specify the "ios::eek:ut" and you
    get some checking that you don't accidentally do input from the compiler.


    > char *buffer = new char[image_length];


    If you find yourself using the vector new, you are probably doing something
    very advanced or a beginner's mistake. In this case, I suspect the latter,
    and the remedy is to use a std::vector. Further looking at your code, you
    continue the mistake and fail to free the allocated data.


    > std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );
    >
    > long long xml_length, image_length;
    >
    > m_in.seekg(0, std::ios::beg);


    Useless, you are at the beginning of the file.

    > m_in.read((char *)&image_length, sizeof(long long));


    Two things:
    1. Use "sizeof image_length" to make clear that it is that variable's size
    that you want to read. Of course, this is non-portable, as others already
    pointed out.
    2. Use of C-style casts is dangerous, as they suppress any warning. In this
    case though, there is no real alternative except two static casts, because
    C++ iostreams are for writing texts and not for writing preformatted data.
    They can be used for that, but typically that results in awkward code like
    the above.

    > m_in.seekg(sizeof(long long));


    Also redundant, since reading advances the read position automatically.


    Good luck!

    Uli

    --
    Domino Laser GmbH
    Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932


    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    Ulrich Eckhardt, Sep 13, 2011
    #8
  9. fryme

    Daniel James Guest

    In article
    <>,
    Fryme wrote:
    > I'm trying to create my own file format.


    Others have already questioned the wisdom of such an approach.

    Rather than inventing a whole new file format that is peculiar to your
    application, why not use a standard container format (such as the
    zipfile format) and write your image file and a descriptive text file
    to that?

    Not only will the resultant file be in a recognizably standard format,
    but the process of reading and writing the file can be simplified,
    relative to your home-grown approach, by the use of one of many freely
    available libraries.

    --
    Regards,
    Daniel.
    [Lest there be any confusion I am NOT the Boost maintainer of the same
    name]


    [ See http://www.gotw.ca/resources/clcm.htm for info about ]
    [ comp.lang.c++.moderated. First time posters: Do this! ]
     
    Daniel James, Sep 13, 2011
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. D. Shane Fowlkes
    Replies:
    3
    Views:
    631
    S. Justin Gengo
    Feb 24, 2004
  2. news.frontiernet.net
    Replies:
    6
    Views:
    1,126
    news.frontiernet.net
    Apr 16, 2004
  3. loveNUNO
    Replies:
    2
    Views:
    924
    loveNUNO
    Nov 20, 2003
  4. Replies:
    4
    Views:
    957
    M.E.Farmer
    Feb 13, 2005
  5. Merciadri Luca
    Replies:
    4
    Views:
    818
Loading...

Share This Page