How to include an external (binary) file into a java program at compile (!) time ???

Discussion in 'Java' started by Tobias Merler, Sep 30, 2004.

  1. I have a java class and a second file which contains some binary data (assume a zip archive or a gif picture).

    At the moment when I compile the class I would like to include this file into a variable of this class.
    I think of a variable declaration like:

    byte[] mybinaryfile = <include dummy.zip>

    How can I implement this idea?
    Remember again: I don't want to load this file at runtime but this file should be
    internal part of the one and only java class.

    Tobias
     
    Tobias Merler, Sep 30, 2004
    #1
    1. Advertising

  2. Tobias Merler

    Dave Glasser Guest

    (Tobias Merler) wrote on Fri, 1 Oct 2004 00:14:54 +0200
    in comp.lang.java.programmer:

    >I have a java class and a second file which contains some binary data (assume a zip archive or a gif picture).
    >
    >At the moment when I compile the class I would like to include this file into a variable of this class.
    >I think of a variable declaration like:
    >
    >byte[] mybinaryfile = <include dummy.zip>
    >
    >How can I implement this idea?
    >Remember again: I don't want to load this file at runtime but this file should be
    >internal part of the one and only java class.



    The only way I can think of is to declare the file contents as an
    array of bytes right in your Java class file, like this:

    byte[] mybinary file = {0x65, 0x66, ... };

    You could easily write a Java program (or Perl script or whatever)
    that would read the binary file and generate the snippet of code for
    that declaration.


    --
    Check out QueryForm, a free, open source, Java/Swing-based
    front end for relational databases.

    http://qform.sourceforge.net

    If you're a musician, check out RPitch Relative Pitch
    Ear Training Software.

    http://rpitch.sourceforge.net
     
    Dave Glasser, Oct 1, 2004
    #2
    1. Advertising

  3. Tobias Merler

    Jacob Guest

    Re: How to include an external (binary) file into a java programat compile (!) time ???

    Tobias Merler wrote:

    > I have a java class and a second file which contains some binary data (assume a zip archive or a gif picture).
    >
    > At the moment when I compile the class I would like to include this file into a variable of this class.
    > I think of a variable declaration like:
    >
    > byte[] mybinaryfile = <include dummy.zip>
    >
    > How can I implement this idea?
    > Remember again: I don't want to load this file at runtime but this file should be
    > internal part of the one and only java class.


    There is really no difference in what you are requesting (textually
    import the content of the file into the .java file) and actually
    load the file at runtime, other than the latter being more elegant
    and better supported by the tools you have available.

    If you insist on the former approach you will need to do a
    preprocessing step (perl, m4, etc.) as Dave G. suggests.

    Otherwise, put your binary file in the classpath (i.e. along
    with your class files) and refer to it from you java source like this:

    String fileName = "myfile.zip";
    String packageName = getClass().getPackage().getName();
    String packageLocation = packageName.replace ('.', '/');
    String filePath = "/" + packageLocation + "/" + fileName;

    InputStream stream = getClass().getResourceAsStream (filePath);

    At this point you have a handle to your (yet unloaded) file.
    You can then load it into a byte[] using the stream.read() methods.

    Remember: By this approach the file is *not* accessed through the
    client file system (which I guess is your concern), but from within
    the application deployment unit.
     
    Jacob, Oct 1, 2004
    #3
  4. Dave Glasser <> writes:

    > byte[] mybinary file = {0x65, 0x66, ... };


    This is immensely inefficient sompared to reading it as a resource
    from the classpath. It's the same as

    byte[] mybinaryFile = new byte[...];
    mybinaryFile[0] = 0x65;
    mybinaryFile[1] = 0x66;
    ....
     
    Tor Iver Wilhelmsen, Oct 1, 2004
    #4
  5. Re: How to include an external (binary) file into a java programat compile (!) time ???

    Tor Iver Wilhelmsen wrote:

    > Dave Glasser <> writes:
    >
    >
    >>byte[] mybinary file = {0x65, 0x66, ... };

    >
    >
    > This is immensely inefficient sompared to reading it as a resource
    > from the classpath.


    Most importantly, it will only work for pretty small files because of the 64k
    bytecode limitation on method size.
     
    Michael Borgwardt, Oct 1, 2004
    #5
  6. You could download Jatha, which is a preprocessor that generates plain
    JAVA sources from JAVA sources with macros.
    You can find it in here: http://www.cybercom.net/~kimbly/jatha/

    The idea is to transform a JAVA file like the follwing (note the
    ..jatha extension) into a plain JAVA file without any macro.

    File IncludeTest.java.jatha:
    ----------------------------------------------------------------------
    public class IncludeTest {
    private static int[] mybinaryfile = @INCLUDE_STREAM(8, dummy.zip);

    public static void main(String[] args) {
    for (int i = 0; i < mybinaryfile.length; i++) {
    System.out.print(mybinaryfile + " ");
    if ((i % 20) == 0)
    System.out.println();
    }
    }
    }
    ----------------------------------------------------------------------

    In this file there is a macro I created: INCLUDE_STREAM, which is a
    custom JAVA class
    based on the Jatha library, and which does all the work on reading the
    provided binary file
    to write its content appropriately in the generated JAVA source:

    File macros/INCLUDE_STREAM.java:
    ----------------------------------------------------------------------
    package macros;

    import java.io.*;
    import jatha.Macro;
    import jatha.Expander;

    public class INCLUDE_STREAM extends Macro {
    static final String LS = System.getProperty("line.separator");

    public void expand(String[] args, Writer out, Expander expander) {
    if (args.length != 2)
    expander.error("INCLUDE_STREAM macro takes two arguments:
    startColumn and filename.");

    String startColumn = args[0];
    String fileName = args[1];
    BufferedInputStream bis = null;

    try {
    int col = Integer.parseInt(startColumn);
    byte[] buffer = new byte[20];
    int len;

    bis = new BufferedInputStream(new
    FileInputStream(fileName));
    out.write("{" + LS);

    while ((len = bis.read(buffer)) != -1) {
    tab(out, col);

    for (int i = 0; i < len; i++)
    out.write(format(buffer) + ", ");

    out.write(LS);
    }

    tab(out, col);
    out.write("}");
    }
    catch (NumberFormatException e) {
    expander.error("The startColumn element (" + startColumn +
    ") is not an int.");
    }
    catch (IOException e) {
    expander.error("IO error while reading file '" + fileName
    + "': " + e.getMessage());
    }
    finally {
    try { bis.close(); }
    catch (Exception e) {}
    }
    }

    void tab(Writer out, int col) throws IOException {
    for (int i = 0; i < col; i++)
    out.write(" ");
    }

    String format(int n) {
    String nStr = Integer.toHexString(n & 0xff);

    if (nStr.length() == 1)
    nStr = "0" + nStr;

    return "0x" + nStr;
    }
    }
    ----------------------------------------------------------------------

    You simply build this macro JAVA class,
    then execute Jatha from the folder where the .jatha file is located:

    > java jatha.Main


    And this will convert IncludeTest.java.jatha into IncludeTest.java:

    File IncludeTest.java:
    ----------------------------------------------------------------------
    public class IncludeTest {
    private static int[] mybinaryfile = {
    0x50, 0x4b, 0x03, 0x04, 0x14, 0x00, 0x00, 0x00, 0x08, 0x00,
    0x78, 0x1d, 0x41, 0x31, 0xb0, 0x9a, 0xc9, 0xbb, 0x27, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x64, 0x75, 0x6d, 0x6d, 0x79, 0x2e,
    0x74, 0x78, 0x74, 0x50, 0x4b, 0x05, 0x06, 0x00, 0x00, 0x00,
    0x00, 0x01, 0x00, 0x01, 0x00, 0x37, 0x00, 0x00, 0x00, 0x4e,
    0x00, 0x00, 0x00, 0x00, 0x00,
    };

    public static void main(String[] args) {
    for (int i = 0; i < mybinaryfile.length; i++) {
    System.out.print(mybinaryfile + " ");
    if ((i % 20) == 0)
    System.out.println();
    }
    }
    }
    ----------------------------------------------------------------------

    Cheers
     
    Jean-Francois Briere, Oct 1, 2004
    #6
  7. Tobias Merler

    Jacob Guest

    Re: How to include an external (binary) file into a java programat compile (!) time ???

    Jean-Francois Briere wrote:

    > You could download Jatha, which is a preprocessor that generates plain
    > JAVA sources from JAVA sources with macros.


    But why?

    Compared to accessing this runtime from the deployment unit
    (.jar say) this approach seems unnecessary complex.
    This goes for the OP problem in particular (as it contains
    binary data), but also in general.
     
    Jacob, Oct 1, 2004
    #7
  8. Tobias Merler

    Dave Glasser Guest

    Tor Iver Wilhelmsen <> wrote on 01 Oct
    2004 09:04:20 +0200 in comp.lang.java.programmer:

    >Dave Glasser <> writes:
    >
    >> byte[] mybinary file = {0x65, 0x66, ... };

    >
    >This is immensely inefficient sompared to reading it as a resource
    >from the classpath. It's the same as
    >
    >byte[] mybinaryFile = new byte[...];
    >mybinaryFile[0] = 0x65;
    >mybinaryFile[1] = 0x66;


    A couple of points: First, I'm well aware of how to use ClassLoader's
    getResource() or getResourceAsStream() methods, but I decided not to
    question the OP's reasons when he stated:

    > Remember again: I don't want to load this file at
    > runtime but this file should be internal part of the
    > one and only java class.


    and instead give him an answer that, AFAICT, satisfied his
    requirement.

    Second, although I wouldn't bet money on it, I don't think you're
    correct when you say:

    >It's the same as
    >
    >byte[] mybinaryFile = new byte[...];
    >mybinaryFile[0] = 0x65;
    >mybinaryFile[1] = 0x66;


    Since my code snippet (which should have said "mybinaryfile" instead
    of "mybinary file") declared the array and initialized it using an
    array literal, I believe the array would be created and initialized
    entirely at compile time. It would do exactly what the OP wanted, it
    just wouldn't do it in the syntactically clean and simple way he was
    looking for.

    Third, even if you're correct that my code would be equivalent to:

    >byte[] mybinaryFile = new byte[...];
    >mybinaryFile[0] = 0x65;
    >mybinaryFile[1] = 0x66;


    it wouldn't be all that expensive compared to reading the same number
    of bytes into the same array from a file; in fact it might be less
    expensive. Even though populating it from a file may not assign each
    array element one at a time (although it may), you wouldn't have all
    the buffering and the loading of extra java.io classes and the disk
    I/O going on under the covers, using up resources and CPU cycles.


    --
    Check out QueryForm, a free, open source, Java/Swing-based
    front end for relational databases.

    http://qform.sourceforge.net

    If you're a musician, check out RPitch Relative Pitch
    Ear Training Software.

    http://rpitch.sourceforge.net
     
    Dave Glasser, Oct 2, 2004
    #8
  9. Tobias Merler

    Dave Glasser Guest

    Michael Borgwardt <> wrote on Fri, 01 Oct
    2004 09:55:43 +0200 in comp.lang.java.programmer:

    >Tor Iver Wilhelmsen wrote:
    >
    >> Dave Glasser <> writes:
    >>
    >>
    >>>byte[] mybinary file = {0x65, 0x66, ... };

    >>
    >>
    >> This is immensely inefficient sompared to reading it as a resource
    >> from the classpath.

    >
    >Most importantly, it will only work for pretty small files because of the 64k
    >bytecode limitation on method size.


    That's only if the OP decided to declare it as a local variable,
    rather than an instance or static variable.


    --
    Check out QueryForm, a free, open source, Java/Swing-based
    front end for relational databases.

    http://qform.sourceforge.net

    If you're a musician, check out RPitch Relative Pitch
    Ear Training Software.

    http://rpitch.sourceforge.net
     
    Dave Glasser, Oct 2, 2004
    #9
  10. Tobias Merler

    Carl Howells Guest

    Re: How to include an external (binary) file into a java programat compile (!) time ???

    Dave Glasser wrote:
    >
    > That's only if the OP decided to declare it as a local variable,
    > rather than an instance or static variable.


    Nope. And that's because you're wrong about the other point. Decompile
    a .class file sometime using javap -c, and look at how array constants
    are initialized. It's with a series of assignments. Each of those
    assignments will be a few bytes of code, too... So you're actually
    looking at significantly under 64k as the maximum size for a byte array
    declared anywhere in java code.
     
    Carl Howells, Oct 2, 2004
    #10
  11. Tobias Merler

    Carl Howells Guest

    Re: How to include an external (binary) file into a java programat compile (!) time ???

    Carl Howells wrote:
    > Dave Glasser wrote:
    >
    >>
    >> That's only if the OP decided to declare it as a local variable,
    >> rather than an instance or static variable.

    >
    >
    > Nope. And that's because you're wrong about the other point. Decompile
    > a .class file sometime using javap -c, and look at how array constants
    > are initialized. It's with a series of assignments. Each of those
    > assignments will be a few bytes of code, too... So you're actually
    > looking at significantly under 64k as the maximum size for a byte array
    > declared anywhere in java code.


    Ugh. I hate speaking unclearly. ... the maximum size for a byte array
    declared *using an array literal* anywhere in java code.
     
    Carl Howells, Oct 2, 2004
    #11
  12. Tobias Merler

    Dave Glasser Guest

    Carl Howells <> wrote on Fri, 01 Oct 2004 16:21:42
    -0700 in comp.lang.java.programmer:

    >Dave Glasser wrote:
    >>
    >> That's only if the OP decided to declare it as a local variable,
    >> rather than an instance or static variable.

    >
    >Nope. And that's because you're wrong about the other point. Decompile
    >a .class file sometime using javap -c, and look at how array constants
    >are initialized. It's with a series of assignments.


    I did, and you're correct. Live and learn.


    --
    Check out QueryForm, a free, open source, Java/Swing-based
    front end for relational databases.

    http://qform.sourceforge.net

    If you're a musician, check out RPitch Relative Pitch
    Ear Training Software.

    http://rpitch.sourceforge.net
     
    Dave Glasser, Oct 2, 2004
    #12
  13. Dave Glasser <> writes:

    > I did, and you're correct. Live and learn.


    You probably made the assumption that class files have a data segment
    like binary code formats (COFF/PE), but they don't - except for string
    literals, which end up in a "string pool".
     
    Tor Iver Wilhelmsen, Oct 2, 2004
    #13
  14. By the way guys, those will eventually produce an error:

    byte[] mybinaryfile = {0x65, 0x66, ... };

    -or-

    byte[] mybinaryFile = new byte[...];
    mybinaryFile[0] = 0x65;
    mybinaryFile[1] = 0x66;
    ....

    Because in any binary file, you will have some bytes from 0x80 to 0xFF.

    The correct code is:

    int[] mybinaryfile = {0x65, 0x66, ... };

    -or-

    int[] mybinaryFile = new int[...];
    mybinaryFile[0] = 0x65;
    mybinaryFile[1] = 0x66;
    ....
     
    Jean-Francois Briere, Oct 2, 2004
    #14
  15. Tobias Merler

    Yakov Guest

    > How can I implement this idea?
    > Remember again: I don't want to load this file at runtime but this file should be
    > internal part of the one and only java class.
    >
    > Tobias


    Since it was not explained why the file must be a part of the class,
    it's hard to give a proper advice, but here's one of the solutions:

    Write a program that reads this file into a variable and then
    serializes the class into a file using writeObject(this). Now the
    serialized version of this class includes the file. Now you any
    program can recreate the instance of this class (that includes the
    file) using readObject().

    Regards,
    Yakov Fain
     
    Yakov, Oct 3, 2004
    #15
  16. Tobias Merler

    Ann Guest

    but OP wants to not have any code at runtime to do it



    "Yakov" <> wrote in message
    news:...
    > > How can I implement this idea?
    > > Remember again: I don't want to load this file at runtime but this file

    should be
    > > internal part of the one and only java class.
    > >
    > > Tobias

    >
    > Since it was not explained why the file must be a part of the class,
    > it's hard to give a proper advice, but here's one of the solutions:
    >
    > Write a program that reads this file into a variable and then
    > serializes the class into a file using writeObject(this). Now the
    > serialized version of this class includes the file. Now you any
    > program can recreate the instance of this class (that includes the
    > file) using readObject().
    >
    > Regards,
    > Yakov Fain
     
    Ann, Oct 3, 2004
    #16
  17. Tobias Merler

    Yakov Guest

    "Ann" <> wrote in message news:<vHL7d.310134$Fg5.58253@attbi_s53>...
    > but OP wants to not have any code at runtime to do it


    The method that reads this file will be called only once during the
    very first run only and never again

    Regards,
    Yakov Fain
     
    Yakov, Oct 3, 2004
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nagaraj
    Replies:
    1
    Views:
    913
    Lionel B
    Mar 1, 2007
  2. timjowers
    Replies:
    7
    Views:
    4,417
    heyjude
    Feb 2, 2011
  3. Andreas Bogenberger
    Replies:
    3
    Views:
    1,003
    Andreas Bogenberger
    Feb 22, 2008
  4. Carter
    Replies:
    2
    Views:
    527
    Carter
    Mar 4, 2009
  5. Alythh

    include external perl program

    Alythh, Mar 8, 2005, in forum: Perl Misc
    Replies:
    5
    Views:
    143
    Tad McClellan
    Mar 9, 2005
Loading...

Share This Page