New language on JVM?

Discussion in 'Java' started by jamesl, Dec 12, 2010.

  1. jamesl

    jamesl Guest

    jamesl, Dec 12, 2010
    #1
    1. Advertising

  2. jamesl

    Stefan Ram Guest

    BGB <> writes:
    >in my case, I would like (decent) C and C# support...


    The memory model might be a problem with C. C allows to get
    (and sometimes print) the addresses of objects and functions
    and to allocate structures on the heap, to get the offset of
    structure members and so on.

    There is at least one Intel-PC-Emulator (386 based) for the
    JVM: This will allow you to run 386 code which can come from
    a C or C++ compiler on the JVM. An approach without such an
    additional layer might not be possible. Possibly, you also
    could install Microsoft® Windows and then run C# code.

    However, it should be very difficult to access the real
    environment in this case.

    >also maybe dynamic compilation, ...


    We already have this in the JDK (but not in the JRE).

    >I am doing my own implementation of the JVM,


    This should take a lot of time, but are you getting paid
    for it? If not, it is really admirable if your are able to
    manage your life in such a way that there even is enough
    time for such projects!
    Stefan Ram, Dec 12, 2010
    #2
    1. Advertising

  3. jamesl

    Arne Vajhøj Guest

    On 12-12-2010 00:42, BGB wrote:
    > On 12/11/2010 8:18 PM, jamesl wrote:
    >> If you have ever wanted to implement a new language on top of the JVM,
    >> then this series may be of
    >> interest to you: Redline: Smalltalk in 'small talks' ....
    >> http://blog.redline.st/2010/12/12/redline-smalltalk-in-“small-talks”/
    >>
    >> More than just Smalltalk is covered.

    >
    > in my case, I would like (decent) C and C# support...
    > also maybe dynamic compilation, ...


    C# is done:

    http://dev.mainsoft.com/Default.aspx?tabid=130#MS_IL_to_Java_ByteCode_Compilation

    I am skeptical about whether C can run on the JVM as I suspect
    some of the Java safeness features are implemented at the
    JVM level.

    There are plenty of dynamic compilation possibilities
    for Java.

    Arne
    Arne Vajhøj, Dec 12, 2010
    #3
  4. jamesl

    BGB Guest

    On 12/11/2010 10:54 PM, Stefan Ram wrote:
    > BGB<> writes:
    >> in my case, I would like (decent) C and C# support...

    >
    > The memory model might be a problem with C. C allows to get
    > (and sometimes print) the addresses of objects and functions
    > and to allocate structures on the heap, to get the offset of
    > structure members and so on.
    >


    or add pointers, sizeof, offsetof, ..., to the VM...

    Java doesn't need to use them, and possibly something like in .NET can
    be done to deal with their existence:
    untrusted code trying to use these features is immediately rejected.
    nearly everything else then will pretend they don't exist.


    I have already added struct support, and have considered some idle ideas
    as to how to deal with unions (currently involving hacks with structs).

    note: these pointers will be non-boxed "raw pointers".
    currently, some notable restrictions are also placed on them (currently,
    no pointer-based types support virtual methods, for possibly
    obvious-enough reasons, ...).


    the C compiler code may allow some extensions to ease working with
    non-pointer reference types (and possibly extensions to support "opaque
    strings"), hence some amount of code may be written which can gloss over
    whether it is running in native land or tweaked-JVM land (I already do
    this to some extent in my framework for other reasons).

    hence, with pointer-free code it may be possible to produce non-extended
    bytecode (and thus be usable on the standard JVM).

    other extensions may also be used to ease interaction with the Java
    object model, ...

    so, IOW, something along vaguely similar lines to C++/CLI (although
    starting with a C base, and adapting it to JVM-based technology).


    admittedly, the above is a lot of why I am doing my own implementation:
    there is probably little hope that Oracle would at all cooperate with
    something like this (nor am I that inclined to wait for things to cycle
    through their "process").


    note, another reason for this:
    it may help C code also gloss over whether it is running on a 32 or 64
    bit target, which is currently an issue (I am imagining supporting
    common binary code runnable on both Windows and Linux and in both 32 and
    64 bit mode...).



    > There is at least one Intel-PC-Emulator (386 based) for the
    > JVM: This will allow you to run 386 code which can come from
    > a C or C++ compiler on the JVM. An approach without such an
    > additional layer might not be possible. Possibly, you also
    > could install Microsoft® Windows and then run C# code.
    >
    > However, it should be very difficult to access the real
    > environment in this case.
    >


    yes...

    this is exactly the sort of thing I don't want to do.
    AFAIK, many prior "C to JVM" compilers have gone the route of simulating
    byte addressable memory via arrays and similar, but this sucks too hard IMO.


    direct access to the real environment is personally important (among
    other things), and the awkwardness of this has actually been part of why
    I have not used the JVM much in the past. (and my existing C codebase is
    not exactly small...).

    so, the C implementation needs to be all of memory, data, and ABI
    compatible with native C code.


    >> also maybe dynamic compilation, ...

    >
    > We already have this in the JDK (but not in the JRE).
    >


    yeah, because JDK includes javac and JRE does not?...


    >> I am doing my own implementation of the JVM,

    >
    > This should take a lot of time, but are you getting paid
    > for it? If not, it is really admirable if your are able to
    > manage your life in such a way that there even is enough
    > time for such projects!
    >


    well, I am a hobbyist...
    but also a college student (currently going for a CS major), and
    probably college students have a bit more free time than people with a
    job...

    it is being based on JavaME though, as there is little chance of me
    writing an entire JavaSE implementation by myself (and I don't really
    have much need to burn myself out trying to make AWT and Swing and so on
    work...).

    another issue is that there are some newer JVM features (generics, ...)
    for which I don't have much up-to-date information regarding how they
    are implemented (currently, I am not using them).


    also, I have been fiddling around with compiler and VM technologies for
    much of a decade, so I have a bit of a head start. for example, I ended
    up more just writing a "JVM-like skin" for my existing stuff, rather
    than having to start clean (implementing an entire VM framework from the
    ground up).


    similarly, I already have both a C compiler in place (for native code
    though), as well as much of what is needed for a Java and C# compiler,
    and for compiling to JBC (the main part needing work, not yet complete).
    so, mostly the work is in bridging things and making it all work.


    but, all of this is at-best "experimental"...
    BGB, Dec 12, 2010
    #4
  5. jamesl

    BGB Guest

    On 12/12/2010 8:28 AM, Arne Vajhøj wrote:
    > On 12-12-2010 00:42, BGB wrote:
    >> On 12/11/2010 8:18 PM, jamesl wrote:
    >>> If you have ever wanted to implement a new language on top of the JVM,
    >>> then this series may be of
    >>> interest to you: Redline: Smalltalk in 'small talks' ....
    >>> http://blog.redline.st/2010/12/12/redline-smalltalk-in-“small-talks”/
    >>>
    >>>
    >>> More than just Smalltalk is covered.

    >>
    >> in my case, I would like (decent) C and C# support...
    >> also maybe dynamic compilation, ...

    >
    > C# is done:
    >
    > http://dev.mainsoft.com/Default.aspx?tabid=130#MS_IL_to_Java_ByteCode_Compilation
    >


    this is a way to do it...
    admittedly it is a bit crufty IMO, but oh well.


    I am doing a direct compiler (custom written, in C...).
    I will see if I can make it workable within the confines of standard
    bytecode (probably will prevent using 'unsafe' features though, but no
    surprise there).


    > I am skeptical about whether C can run on the JVM as I suspect
    > some of the Java safeness features are implemented at the
    > JVM level.
    >


    yep.
    partly it is VM-level safeness, and partly it is VM-level lack of unsafe
    features (built in pointer and struct support, ...).

    hence, VM extensions, ...

    I don't want yet more dynamic languages, I want the ability to run C and
    similar on the VM, hence part of why I am doing my own implementation
    (and allowing for tweaking and extensions).


    > There are plenty of dynamic compilation possibilities
    > for Java.
    >


    agreed...

    one can produce bytecode and shove it into the classloader, for example...

    if one has JDK they can invoke javac (but, it is a little bit of a hack
    though, and "compile this Java code and load it" is not exactly a
    standard API feature, although yes, it is more standard than throwing
    together ones' own VM to do the job).

    ....

    but, alas...
    BGB, Dec 12, 2010
    #5
  6. jamesl

    Arne Vajhøj Guest

    On 12-12-2010 13:29, BGB wrote:
    > On 12/12/2010 8:28 AM, Arne Vajhøj wrote:
    >> On 12-12-2010 00:42, BGB wrote:
    >>> On 12/11/2010 8:18 PM, jamesl wrote:
    >>>> If you have ever wanted to implement a new language on top of the JVM,
    >>>> then this series may be of
    >>>> interest to you: Redline: Smalltalk in 'small talks' ....
    >>>> http://blog.redline.st/2010/12/12/redline-smalltalk-in-“small-talks”/
    >>>>
    >>>> More than just Smalltalk is covered.
    >>>
    >>> in my case, I would like (decent) C and C# support...
    >>> also maybe dynamic compilation, ...

    >>
    >> C# is done:
    >>
    >> http://dev.mainsoft.com/Default.aspx?tabid=130#MS_IL_to_Java_ByteCode_Compilation

    >
    > this is a way to do it...
    > admittedly it is a bit crufty IMO, but oh well.


    To support multiple .NET languages it may actually
    be the smart way.

    >> There are plenty of dynamic compilation possibilities
    >> for Java.

    >
    > agreed...
    >
    > one can produce bytecode and shove it into the classloader, for example...
    >
    > if one has JDK they can invoke javac (but, it is a little bit of a hack
    > though, and "compile this Java code and load it" is not exactly a
    > standard API feature, although yes, it is more standard than throwing
    > together ones' own VM to do the job).


    Java has had a documented compiler API since 1.6.

    (I think that API is unnecessary convoluted, but it exists)

    Arne
    Arne Vajhøj, Dec 12, 2010
    #6
  7. jamesl

    Roedy Green Guest

    On Sat, 11 Dec 2010 22:42:43 -0700, BGB <> wrote,
    quoted or indirectly quoted someone who said :

    >in my case, I would like (decent) C and C# support...
    >also maybe dynamic compilation, ...


    It would have to be a C-like language, or have lots of native code to
    implement basic functionality. There are things you can do in C that
    are forbidden in the JVM -- e.g. store data on top of other data
    accessing it with a different descriptor, use pointers to change
    arbitrary bits in ram.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com

    Doubling the size of a team will probably make it produce even more slowly.
    The problem is the more team members, the more secrets, the less each team
    member understands about how it all fits together and how his changes may
    adversely affect others.
    Roedy Green, Dec 12, 2010
    #7
  8. jamesl

    BGB Guest

    On 12/12/2010 2:33 PM, Roedy Green wrote:
    > On Sat, 11 Dec 2010 22:42:43 -0700, BGB<> wrote,
    > quoted or indirectly quoted someone who said :
    >
    >> in my case, I would like (decent) C and C# support...
    >> also maybe dynamic compilation, ...

    >
    > It would have to be a C-like language, or have lots of native code to
    > implement basic functionality. There are things you can do in C that
    > are forbidden in the JVM -- e.g. store data on top of other data
    > accessing it with a different descriptor, use pointers to change
    > arbitrary bits in ram.



    actually, I was working on adding VM support for all of this (structs,
    pointers, unions, ...), despite it being non-standard (so, C would work
    on my VM, but, sadly, not on a standard JVM).

    (as noted elsewhere, emulation is not a valid option in my case).


    currently uncertain though is the 'best' way to handle C style linking
    semantics (ideally without having to use VM extensions or redirect
    function calls via API calls, IOW: call a method to call a function).

    better would be to call directly via static method calls, but this is
    complicated some by it being less obvious which functions exist in which
    source modules (it would require a compiler trick to be able to figure
    this out).


    supporting non-class static methods or partial classes is a possible way
    around this, but, sadly, this would also be a VM extension (if support
    were added).

    a non-class static method would simply be an 'invokestatic' with a
    method where the class is null, which would be interpreted as being a
    special case.


    possible hack:
    any class-names starting with '$' (or 'V_$' or similar) are not real
    classes, but infact are considered as a part of the containing package.


    also possible:
    the in-classfile classname doesn't necessarily match the external
    filename, and this may be used to support both partial and non-class
    methods.

    say:
    myapp/Foo$14A9.class
    myapp/Foo$3C9F.class
    ....

    so, all would self-identify as "myapp/Foo", and all merge into a single
    class in the VM.

    myapp/_$2C9A.class
    could identify itself simply as "myapp" and thus merge into the
    containing package.

    where after the '$' would be a hash or a gensym, so that clashes can be
    avoided (if a hash, it could be a hash of the original filename or similar).


    note:
    a basic
    "myapp/Foo.class" or "myapp/_.class" would not exist, as to help avoid
    confusion with inner classes.


    C scoping would work by assuming than an open-ended partial class exists
    representing the entire C toplevel (any C functions would then be
    assumed to be a method in this class).


    I guess it is a big question over whether a standard JVM C-subset needs
    to be able to exist, or if C support will simply not work on a standard
    JVM...


    or such...
    BGB, Dec 12, 2010
    #8
  9. jamesl

    BGB Guest

    On 12/12/2010 2:08 PM, Arne Vajhøj wrote:
    > On 12-12-2010 13:29, BGB wrote:
    >> On 12/12/2010 8:28 AM, Arne Vajhøj wrote:
    >>> On 12-12-2010 00:42, BGB wrote:
    >>>> On 12/11/2010 8:18 PM, jamesl wrote:
    >>>>> If you have ever wanted to implement a new language on top of the JVM,
    >>>>> then this series may be of
    >>>>> interest to you: Redline: Smalltalk in 'small talks' ....
    >>>>> http://blog.redline.st/2010/12/12/redline-smalltalk-in-“small-talks”/
    >>>>>
    >>>>>
    >>>>> More than just Smalltalk is covered.
    >>>>
    >>>> in my case, I would like (decent) C and C# support...
    >>>> also maybe dynamic compilation, ...
    >>>
    >>> C# is done:
    >>>
    >>> http://dev.mainsoft.com/Default.aspx?tabid=130#MS_IL_to_Java_ByteCode_Compilation
    >>>

    >>
    >> this is a way to do it...
    >> admittedly it is a bit crufty IMO, but oh well.

    >
    > To support multiple .NET languages it may actually
    > be the smart way.
    >


    well, this is assuming that one is supporting C# for sake of supporting
    ..NET, rather than supporting C# for sake of having "something like Java
    but more C++ like and with more nifty features...".

    if generalized (say, to support dynamically loading .NET assemblies),
    this would mean essentially dragging around an implementation of both
    VM's at the same time, and is not likely to be "that" much better than
    the option of more simply using Mono and IKVM...


    and, in my case, I could just go and finish writing the ECMA-335 loader
    for my VM...

    I decided against this though, as JBC seemed a little closer to my goals
    (even though it would still require a little more hacking than MSIL
    would have).


    actually, less effort may well have just been to just use one of my own
    bytecode formats, but I have my reasons.


    like, options:
    use JBC, potentially have to hack it all to hell to support wanted
    featureset;
    use MSIL, have wanted featureset, face architectural and usage-related
    conflicts (likely more effort than hacking JBC);
    use my own bytecode, but have no direct binary compatibility with
    anything...


    >>> There are plenty of dynamic compilation possibilities
    >>> for Java.

    >>
    >> agreed...
    >>
    >> one can produce bytecode and shove it into the classloader, for
    >> example...
    >>
    >> if one has JDK they can invoke javac (but, it is a little bit of a hack
    >> though, and "compile this Java code and load it" is not exactly a
    >> standard API feature, although yes, it is more standard than throwing
    >> together ones' own VM to do the job).

    >
    > Java has had a documented compiler API since 1.6.
    >
    > (I think that API is unnecessary convoluted, but it exists)
    >


    yes, ok.


    may go look at it...
    BGB, Dec 12, 2010
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kevin Hooke

    Re: Handling both MS JVM and Sun JVM

    Kevin Hooke, Aug 26, 2003, in forum: Java
    Replies:
    2
    Views:
    865
  2. Lasse
    Replies:
    1
    Views:
    682
    Jon A. Cruz
    Jan 5, 2004
  3. Young-Jin Lee

    MS JVM and Sun JVM problem

    Young-Jin Lee, Jan 20, 2004, in forum: Java
    Replies:
    3
    Views:
    563
    Mickey Segal
    Jan 21, 2004
  4. Replies:
    9
    Views:
    501
    Chris Uppal
    Jan 4, 2005
  5. Replies:
    28
    Views:
    1,143
Loading...

Share This Page