Annotations processing + type introspection + code generation

[Alex J]

Hi all,

I need a annotation processing tool with the possibility to generate
source code.
What I want to do is to process specific annotations *and* generate
boilerplate code to the newly introduced classes that I plan to use
via ServiceLoader.

In fact, I can do that with Open JDK (by using the standard annotation
processing facility which is built-in to the javac compiler starting
from the Java 6), but deep type introspection and code generations
require to use sun-specific classes (basically com.sun.tools.javac.*)
which obviously is an ugly hack, not to mention that
(1) they change internal AST and Type tree representations from
release to release
(2) the interface to internal AST (JCTree descendants) is ugly - most
classes expose public fields, extensively use bit flags and have
complex and cluttered API.

I believe that there is a better way to do that (annotations
processing + type (AST) introspection + code generation) but I don't
know what tool I should use.

P.S.: I'm using maven and I'd like to do that in IDE-independent way.
P.P.S: Downgrading to JDK 5 is inappropriate.

 

[Robert Klemme]

Hi all,

I need a annotation processing tool with the possibility to generate
source code.
What I want to do is to process specific annotations *and* generate
boilerplate code to the newly introduced classes that I plan to use
via ServiceLoader.

In fact, I can do that with Open JDK (by using the standard annotation
processing facility which is built-in to the javac compiler starting
from the Java 6), but deep type introspection and code generations
require to use sun-specific classes (basically com.sun.tools.javac.*)
which obviously is an ugly hack, not to mention that
(1) they change internal AST and Type tree representations from
release to release
(2) the interface to internal AST (JCTree descendants) is ugly - most
classes expose public fields, extensively use bit flags and have
complex and cluttered API.

I believe that there is a better way to do that (annotations
processing + type (AST) introspection + code generation) but I don't
know what tool I should use.

P.S.: I'm using maven and I'd like to do that in IDE-independent way.
P.P.S: Downgrading to JDK 5 is inappropriate.

Why do you want to generate code? I mean, you could process annotations
at runtime. Create appropriate classes for the handling and configure
appropriate graphs of objects which do the work (or even use reflection).

The only reason I can think of which would require to generate code
would be ultra high speed requirements. But before I would go down that
road I'd first make sure that the other approach is not fast enough
(i.e. implement and measure). If it is, you can stop there and be done.

Kind regards

robert

 

[markspace]

The only reason I can think of which would require to generate code
would be ultra high speed requirements.

I think generated code is somewhat more common that you assume. Since
the JVM byte codes and class format is public and relatively stable,
it's quite easy to generate code in a manner that's compatible with
many, or all, JVMs.

There's also things you can't do at runtime. For example, the Proxy
class requires an interface to extend. But many POJO objects don't
declare a separate interface. And requiring the programmer to declare
an interface for all POJO types within a framework is pretty odious, and
old school.

I'm playing around with JSF and JPA right now. I notice that all my
Entity classes are decorated by extending them with generated code. I'm
sure if there were a better way to do it, they would. But the generated
code seems to be the way to go in this instance.

 

[Robert Klemme]

I think generated code is somewhat more common that you assume. Since
the JVM byte codes and class format is public and relatively stable,
it's quite easy to generate code in a manner that's compatible with
many, or all, JVMs.

There's also things you can't do at runtime. For example, the Proxy
class requires an interface to extend. But many POJO objects don't
declare a separate interface. And requiring the programmer to declare an
interface for all POJO types within a framework is pretty odious, and
old school.

I'm playing around with JSF and JPA right now. I notice that all my
Entity classes are decorated by extending them with generated code. I'm
sure if there were a better way to do it, they would. But the generated
code seems to be the way to go in this instance.

OK, good point! I had generation of boilerplate code (e.g. via things
like XDoclet) in mind which is really 20th century. Depending on
the use case reflection could still be sufficiently fast enough - and
it's likely simpler than byte code generation.

Kind regards

robert

 

[Alex J]

[snip]
Why do you want to generate code?  I mean, you could process annotations
at runtime.  Create appropriate classes for the handling and configure
appropriate graphs of objects which do the work (or even use reflection).

I need to generate *source* code, the boilerplate code is complex
enough, unfortunately bytecode is not an option.

One of the reason why I need generated code is that the boilerplate
code is quite sofisticated, learning generated docs and debugging the
generated code most likely will be required.

There are other reasons, but I think that this one would be
sufficient.

The only reason I can think of which would require to generate code
would be ultra high speed requirements.  But before I would go down that
road I'd first make sure that the other approach is not fast enough
(i.e. implement and measure).  If it is, you can stop there and be done..

Reflection performance is not a bottleneck, I need source code as I
mentioned earlier.
The convenient way of doing so would be to build AST by hand by using
OpenJDK internal AST facilities, but as I mentioned earlier this way
is too far from perfection.

 

[Alex J]

OK, good point!  I had generation of boilerplate code (e.g. via things
like XDoclet) in mind which is really 20th century.  Depending on
the use case reflection could still be sufficiently fast enough - and
it's likely simpler than byte code generation.

XDoclet (http://en.wikipedia.org/wiki/XDoclet, right?) looks quite
frightening

Well, it's not a problem to generate code by simple print calls,
entirely without AST or things like that.

The main issue is the power enough type introspection as I need to
know object model and sometimes methods implementation before code
generation. Annotations processing API does not fit my needs, that's
why I started to learn javac internals to cast APT interfaces to javac
internal interfaces to access AST.

 

[Robert Klemme]

[snip]
Why do you want to generate code? I mean, you could process annotations
at runtime. Create appropriate classes for the handling and configure
appropriate graphs of objects which do the work (or even use reflection).

I need to generate *source* code, the boilerplate code is complex
enough, unfortunately bytecode is not an option.

One of the reason why I need generated code is that the boilerplate
code is quite sofisticated, learning generated docs and debugging the
generated code most likely will be required.

There are other reasons, but I think that this one would be
sufficient.

Hm... If you need to generate a lot identical code there is something
wrong: that identical code could be better implemented directly and
used. So that leaves adapters and stuff Mark mentioned for generation.
I am not convinced yet that it is a good idea to generate loads of
complex code.

Can you shed some more light on the nature of the application you are
going to implement?

Kind regards

robert

 

[Alex J]

[snip]
Why do you want to generate code?  I mean, you could process annotations
at runtime.  Create appropriate classes for the handling and configure
appropriate graphs of objects which do the work (or even use reflection).

I need to generate *source* code, the boilerplate code is complex
enough, unfortunately bytecode is not an option.

One of the reason why I need generated code is that the boilerplate
code is quite sofisticated, learning generated docs and debugging the
generated code most likely will be required.

There are other reasons, but I think that this one would be
sufficient.

Hm...  If you need to generate a lot identical code there is something
wrong: that identical code could be better implemented directly and
used.  So that leaves adapters and stuff Mark mentioned for generation.
  I am not convinced yet that it is a good idea to generate loads of
complex code.

Can you shed some more light on the nature of the application you are
going to implement?

Yes, that's a prototype of tool that helps to implement a part of code
for you.

You (as a user) define interface, the tool tries to guess what you
want based on the name patterns, references to domain objects, etc.

Say you have the following:

@Specification // tells tool "this interface is a subject to be
processed"
interface PersonDao {
long savePerson(String name, int age);
Person getPerson(long id);
List<Long> getPersons();
}

interface Person {
String getName();
int getAge();
}

Then tool defines final class PersonDaoImpl and PersonImpl:

final class PersonDaoImpl implements PersonDao {
private static final class PersonImpl extends JdbcDaoSupport
implements Person {
private final String name;
private final int age;

PersonImpl(String name, int age) { this.name = name; this.age =
age; }

@Override public String getName() { return name; }
@Override public int getAge() { return age; }
}

@Override public long savePerson(String name, int age) {
getJdbcTemplate().update("INSERT INTO person (name, age) VALUES
(?, ?)", name, age);
return getJdbcTemplate.queryForInt("CALL identity()");
}
//.. etc.
}


This is not that simple as I gave above (as much more complex human-
friendly definitions might be involved), but this clearly describes
the idea.
I'm implementing this out of pure interest, as my hobby project