Getting my head around generics.

[Roedy Green]

Is this true?

If I take a simple container class e.g .

public class Stack

and put <t> in like this

public class Stack<T>

and replace every word "Object" with "T" in the class body I will
have a new checked version of the container class.

Oddly if I examine the byte codes generated they are exactly the same
as the old ones. Nothing has changed.

Everything still works inside with objects.

There are no casts added anywhere.

There are no type checks added anywhere.

However if you look at he byte code of the class that USES you checked
class, you will see it had added cast checks on parameters of method
calls to the container, and cast conversions of object results back to
the specific type.

The compiler knows the precise type of the call an compile time, so it
can hard code these in with the correct types.

This is a rinky dink way do to generics.

Some of the restrictions then are:

1. the container cannot construct new Objects of type T, only objects.

2. It cannot construct arrays of new Objects of type T, only Object[].

3. The container has no access to T methods other than those of
Object. It is a pure container. It cannot do anything clever with the
objects it contains.


However when you use the <X extends Dog> syntax then the code
generated for the container is done internally with Dog references.
That gives the container access to the Dog methods, but not the Dog
constructor or the constructors of any of the subclasses of Dog,
except by heroic measures using reflection or getClass.newInstance.
without constructor parameters.

the big drawback then of Java's way of doing Generics is you can't
create new objects of type T or arrays of objects of type T.

Oddly you can create objects explicitly of type Dog or Dalmatian, but
not the generic T.

You then puzzle how does ArrayList manage to allocate an array to hold
its objects internally? It cheats. it allocates an array of Objects
and does and illegal cast to (T[]) which conveniently just triggers a
warning message. It is all objects inside anyway.





--
Bush crime family lost/embezzled $3 trillion from Pentagon.
Complicit Bush-friendly media keeps mum. Rumsfeld confesses on video.
http://www.infowars.com/articles/us/mckinney_grills_rumsfeld.htm

Canadian Mind Products, Roedy Green.
See http://mindprod.com/iraq.html photos of Bush's war crimes

 

[Stefan Schulz]

Is this true?

If I take a simple container class e.g .

public class Stack

and put <t> in like this

public class Stack<T>

and replace every word "Object" with "T" in the class body I will
have a new checked version of the container class.

You will have an unchecked version. All the checking is just "syntactic
sugar" around a class that still operates on Objects.

However if you look at he byte code of the class that USES you checked
class, you will see it had added cast checks on parameters of method
calls to the container, and cast conversions of object results back to
the specific type.

The compiler knows the precise type of the call an compile time, so it
can hard code these in with the correct types.

This is a rinky dink way do to generics.

It's called type erasure, and is the way sun decided to implement
Generics. Not really the nicest way to do so, but at least not as broken
as C++ Templates.

Some of the restrictions then are:

1. the container cannot construct new Objects of type T, only objects.

2. It cannot construct arrays of new Objects of type T, only Object[].

3. The container has no access to T methods other than those of
Object. It is a pure container. It cannot do anything clever with the
objects it contains.

However when you use the <X extends Dog> syntax then the code generated
for the container is done internally with Dog references. That gives the
container access to the Dog methods, but not the Dog constructor or the
constructors of any of the subclasses of Dog, except by heroic measures
using reflection or getClass.newInstance. without constructor
parameters.

You can, if you specify the Parameter to be bounded by some superclass
like this: <T extends MyClassOrInterface>. If you do that, "behind the
scenes" all Ts will be replaced with MyClassOrInterface, and you can
access the methods declared there as well. Since you do not know, at
compile time, which of the various subclasses of MyClassOrInterface will
be used in this instance of the generic class, you naturally do not know
which constructors will be available. Simple, right?

Oddly you can create objects explicitly of type Dog or Dalmatian, but
not the generic T.

As said above, it all comes down to the class not having any idea of what
T is on any given instance.

You then puzzle how does ArrayList manage to allocate an array to hold
its objects internally? It cheats. it allocates an array of Objects and
does and illegal cast to (T[]) which conveniently just triggers a
warning message. It is all objects inside anyway.

Yes. But this is kind of logical given the above explaination.

 

[Tim Miller]

Is this true?

If I take a simple container class e.g .

public class Stack

and put <t> in like this

public class Stack<T>

and replace every word "Object" with "T" in the class body I will
have a new checked version of the container class.

Oddly if I examine the byte codes generated they are exactly the same
as the old ones. Nothing has changed.

Yes, nothing shoudl change. Really, it is only the typechecker
that cares about the generics, seeing that it is compile-time, not
runtime, type consistency that we are after.

Everything still works inside with objects.

There are no casts added anywhere.

There are no type checks added anywhere.

However if you look at he byte code of the class that USES you checked
class, you will see it had added cast checks on parameters of method
calls to the container, and cast conversions of object results back to
the specific type.

Yes, this is correct too. The code generator knows that because
the typechecking succeeded, that the typecasts are all legal. As
far as I know, IF the only different in java 1.5 were to have been
generics and enums, you would still have been able to run the
bytecode on a java 1.4.x JVM.

The compiler knows the precise type of the call an compile time, so it
can hard code these in with the correct types.

This is a rinky dink way do to generics.

Not really. Otherwise, they would have had to have come up with a
new way to represent generics in bytecode.

the big drawback then of Java's way of doing Generics is you can't
create new objects of type T or arrays of objects of type T.

Yes, I agree this is a bit annoying.

Oddly you can create objects explicitly of type Dog or Dalmatian, but
not the generic T.

You then puzzle how does ArrayList manage to allocate an array to hold
its objects internally? It cheats. it allocates an array of Objects
and does and illegal cast to (T[]) which conveniently just triggers a
warning message. It is all objects inside anyway.

Yes, but it still ensures type consistency at compile time, which
is why generics were introduced. The warning message is a bit of a
pain, and I really do believe that generics should work with
arrays, because these warnings are VERY misleading.

Tim

 

[Roedy Green]

which of the various subclasses of MyClassOrInterface will
be used in this instance of the generic class, you naturally do not know
which constructors will be available.

Bong! I kept thinking that constructors are inherited just like
methods. This is what screws up using the constructor in any generic
way. Just because Dog has a given constructor does not be Dalmatian
has that same one. Unlike methods constructors can be decommitted by
inheritors.

to fix this there need to be a way of guaranteeing that a constructor
is available in all subclasses.

It could be "final" or "mandatory" -- something each level must
implement.

Perhaps it could also be handled with no change new keywords by
letting you use the base class constructors and getting an
MissingConstructor exception at runtime if the subclass does not
implement it, or at compile time when the generic class is used with a
specific class.
--
Bush crime family lost/embezzled $3 trillion from Pentagon.
Complicit Bush-friendly media keeps mum. Rumsfeld confesses on video.
http://www.infowars.com/articles/us/mckinney_grills_rumsfeld.htm

Canadian Mind Products, Roedy Green.
See http://mindprod.com/iraq.html photos of Bush's war crimes

 

[Stefan Schulz]

Perhaps it could also be handled with no change new keywords by
letting you use the base class constructors and getting an
MissingConstructor exception at runtime if the subclass does not
implement it, or at compile time when the generic class is used with a
specific class.

This would have to be a runtime action since the generic class needs not
be available in source at all, and therefore you can not know which
constructors are requested.

 

[Thomas G. Marshall]

Roedy Green coughed up:

Is this true?

If I take a simple container class e.g .

public class Stack

and put <t> in like this

public class Stack<T>

and replace every word "Object" with "T" in the class body I will
have a new checked version of the container class.

Oddly if I examine the byte codes generated they are exactly the same
as the old ones. Nothing has changed.

Everything still works inside with objects.

There are no casts added anywhere.

There are no type checks added anywhere.

Hello erasure. Goodbye runtime protection. (@#$%ing grumble grumble...)
Actually the "goodbye" is inaccurate since there never /was/ runtime
protection to begin with.

However if you look at he byte code of the class that USES you checked
class, you will see it had added cast checks on parameters of method
calls to the container, and cast conversions of object results back to
the specific type.

The latter is of course well understood---once we've gotten past the
compilation phase, the datatypes are assumed to be correct. The cast
converstions are placed in.

The former of what you mention takes me a litttle by surprise. I would have
expected no such cast "checks" at all, unless I misread what you're saying
here.

....[rip]...

 

[Tim Tyler]

Hello erasure. Goodbye runtime protection. (@#$%ing grumble grumble...)
Actually the "goodbye" is inaccurate since there never /was/ runtime
protection to begin with.

You /could/ have runtime protection and operational reflection - if you
were prepared to build your own class for each type of collection you want.

The problem there is that you can't concisely say that you want a
collection of type Foo, please - any there's no factory method that
accepts a type as a parameter and returns a collecton object that handles
objects of that type either.

Similarly, there's no way to request a collection of objects of a type
which is only known at runtime - unless you are prepared to spit out your
own bytecode and load it using your own classloader.

That sort of thing is not what Java's language designers were expecting
you to want to do.

 

[Dale King]

Yes, this is correct too. The code generator knows that because the
typechecking succeeded, that the typecasts are all legal. As far as I
know, IF the only different in java 1.5 were to have been generics and
enums, you would still have been able to run the bytecode on a java
1.4.x JVM.

Well enums would not work, because you would need the java.lang.Enum
class in the 1.4 JVM. But you are correct about generics and as I said
yesterday in another thread the fact that we don't have a good way to
compile generics to run in 1.4 is a major problem and will slow the
overall adoption of generics significantly.

I thought all along that we were going to have this ability to run
generics in 1.4 and thought the limitations in Java generics were a
valid limitation. But since we don't have that ability the limitations
make no sense. Why come up with a scheme to do a limited form of
generics without requiring JVM changes when you require a new JVM anyway
to run them? Why not go ahead and do a complete implementation of
generics with JVM changes to fully support it?

 

[googmeister]

the big drawback then of Java's way of doing Generics is you can't

Yes, I agree this is a bit annoying.

I think this is more than annoying. It is a complete botch. You should
be able to code up data structures without needing a cast or having
the compiler spew out warnings. Isn't this supposed to be one of the
prime benefits of generics? With Java generics, it's not possible to
even write an array implementation of a generic stack without a cast
and a warning. Sure it's great that the stack client no longer needs
the cast, but it's a hollow victory in my opinion.

 

[Thomas G. Marshall]

(e-mail address removed) coughed up:

I think this is more than annoying. It is a complete botch. You should
be able to code up data structures without needing a cast or having
the compiler spew out warnings. Isn't this supposed to be one of the
prime benefits of generics? With Java generics, it's not possible to
even write an array implementation of a generic stack without a cast
and a warning. Sure it's great that the stack client no longer needs
the cast, but it's a hollow victory in my opinion.

Arrays already have fully type-safe types.

<typename>[ ] <ID> = new <typename>[ <arrayLen> ];

 

[Tim Tyler]

I thought all along that we were going to have this ability to run
generics in 1.4 and thought the limitations in Java generics were a
valid limitation. But since we don't have that ability the limitations
make no sense. Why come up with a scheme to do a limited form of
generics without requiring JVM changes when you require a new JVM anyway
to run them? Why not go ahead and do a complete implementation of
generics with JVM changes to fully support it?

There are sill reasons, though probably not very good ones.

Changes to the bytecode are still serious - since they interfere
with code decompilers, bytecode manipulation tools,
refactoring programs and numerous other third party tools.

Also, there are ways around the problem today - if new bytecode
was needed, translating generic apps to work under earlier Java
VMs would become *much* more difficult.

 

[Roedy Green]

Why not go ahead and do a complete implementation of
generics with JVM changes to fully support it?

The reason is Sun has a lot of people signed up to keep JVMs up to
date. They did not want to force a lot of work on them.

the new classes etc just makes work fro themselves, not for every JVM
implementation. I understand though there are many new native methods
which presumably are the JVM vendor's problem to implement.

--
Bush crime family lost/embezzled $3 trillion from Pentagon.
Complicit Bush-friendly media keeps mum. Rumsfeld confesses on video.
http://www.infowars.com/articles/us/mckinney_grills_rumsfeld.htm

Canadian Mind Products, Roedy Green.
See http://mindprod.com/iraq.html photos of Bush's war crimes

 

[googmeister]

Arrays already have fully type-safe types.

<typename>[ ] <ID> = new <typename>[ <arrayLen> ];

Not sure I follow. I want to implement a parameterized stack
with an array so that the client can write cast-free code
like the following.

ArrayStack<String> stack = new ArrayStack<String>();
stack.push("test");
String s = stack.pop();

This is possible with generics, but the implementation of
ArrayStack seems to require a cast somewhere. For
example, I'm not allowed to do the following.

public class ArrayStack<T> {
private T[] s = new T[initCapacity];
...

Instead I implement it as follows

public class ArrayStack<T> {
private Object[] s = new Object[initCapacity];

and have to cast the Object to type T in pop().

Are you claiming I can write a generic ArrayStack
with any casts or warnings? I'd love to learn how.

 

[Thomas G. Marshall]

(e-mail address removed) coughed up:

Arrays already have fully type-safe types.
<typename>[ ] <ID> = new <typename>[ <arrayLen> ];

Not sure I follow.

It was *I* who didn't understand what you were asking.

I want to implement a parameterized stack
with an array so that the client can write cast-free code
like the following.

ArrayStack<String> stack = new ArrayStack<String>();
stack.push("test");
String s = stack.pop();

This is possible with generics, but the implementation of
ArrayStack seems to require a cast somewhere. For
example, I'm not allowed to do the following.

public class ArrayStack<T> {
private T[] s = new T[initCapacity];
...

Instead I implement it as follows

public class ArrayStack<T> {
private Object[] s = new Object[initCapacity];

and have to cast the Object to type T in pop().

Are you claiming I can write a generic ArrayStack
with any casts or warnings? I'd love to learn how.

You've done a wonderful job of explaining the situation, and I have done a
botched job of understanding it. Thanks for the clarification.

 

[Thomas G. Marshall]

Roedy Green coughed up:

The reason is Sun has a lot of people signed up to keep JVMs up to
date. They did not want to force a lot of work on them.

the new classes etc just makes work fro themselves, not for every JVM
implementation. I understand though there are many new native methods
which presumably are the JVM vendor's problem to implement.

Furthermore, I have the strongest suspicion that if they /had/ implemented
full runtime type protection which required new bytecode, etc., that there
would be just as many people bitching about that change. I also suspect
that a large subset of them would be the ones complaining now.

Present company excluded of course. I am in the former camp: If they were
going to bother with this at all, they really should have done it correctly,
even if it broke things here and there and made people temporarily
miserable. It's better to be very miserable for a short time than slightly
miserable for /ever/.

 

[Thomas Weidenfeller]

Changes to the bytecode are still serious - since they interfere
with code decompilers, bytecode manipulation tools,
refactoring programs and numerous other third party tools.

Yes, they are. But that didn't prevent Sun from changing the .class file
format (yet not the bytecode inside the files) for 1.5 once again. Which
has an equal devastating effect on these tools.

So you already have two reasons why 1.5 class files don't run on earlier
VMs: The missing special-purpose classes like Enum, and the new class
file format. New byte code instructions couldn't have made it much worse.

/Thomas

 

[Roedy Green]

So you already have two reasons why 1.5 class files don't run on earlier
VMs: The missing special-purpose classes like Enum, and the new class
file format. New byte code instructions couldn't have made it much worse.

is the new class file format just to handle annotations? did they not
leave an escape hatch in the old design to allow new fields to be
added in a compatible way?

--
Bush crime family lost/embezzled $3 trillion from Pentagon.
Complicit Bush-friendly media keeps mum. Rumsfeld confesses on video.
http://www.infowars.com/articles/us/mckinney_grills_rumsfeld.htm

Canadian Mind Products, Roedy Green.
See http://mindprod.com/iraq.html photos of Bush's war crimes

 

[Thomas Weidenfeller]

is the new class file format just to handle annotations? did they not
leave an escape hatch in the old design to allow new fields to be
added in a compatible way?

enums, annotations, signatures, varargs, and probably a couple more. The
following document will tell you:

http://java.sun.com/docs/books/vmspec/2nd-edition/ClassFileFormat-Java5.pdf


BTW, as I just learned, Sun did not only change the file format, they
also changed the instruction set of the VM. The ldc, putfield and
putstatic instructions were changed.

http://java.sun.com/docs/books/vmspec/2nd-edition/Java5-Instructions2.pdf

That makes it even more strange why Sun didn't dare to change the
instruction set to support generics, but instead just bolted them
somehow onto the language.

/Thomas

 

[Thomas Weidenfeller]

I thought all along that we were going to have this ability to run
generics in 1.4 and thought the limitations in Java generics were a
valid limitation. But since we don't have that ability the limitations
make no sense.

You could try to get that thing to work:

http://retroweaver.sourceforge.net/

/Thomas

 

[Chris Uppal]

So you already have two reasons why 1.5 class files don't run on earlier
VMs: The missing special-purpose classes like Enum, and the new class
file format. New byte code instructions couldn't have made it much worse.

That's probably misleading. It's much more nearly true to say that the
classfile format doesn't change -- in the sense that the format is extensible,
and nothing has been added to it over the years that doesn't use that
extensibility in ways that are completely transparent to parsers that only know
about the previous formats.

HOWEVER. Sun have made yet another of their ghastly botches in 1.5. There's a
completely unecessary extension to the semantics of a few bytecodes (as you
mentioned in a later post). Also, they've buggered about with how synthetic
methods (and so on) are marked in the format in ways that would confuse an
earlier parser.

-- chris