Callbacks and checked exceptions

[Harold Yarmouth]

One of the glaring limitations of Java seems to be in connection with
callback objects.

To illustrate it, consider trying to make a generic cache class, backed
by a map.

public class Cache<K,V> {
protected Map<K,V> map;
private Getter getter;

public interface Getter<KK,VV> {
VV get (KK key);
}

Cache (Getter getter) {
if (getter == null) {
throw new IllegalArgumentException();
}
this.getter = getter;
}

public VV get (KK key) {
VV result = map.get(key);
if (result != null) return result;
result = getter.get(key);
if (result != null) map.put(key, result);
return result;
}
}

Simple enough, right? It returns a cached value but is provided with an
object capable of obtaining a new one.

The problem occurs when someone wants to use a getter that, say, reads
items from disk. The compiler won't let their getter throw IOException.

The class above could be changed to add throws clauses on both get
methods, but what should they throw? It really depends on the
implementation.

Declaring them as just "throws Exception" is ugly.

It is possible to create a generic version: Cache<K,V,E extends
Exception> with both gets declared as "throws E". This runs into more
problems:

First, if more than one unrelated exception type might be thrown by a
getter, E will have to just be Exception, and we're back to square one.

Second, there's no "proper" way of saying that this particular cache's
getter doesn't throw anything (checked) at all. (There's a bit of a
kludge: make E RuntimeException or a subclass. Works, but is ugly.)

Perhaps Java needs "generic exception lists"? Where a type parameter
that is only used in throws clauses, is bounded below by Throwable, and
is the last in the parameter list can be replaced with a whole list of
exception types (so long as they all fit the bound).

This would allow the Cache<K,V,E extends Exception> to be instantiated
as Cache<String, BigExpensiveThingy, IOException, SAXException> for
example, and the get method would be understood by the compiler to be
capable of throwing IOException, SAXException, and errors and unchecked
exceptions, but not, say, AWTException. Likewise its getter's get method
could be implemented to read and parse XML files from disk and just let
escape the IOExceptions and SAXExceptions thus generated from time to time.

Currently, the best that can be done in this case is declare the Cache
as Cache<String, BigExpensiveThingy, Exception>, and if you wrap a call
to its get method in catch(SAXException e) do this, catch(IOException e)
do that, or in a method that can throw both, the compiler will complain
because it thinks there's a chance that the checked exception
AWTException (among others) could come bubbling up from inside of the
call to get and go unhandled. So you need to declare "throws Exception"
wherever you use the cache (thus moving the problem one level up the
hierarchy of method calls) or, worse, "catch (Exception e)" which is
generally bad practise.

Mitigatable:

catch (Exception e) {
throw (RuntimeException)e;
}

Ugly, but stops it from eating RuntimeExceptions and shouldn't throw
ClassCastException if you really have handled every checked exception
your getter can throw. (Unless, of course, it REthrows
ClassCastException.) You still lose the compiler's double-checking that
you've actually handled properly every exception type that really can be
thrown; this is as bad as having thought-to-be-correct generics code
that produces an unchecked conversion warning, though no worse.

Cache could be declared <K,V,E1 extends Exception, E2 extends Exception>
or one could even have E3 and above. Then things get even uglier for the
folks whose getters aren't throwing much of anything:

MyDiskFileGetterThatDoesNotUseXML implements Cache.Getter<String,
BigExpensiveObject, IOException, RuntimeException> (or 2x IOException);
MyExpensiveComputation implements Cache.Getter<Integer,
BigExpensiveMathematicalThingamabob, RuntimeException,
RuntimeException>; and so forth.

So our present-day alternatives, in descending order from bad to worse
to positively abysmal:

Clutter up the code with extra exception patameters that are often
redundant;

Clutter up the code with catches that cast and rethrow, and lose some
amount of exception type-safety;

Abuse RuntimeExceptions (or worse, Errors) to wrap checked exceptions
and smuggle them out of the getter (and lose even more exception
type-safety); and

Throw nothing, and indicate errors by returning out-of-band values
(which is not only horrible practise, but will clutter up the cache);

Modify Cache.Getter to return some sort of Pair<KK,Boolean> so it can
simultaneously return a value and tell the cache whether to actually put
it in the map (which leads to code that's just freaking awful -- caches
with key type Object that sometimes return IOExceptions -- not throw,
return -- and so forth -- so much for type safety).

Object result = myCache.get(key);
if (result instanceof IOException || result instanceof SAXException)
throw result;
BigExpensiveThingy bet = (BigExpensiveThingy)result;
// I want to throw up!

java.lang.ClassCastException
at line 33 of AwfulCode.java

// I give up.
throw new Upchuck();

We could really use "exception type parameter varargs". Really, we
could. Checked exceptions just don't play nice with "inversion of
control" like patterns otherwise.

 

[Arne Vajhøj]

One of the glaring limitations of Java seems to be in connection with
callback objects.

To illustrate it, consider trying to make a generic cache class, backed
by a map.

public class Cache<K,V> {
protected Map<K,V> map;
private Getter getter;

public interface Getter<KK,VV> {
VV get (KK key);
}

Cache (Getter getter) {
if (getter == null) {
throw new IllegalArgumentException();
}
this.getter = getter;
}

public VV get (KK key) {
VV result = map.get(key);
if (result != null) return result;
result = getter.get(key);
if (result != null) map.put(key, result);
return result;
}
}

Simple enough, right? It returns a cached value but is provided with an
object capable of obtaining a new one.

The problem occurs when someone wants to use a getter that, say, reads
items from disk. The compiler won't let their getter throw IOException.

The class above could be changed to add throws clauses on both get
methods, but what should they throw? It really depends on the
implementation.

Declaring them as just "throws Exception" is ugly.

It is possible to create a generic version: Cache<K,V,E extends
Exception> with both gets declared as "throws E". This runs into more
problems:

First, if more than one unrelated exception type might be thrown by a
getter, E will have to just be Exception, and we're back to square one.

Second, there's no "proper" way of saying that this particular cache's
getter doesn't throw anything (checked) at all. (There's a bit of a
kludge: make E RuntimeException or a subclass. Works, but is ugly.)

Perhaps Java needs "generic exception lists"? Where a type parameter
that is only used in throws clauses, is bounded below by Throwable, and
is the last in the parameter list can be replaced with a whole list of
exception types (so long as they all fit the bound).

This would allow the Cache<K,V,E extends Exception> to be instantiated
as Cache<String, BigExpensiveThingy, IOException, SAXException> for
example, and the get method would be understood by the compiler to be
capable of throwing IOException, SAXException, and errors and unchecked
exceptions, but not, say, AWTException. Likewise its getter's get method
could be implemented to read and parse XML files from disk and just let
escape the IOExceptions and SAXExceptions thus generated from time to time.

Currently, the best that can be done in this case is declare the Cache
as Cache<String, BigExpensiveThingy, Exception>, and if you wrap a call
to its get method in catch(SAXException e) do this, catch(IOException e)
do that, or in a method that can throw both, the compiler will complain
because it thinks there's a chance that the checked exception
AWTException (among others) could come bubbling up from inside of the
call to get and go unhandled. So you need to declare "throws Exception"
wherever you use the cache (thus moving the problem one level up the
hierarchy of method calls) or, worse, "catch (Exception e)" which is
generally bad practise.

Mitigatable:

catch (Exception e) {
throw (RuntimeException)e;
}

Ugly, but stops it from eating RuntimeExceptions and shouldn't throw
ClassCastException if you really have handled every checked exception
your getter can throw. (Unless, of course, it REthrows
ClassCastException.) You still lose the compiler's double-checking that
you've actually handled properly every exception type that really can be
thrown; this is as bad as having thought-to-be-correct generics code
that produces an unchecked conversion warning, though no worse.

Cache could be declared <K,V,E1 extends Exception, E2 extends Exception>
or one could even have E3 and above. Then things get even uglier for the
folks whose getters aren't throwing much of anything:

MyDiskFileGetterThatDoesNotUseXML implements Cache.Getter<String,
BigExpensiveObject, IOException, RuntimeException> (or 2x IOException);
MyExpensiveComputation implements Cache.Getter<Integer,
BigExpensiveMathematicalThingamabob, RuntimeException,
RuntimeException>; and so forth.

So our present-day alternatives, in descending order from bad to worse
to positively abysmal:

Clutter up the code with extra exception patameters that are often
redundant;

Clutter up the code with catches that cast and rethrow, and lose some
amount of exception type-safety;

Abuse RuntimeExceptions (or worse, Errors) to wrap checked exceptions
and smuggle them out of the getter (and lose even more exception
type-safety); and

Throw nothing, and indicate errors by returning out-of-band values
(which is not only horrible practise, but will clutter up the cache);

Modify Cache.Getter to return some sort of Pair<KK,Boolean> so it can
simultaneously return a value and tell the cache whether to actually put
it in the map (which leads to code that's just freaking awful -- caches
with key type Object that sometimes return IOExceptions -- not throw,
return -- and so forth -- so much for type safety).

Object result = myCache.get(key);
if (result instanceof IOException || result instanceof SAXException)
throw result;
BigExpensiveThingy bet = (BigExpensiveThingy)result;
// I want to throw up!

java.lang.ClassCastException
at line 33 of AwfulCode.java

// I give up.
throw new Upchuck();

We could really use "exception type parameter varargs". Really, we
could. Checked exceptions just don't play nice with "inversion of
control" like patterns otherwise.

You missed the only good solution.

Getter is an interface. The method get throws an GetterException. The
implementations of Getter catches the implementation specific exceptions
and throw a GetterException. Cache get also throws an GetterException.

Arne

 

[Harold Yarmouth]

You missed the only good solution.

No, I did not. I mentioned wrapping the exceptions earlier in my post as
a possible but less-than-100%-satisfactory solution.

I have a suspicion you did not read the whole post carefully.

I also have a suspicion, based on your recent behavior in another
thread, that you are "spoiling for a fight" rather than attempting to
actually be helpful. That kind of nonconstructive attitude will not be
helpful here, and will make this newsgroup less useful for everybody. So
if you dislike me for some reason, I suggest you just ignore me.

 

[Harold Yarmouth]

No, I did not. I mentioned wrapping exceptions in my post, as a
less-than-100%-satisfactory solution.

Arne, your (or the) solution is actually cleaner and more robust than a
parametrized Exception would be.

I don't think so.

The lack of parametrized Exceptions in Java causes no pain.

This is not true.

You, too, were involved in that other thread, and I suspect you too are
posting not to be helpful but to cause a problem. This is not good. If
you don't like me, for some reason, you should ignore me rather than try
to stir up trouble.

 

[Daniel Pitts]

No, I did not. I mentioned wrapping exceptions in my post, as a
less-than-100%-satisfactory solution.


I don't think so.


This is not true.

You, too, were involved in that other thread, and I suspect you too are
posting not to be helpful but to cause a problem. This is not good. If
you don't like me, for some reason, you should ignore me rather than try
to stir up trouble.

Arne and Lew tend to have engineering suggestions that are very useful.

Why is wrapping exceptions less-than-100%-satisfactory?

 

[Harold Yarmouth]

Arne and Lew tend to have engineering suggestions that are very useful.

Unfortunately, they seem to be unable to limit their participation to
only dispassionate remarks about engineering.

Why is wrapping exceptions less-than-100%-satisfactory?

Because it is a bit of a kludge that is, in principle, avoidable with a
slight change to how generics work.

 

[Mike Schilling]

Unfortunately, they seem to be unable to limit their participation
to
only dispassionate remarks about engineering.


Because it is a bit of a kludge that is, in principle, avoidable
with
a slight change to how generics work.

That's the interesting question, I think: whether it is possible with
a slight change to how generics work, that is, whether it's possible
given erasure. [1]. The erasure of

void meth() throws E

is necessarily

void meth() throws Exception [2]

for the same reason that the erasure of

T meth2()

is
Object meth2()

requiring (in a context where T == String)

String t = meth2();

to be rewritten by the compiler as

String t = (String)meth2();

This isn't an area in which I have the depth of understanding I'd
like, but it seems to me that (in a context where E == IOException)

meth()

could be rewritten as

try
{
meth();
}
catch (RuntimeException ex)
{
// No need to generate this if RuntimeException is already
caught
throw ex;
}
catch (IOException ex)
{
// No need to generate this if IOException is already caught
throw ex;
}
catch (Exception ex)
{
// No need to generate this if Exception is already caught.
The compiler will prevent unexpected exceptions from being
// rethrown.
throw new UnexpectedGenericException(ex, IOException.class);
}

where UnexpectedGenericException is a new RuntimeException thrown
explicitly when generics has been "fooled" in some fashion. It seems
to me that the result is

A. Feasible, since the compiler knows enough from the type system to
produce it.
B. Safe, since no unexpected checked exception is ever thrown.
C. Not terribly expensive in CPU time. It's free if no exception is
thrown, though one throw always turns into two. At least the search
for a catch block is always as cheap as possible.
D Fairly expensive in code size, since each method call usually
requires (at least) three catch blocks, though I believe there's
compiler magic to allow reducing this, that is, to use the same
generated catch blocks for each call to meth() (or any other method
declared to throw the same generic exceptions) within a single method.

That is, I think this feature really could be added without any
significant changes in how generics works.

1. I'm aware that erasure has many uninutive and unfortunate
consequences, but ending it cannot be described as a slight change.
2. Or possibly throws Throwable, but for simplicity let's assume that
the "generic throws" feature is limited to Exceptions.

 

[Andreas Leitgeb]

Because it is a bit of a kludge that is, in principle, avoidable with a
slight change to how generics work.

I don't think, that "slight change" is an appropriate description
for what would need to be done for this to work.

I do agree to the principle, that it would be a good thing, being
able to have an actual list of (checked) Exceptions for a method of
a generic class, that is not hardcoded, but inferred from outside
(upon usage), such that the compiler can make sure, that all checked
Exceptions given by some parameter are individually(!) handled.

But: it's far not as simple as you made it look like:

The exception list is a property of each method, not of the whole
class. (Interfaces with just one method come up frequently, but
are technically just one not-in-any-way-special case.)
Generics parameter lists, however are for the whole class.

How would you (discussing proposed syntax here) declare, that
Cache's get() throws all these exceptions as inferred from the
used Getter-implementation, but some sooner or later added
"flush()" doesn't, and that some further (fictive) method
"getAndCacheFromSomewhereElse(Getter<...> g, KK k)" may
throw even other exceptions as defined by the other Getter ?

Unless you come up with solutions to these points, I fear that
wrapping nested Exceptions in a GetterException with all its
disadvantages like inability of the Compiler to enforce my
thorough handling of all and only the relevant "rootCauses"
is still the only feasible way.

 

[Harold Yarmouth]

That's the interesting question, I think: whether it is possible with
a slight change to how generics work, that is, whether it's possible
given erasure.

Sure it is, since the changes I suggested only matter at compile time,
affecting which checked exceptions the compiler considers "get" able to
throw.

try
{
meth();
}
catch (RuntimeException ex)
{
// No need to generate this if RuntimeException is already
caught
throw ex;
}
catch (IOException ex)
{
// No need to generate this if IOException is already caught
throw ex;
}
catch (Exception ex)
{
// No need to generate this if Exception is already caught.
The compiler will prevent unexpected exceptions from being
// rethrown.
throw new UnexpectedGenericException(ex, IOException.class);
}

I don't think any of that is really necessary. If you had

void myMethod (bar) throws SAXException {
...
try {
foo = cache.get(bar);
} catch (IOException e) {
do whatever
}
...
}

you'd get the same bytecodes as ever. Catch is based on run-time type,
not compile-time type; when an exception is thrown, the JVM unwinds the
stack until it finds the first catch clause that matches the run-time
type of that exception. This means it won't work to *catch* a generic
exception, but that there's no problem with *throwing* (or rethrowing)
one. So catch (E e) won't work (and I'm not proposing to change that)
but catch (IOException e) will, even if the compiler didn't know it was
going to be an IOException at the throw, just an E.

This is ALREADY going on with runtime exceptions and errors. The
compiler doesn't know which of these may actually be thrown from most
methods (say, a rt.jar method the source for which isn't on the system),
and many of the Errors in particular can theoretically pop up anywhere,
but a "catch" will catch its target regardless.

Remember too that checked exceptions, as such, exist only at compile
time; at runtime there's just throwables.

So the above code would propagate SAXException, catch IOException, and
propagate all other exceptions. Including if you fooled generics, and it
threw AWTException. This can already happen in some other cases; there
are several places where undeclared checked exceptions can be generated
in present-day Java. Reflection-using code is definitely one of them and
JNI code is probably another.

If you're proposing that this possibility should be abolished, and all
places that can generate unexpected types of checked exception should
wrap them in a RuntimeException type:

1. That's really rather silly. The "escaped" checked exception will
generally bubble up to the top of the call chain and cause a stack dump
anyway, probably with more information than would be produced by your
proposed RuntimeException. The only exception is if the "escaped"
checked exception bubbled up into a routine that could generate it. Ex:

public void someMethod () throws IOException {
...
doSomething();
...
in = new FileInputStream(foo);
try {
...
} finally {
in.close();
}
...
}

and an unexpected IOException comes up out of doSomething (which
shouldn't really throw that) rather than the FileInputStream constructor
or the other I/O in the body of the try block. In this case,
IOExceptions from the expected sources and the unexpected source all
reach the catch block some few levels higher up. Even then, it can be
presumed that "someMethod" did fail in an I/O error manner, even if in
an unexpected way, so the program is actually quite likely to behave
correctly anyway.

2. If you wanted to do this anyway, the best way would be if every
method that called JNI, reflection, or a method with a generic throws
(and we ALREADY HAVE methods with generic throws, just not with a
varargs-like generic throws), and that didn't have generic throws
itself, was wrapped by the compiler in an implicit additional try ...
catch block like yours above, with catch (RuntimeException) rethrow,
catch (DeclaredException 1) rethrow, catch (DeclaredException 2)
rethrow, ..., catch (Exception e) wrap-and-rethrow. (With
"DeclaredException 1" etc. being whatever checked exceptions are
declared in that method's "throws") Then an unexpected checked exception
will bubble up to a method like described above, get wrapped, and
continue on. The doSomething() above will launch a RuntimeException (or
Error) with a wrapped IOException, which won't be caught by its caller,
unlike the IOExceptions from the input stream usage. Though this will
probably crash the program, it will alert the developer to the presence
of a bug in his generics code. Or reflection, or native code, or
whatever else.

The interesting thing is that "varargs generic throws" doesn't change
anything about the above. A simple

public class Foo<E extends Exception> {
public void method () throws E {
...
}
}

(already legal Java) already raises the issue you describe. The runtime
will catch exceptions based on their run-time type so generification
doesn't make it harder to catch the thrown Es appropriately. And someone
could use unsafe casts to assign a Foo<IOException> to a
Foo<SAXException> reference and use it in doSomething(), resulting in an
unexpected IOException arising where the compiler thought only
SAXException, unchecked exceptions, and errors could occur.

So if there is a problem here, it already exists, rather than being a
problem with my proposal.

That is, I think this feature really could be added without any
significant changes in how generics works.

Indeed.

 

[Harold Yarmouth]

I don't think, that "slight change" is an appropriate description
for what would need to be done for this to work.

I don't agree. It's a simple matter for the compiler to treat

new Cache<Key, Value, Exception1, Exception2>

as having a get method that can throw Exception1, Exception2.

The exception list is a property of each method, not of the whole
class. (Interfaces with just one method come up frequently, but
are technically just one not-in-any-way-special case.)
Generics parameter lists, however are for the whole class.

This does mean that if you have

public class MyClass<K,V,E extends Exception...>

your methods either throw E or don't, and can't be specified as throwing
a subset of E. If you have two methods that will each have their own
separate exception lists, you won't be able to give them both separate
generic "varargs" lists. One would have to have a non-varargs list or
the same varargs list as the other.

I'm not saying my proposal would solve every such case. Just that it
could solve many of them. In the vast majority of cases, you'd want to
use variable exception lists when you had a callback object that might
want to do I/O or other not-just-plain-computation activities. It's
likely that in the vast majority of cases, the exception list will be
the same for every method of the callback object that can throw checked
exceptions, and therefore for every method of the callback-using class
that uses the callback. For example, a callback that unmarshals XML will
probably be able to throw both SAXException and IOException from
whatever methods can throw either.

How would you (discussing proposed syntax here) declare, that
Cache's get() throws all these exceptions as inferred from the
used Getter-implementation, but some sooner or later added
"flush()" doesn't

public void flush() { // Note: no throws clause
...
}

and that some further (fictive) method
"getAndCacheFromSomewhereElse(Getter<...> g, KK k)" may
throw even other exceptions as defined by the other Getter ?

public <...,EE extends Exception> V
getAndCacheFromSomewhereElse(Getter<...,EE>) throws E, EE {
...
}

 

[Harold Yarmouth]

it is very annoying to carry on a technical conversation
when someone keeps insisting on taking every comment as a
personal attack.

I only take comments that contain personal attacks as personal attacks.
Basically that means namecalling and patronizing, confrontational-toned
posts like "you're doing it wrong!"

So, "Harold"

There's no need to put quotation marks around my name. It's my real name.

 

[Andreas Leitgeb]

This does mean that if you have
public class MyClass<K,V,E extends Exception...>
your methods either throw E or don't, and can't be specified as throwing
a subset of E. If you have two methods that will each have their own
separate exception lists, you won't be able to give them both separate
generic "varargs" lists. One would have to have a non-varargs list or
the same varargs list as the other.

That does make sense. I stand corrected.

public void flush() { ... // Note: no throws clause }
public <...,EE extends Exception> V
getAndCacheFromSomewhereElse(Getter<...,EE>) throws E, EE {
...
}

ok, flush was easy in hindsight, but generic methods still beat me.

Getter<String,Long,IOExeption,FubarException> g1 =
new MyIOAndFubarExceptionThrowingStringLongGetterImplementationClass(...);
Getter<String,Long,SnafuExeption> g2 =
new MySnafuExceptionThrowingStringLongGetterImplementationClass(...);
Cache<String,Long,IOExeption,FooBarException> c =
new Cache<String,Long,IOException,FooBarException>( g1 );

try {
... c.get("foo") ... // easy
... c.<???>getAndCacheFromSomewhereElse(g2,"bar") ... //hmm..
} catch(IOExeption ioe) { // Compiler would barf without these three catches.
...
} catch(FubarException fe) {
...
} catch(SnafuException se) {
...
}

For the implementation of Cache's get(), how would that work with
respect to calling the interface-method?
Cache.get() would probably have a "throws Exception" declaration
and thus not be able to have the Compiler check for other Exceptions
thrown in Cache.get()s body. Due to erasure inside the generic
class, the compiler doesn't know which exceptions are going to be
declared outside, and which not.

e.g.
V get(K k) throws Exception
{
...; // check own cache first and return hit, if any
V vg= getter.get( k ); if (vg == null) return null;

V vc= vg.clone(); // make sure it doesn't change later...
// compiler will not notice possible CloneNotSupportedException
// (ok, one could discuss the use of clone() here, but anyway,
// that's just an example of having other code inside get()
// that may make get() throw checked but uncaught exceptions
// whereas to the outside "get()" looks like throwing only
// those declared Exceptions from the Getter, the list of which
// the compiler doesn't yet know while compiling this)

cache.put(k, vc );
return vc;
}

PS: I'm playing devil's advocate here. I'd love to see how this can be
actually solved (just not talked small). I'd be more surprised if it
couldn't.

PPS: Now, after writing it up I think I even know a solution, but it's
a bit awkward, so I hope you independently find a better one.

 

[Andreas Leitgeb]

That does make sense. I stand corrected.

Please forget the other stuff I wrote (and not quoted here). As I
saw from the other followup: that would work all fine. Indeed all
the really complicated parts are already done well by the compiler.

Have you considered writing up a JSR?

 

[Harold Yarmouth]

Please forget the other stuff I wrote (and not quoted here). As I
saw from the other followup: that would work all fine. Indeed all
the really complicated parts are already done well by the compiler.

Have you considered writing up a JSR?

Eh ... I'm hardly qualified. I'm just Joe Java Programmer here.

 

[Daniel Pitts]

Eh ... I'm hardly qualified. I'm just Joe Java Programmer here.

You're as qualified as anyone. You've come up with a working solution,
so you're more qualified than some

 

[Harold Yarmouth]

You're as qualified as anyone. You've come up with a working solution,
so you're more qualified than some

Thanks. But design is one thing, implementation quite another. Also, I
don't sit on whatever committees you need to sit on to get the ears of
whoever I'd need to get the ears of ...