Why is that in JDK8: value used in lambda expression shuld beeffectively final?

S

Steven Simpson

I can't speak for the previous poster. But clearly, closures don't do
anything you _can't_ do using lambdas or even using some other technique.

The point is convenience, expressiveness, readability, and other code
authoring and maintenance such as may be of concern are affected. In the
view of closure proponents (such as my own), these concerns are addressed
in a positive way when closures are available.

Indeed. I'm interested in seeing how much expressiveness (etc) is lost
when one is forced to work around the lack of a feature, by looking at
specific cases.

But that's the point: lambdas, closures, etc. offer a more concise,
expressive way of representing certain algorithm approaches. In this
context, I can confirm that there are plenty of examples of situations
where a true closure offers a more convenient, improved way of achieving
the same desired result, as compared to language syntax without closures.


Do these examples fall into just a couple of categories, in terms of the
guarantees that the user of a function object makes to the provider
about how it will be invoked?:

1. The object will be invoked on the same thread that provided it (e.g.
in an event-driven environment)
2. (1) + the object will not be invoked once the call that provided it
has completed (e.g. in control abstraction)

....or are there more, especially ones which make fewer or alternative
guarantees?

I'm genuinely interested to see if there are more categories. Then, if
they are limited, could a set of type qualifiers express these
guarantees formally, and allow compilers to progressively switch on
features like mutable local capture and various transparencies safely?
 
S

Saxo

Am Freitag, 4. Januar 2013 22:32:38 UTC+1 schrieb Steven Simpson:
On 03/01/13 14:45, Saxo wrote:

I'm still interested in seeing an example that you would normally write
with free variables, demonstrating that lambdas are inadequate for the
things you'd like to do.

Groovy:

def list = [1, 2, 3]
def sum = 0
list.each { sum += it }
println(sum)

Kotlin:

val list = arrayListOf(1, 2, 3)
var sum = 0;
list.forEach { i -> sum += i }
println(sum)

Scala:

val list = List(1, 2, 3)
var sum = 0
list.foreach(sum += _)
println(sum)

Works in all these languages that support closures (and I guess in any other language that supports closures as well), but the same does not work withJDK8 lambdas.

Reality can be tough... There is always the workaround of defining a singleelement array final:

final int sumArray[] = new int[] { 0 };
ints.forEach(i -> {sumArray[0] += i;});
println(sumArray[0]);

I wrote in my blog:

The lambda specification (JSR 335) also says so explicitly: "For both lambda bodies and inner classes, local variables in the enclosing context can only be referenced if they are final or effectively final. A variable is effectively final if it is never assigned to after its initialization.".

And I provided the link to JSR 335: http://cr.openjdk.java.net/~dlsmith/jsr335-0.6.1/B.html

I took some effort in my blog to describe the matter and provided the linksto the respective resources on the JDK8 lambda homepage :).

OTOH, JDK8 lambdas are still enormously useful. No doubt about that. An imense amount of pre-JDK8 boilerplate code can be thrown out of myriads of systems, frameworks and applications. It's a big step ahead. Nevertheless, it can be said that JDK8 lambdas don't go all the way to be called true closures. Apparently, JDK8 lambdas can be assigned to a Runnable. Maybe this was a deliberate choice to stay backwards compatible.

-- Oliver
 
S

Steven Simpson

Am Freitag, 4. Januar 2013 22:32:38 UTC+1 schrieb Steven Simpson:
On 03/01/13 14:45, Saxo wrote:

I'm still interested in seeing an example that you would normally write
with free variables, demonstrating that lambdas are inadequate for the
things you'd like to do.
Groovy:

def list = [1, 2, 3]
def sum = 0
list.each { sum += it }
println(sum)

Kotlin:

val list = arrayListOf(1, 2, 3)
var sum = 0;
list.forEach { i -> sum += i }
println(sum)

Scala:

val list = List(1, 2, 3)
var sum = 0
list.foreach(sum += _)
println(sum)

Sorry, but aren't these the same example that you originally gave, just
in different languages? I'm looking for different examples, perhaps
expressed in a Java-like language with the features you were hoping for.

Apparently, JDK8 lambdas can be assigned to a Runnable.

....if they take no arguments, returning nothing, and throw no checked
exceptions. Java lambdas don't have their own special function types,
and have to be associated with a SAM type as soon as they are
expressed. Type inference is expected to make this implicit in most cases.
 
S

Saxo

Am Samstag, 5. Januar 2013 12:21:55 UTC+1 schrieb Steven Simpson:
Sorry, but aren't these the same example that you originally gave, just
in different languages?

No, because it works in all these languages what with JDK8 lambdas would only work if sum were declared final.

Remember this one:

Once the JDK8 gets
Maybe I had a point there with those 95%, because discussions like this one you will have with 95 java developers out of 100...

-- Oliver
 
S

Steven Simpson

Am Samstag, 5. Januar 2013 12:21:55 UTC+1 schrieb Steven Simpson:
No, because it works in all these languages what with JDK8 lambdas would only work if sum were declared final.

I understand that they work in those languages, and that they don't work
when directly translated into Java, because Java 'closures' lack a
feature provided by the other languages.

To put it another way, if you regard your original example as written in
a fictional language similar to Java - one that supports MLC, and so
doesn't throw an error back at you - it would be equivalent to each of
the Groovy/Kotlin/Scala examples you listed.
 
S

Steven Simpson

[...]
Do these examples fall into just a couple of categories, in terms of the
guarantees that the user of a function object makes to the provider
about how it will be invoked?:

1. The object will be invoked on the same thread that provided it (e.g.
in an event-driven environment)
2. (1) + the object will not be invoked once the call that provided it
has completed (e.g. in control abstraction)

...or are there more, especially ones which make fewer or alternative
guarantees?
Honestly, I'm not aware of a lambda/closure taxonomy that uses those
categories at all. If such exists, I am ignorant of it.

That's because I'm making them up! :-D

The goal I'm trying to reason my way towards is that, since Java is
supposed to provide a degree of safety (and so mutable local capture and
other features are currently missing from its 'closures'), there ought
to be a way to signal to the compiler cases where it is safe to enable
those features. Rather than the provider of the function object
choosing when it is safe (e.g. by annotating shared variables with
@Shared, or manually boxing them in 1-element arrays), the user of the
object should choose by annotating the parameter that conveys the
object, because that user is in a position to make guarantees about the use.

First and foremost is that the "user of a function object" often has no
knowledge that a closure is being used at all. It knows it's received a
function object of some sort (e.g. a delegate instance in C#/.NET), but the
origin of that object could be varied.

That's why I used the term 'function object'.

A given API may or may not make promises regarding usage of the function
object, but these promises likely have little to do with the declaration
and implementation of the function object.

Whatever form the function object was originally expressed as, the one
who invokes it is in a position to make guarantees about how it is
invoked, and I'm suggesting that these promises could influence the
options available for expressing the function object.

Beyond that, wrt point #1: function objects, even those made from closures
with captured variables, may be safely invoked even in a multi-threaded
environment, so long as the code is written correctly. For example,
enforcing volatile or fully-synchronized access to shared variables
(whether due to capturing or otherwise).

I would expect a good proportion of these cases to end up requiring
certain objects to be created explicitly anyway, simply to have
something to synchronize on, in which case, Java's lambdas would be
adequate because the variables would explicitly be not local.

But I don't know, hence I'm asking for examples.

Wrt point #2: because variable capturing involves effectively "lifting" the
variable out of the declaring method's local context, invoking the function
object after the declaring method has completed is not a concern at all.

(That wasn't the concern particularly, though I think there have been
notions that MLC would be achieved by means other than boxing, and such
means would depend on the local's lifetime. I don't know enough about
this to elaborate.)

The idea behind #2 is that there is a class of closure use cases which
are really demanding control abstraction. If the function-object user
makes sufficient promises about how it is invoked, the compiler could
permit a control-abstraction syntax as an alternative to a lambda, in
which the code can do anything that a local block could do, including
throwing, returning, breaking, continuing.

If one can show that practically all cases with a /prima facie/ demand
for additional features from Java lambdas actually fall into categories
#1 or #2, it guides us in developing additional closure features for
Java that satisfy the conflicting demands of safety versus expressiveness.

An argument about category #1 could run something like this... One camp
demands to be able to write:

int sum = 0;
list.forEach((v) -> { sum += v; });

Another camp says, "ouch, don't like that!" because forEach might
execute the lambda in parallel. You should therefore get an error or
warning about shared access to 'sum'.

The first camp says, "okay, we'll take responsibility for 'sum' by
declaring that we know it's shared," and either boxes the variable
manually, or proposes an annotation to silence the warning or permit the
code:

@Shared
int sum = 0;
list.forEach((v) -> { sum += v; });

The problem is that this depends on whether the author has understood
the (informal) guarantees that might be provided by forEach: "This
method invokes its argument multiple times in serial."

Instead of the author of the code above deciding when it's safe to use
MLC, a serial contract for forEach should formally express the
guarantee, (say) by annotating its parameter:

void forEach(@Serial Block action);

The compiler then turns on MLC for any lambda assigned to 'action',
without having to use @Shared or explicit boxing. Of course, the
implementation has to make the corresponding guarantee to meet the
method's contract.

Alongside forEach could be parallelForEach:

void parallelForEach(Block action);

The lack of annotation means that its implementation does not have to
make the guarantee.

The provider of a lambda now has a choice between a potentially parallel
implementation, which won't allow MLC, and a guaranteed serial one,
which will.

The no-MLC camp is happy, because they can provide parallelForEach
exploiting parallelism, and know that the user won't inadvertently mess
about with locals unsafely. The MLC camp is happy, because they can use
forEach to mutate their locals safely.

Some obvious holes include:

* A method accepting a @Serial Block could fail to make the serial
guarantee, and the compiler would not be able to check it. However,
I don't think that's much worse than the compiler not being able to
check that (say) a Comparator implementation isn't returning random
values.
* I'm using annotations as if they are some sort of type qualifier,
which I doubt is a valid way to regard them. Not sure.
 
S

Saxo

Having to declare the free variable of lambda expression final has also it's advantages. This here is taken from the JDK8 lambda FAQ (http://www.lambdafaq.org/what-are-the-reasons-for-the-restriction-to-effective-immutability):

"The restriction on local variables helps to direct developers using lambdas aways from idioms involving mutation; it does not prevent them. Mutable fields are always a potential source of concurrency problems if sharing is not properly managed; disallowing field capture by lambda expressions would reduce their usefulness without doing anything to solve this general problem."

Question is whether this is intentional or a pretext. Unhappily, it also does not matter since this restriction can be circumvented anyway with the single-element-array trick:

final int sumArray[] = new int[] { 0 };
ints.forEach(i -> {sumArray[0] += i;});
println(sumArray[0]);

Whether you declare sumArray final here or not makes no difference in the matter.

Your idea with @Share is interesting. There is f.ex. @Immutable in Groovy. In Groovy you can define those things like your @Share annotation through AST transformations and this way make additions to the language. You can also define such AST transformations in Scala where they are called macros.

If you are interested in those things Groovy or Scala may be worth a look :).
 
R

Robert Klemme

It's the same reason why inner classes can't mutate local variables in
the enclosing scope. The method that accepts the lambda or inner class
instance cannot formally make any guarantee not to invoke the supplied
code in a non-serial way.

I don't think concurrency is the main reason. It has more to do with
the complexity involved in keeping the surrounding scope (local
variables on the stack) alive for an _indefinite amount of time after
the invocation of a method_. For mutations of references or POD values
in the surrounding scope this scope would need to stay around for as
long as the inner / anonymous class instance lives. In this case

public Runnable sample(int count) {
return new Runnable() {
public void run() {
while ( count > 0 ) {
--count;
}
}
};
}

variable "count" would need to be assignable at least as long as the
created instance lives. Same for any local variable declared inside the
method body. If it weren't we would not have proper closure semantics
(e.g. when returning two instances which refer the same variable from
the scope). Since the scope need to be kept for longer, it cannot
reside on the stack. That would introduce two different ways of local
variable access: one via the stack and one via some reference on the
heap. I guess since variable access is such a fundamental feature
language designers did not want to introduce this type of complexity.
(There could also be performance issues lurking.)

With "final" variables it's easy: just copy all used values or
references in hidden member variables of the created instance and be
done. Because original variables are final the scope does not need to
be kept around: since state of variables in the scope cannot change both
values cannot digress. And actually this is what is happening, as you
can see when disassembling code.

Example at github because the disassembly output is quite long.
https://gist.github.com/4478864

The member is declared in line 140 and the constructor in 146. Also you
can see that the value of this field is used in line 195.

This does not change in Java 8 because compatibility with old code was
intended. lambdas are merely syntactic sugar for "old" Java features.
For anyone interested in the details I recommend to read

http://cr.openjdk.java.net/~briangoetz/lambda/collections-overview.html

Kind regards

robert
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,054
Latest member
TrimKetoBoost

Latest Threads

Top