Closures in Java
Project Lambda has so far (2011-07-11) defined the SAM concept (single abstract method), and produced method references, exception transparency, defender methods and lambda expressions.
Method references
Method references help in reducing boiler-plate code in cases like this:
Runnable action = new Runnable() {
public void run() {
obj.meth();
}
};
…to simply this:
Runnable action = obj#meth;
Because Runnable is a SAM type,
i.e. it has exactly one abstract method, Java can build the
anonymous class itself, and know how to implement that method.
The example above binds the action object to obj. Alternatively, you might prefer to
leave this open:
interface FooRunnable {
void run(Foo base);
}
FooRunnable action = #meth;
// same as
FooRunnable action = new FooRunnable() {
public void run(Foo base) {
base.meth();
}
};
You can also reference static methods, e.g. Foo#meth, or interpret #meth as this#meth, according to context. If
necessary, you can disambiguate overloaded methods by specifying a
parameter signature, e.g. #meth(int,
String).
My preferred syntax
The syntax hasn't been under debate much lately, but I'll mention
that I think a cleaner syntax would permit only an expression (not
a type) to the left of #, and
put a method identifier on the right, with sometimes optional class
and package prefixes:
package org.example;
import com.example.Bar;
class Foo {
Runnable run;
Bar bar;
interface XRunnable<T> {
void run(T t);
}
static void meth1() { }
void meth2() { }
static void ex1() {
// static method Foo.meth1
run = #meth1;
run = #Foo.meth1;
run = #org.example.Foo.meth1;
// static method com.example.Bar.meth1
run = #Bar.meth1;
run = #com.example.Bar.meth1;
run = bar#meth1;
// instance method com.example.Bar.meth2
run = bar#meth2;
}
void ex2() {
// this.meth2
run = #meth2;
run = #Foo.meth2
run = #org.example.Foo.meth2;
run = this#meth2;
run = this#Foo.meth2;
run = this#org.example.Foo.meth2;
// instance method Bar.meth2
XRunnable<Bar> barRun = #Bar.meth2;
}
}
The current syntax appears to be:
MethodReference:
Primary? "#" Identifier MethodReferenceParameters?
ClassName "#" Identifier MethodReferenceParameters?
Here's the bit of grammar I'm proposing:
MethodReference:
Primary? "#" MethodName MethodReferenceParameters?
I can think of three logical arguments for this syntax:
-
It makes for simpler parsing. I have a niggling worry that authors of parser writers other than the compiler will find other syntaxes rather burdensome, since those authors have additional requirements such as being able to parse a fragment of source, or being able to recover from errors. Perhaps the worry is unfounded.
-
Language features should be as orthogonal as possible. This should be reflected in the syntax too, and a simpler syntax change introducing one new feature is more likely to be found compatible with other new features than a more complex syntax change.
-
It follows the same pattern as
new. Consider these declarations:public class Outer { public static class Nested { } public class Inner { } }How do you create an
Inneror aNested? Normally, you'd do it insideOuter, so the syntax is the same:// Anywhere in Outer Nested b = new Nested(); // Inside instance method of Outer Inner a = new Inner();
But if you're outside
Outer, you must qualify the nested class's name, or qualify the context of the operation that creates the inner instance:// Outside Outer Outer.Nested b = new Outer.Nested(); // Outside any instance method of Outer Outer ex = new Outer(); Outer.Inner a = ex.new Inner();
Of course, in the right context, these are equivalent:
// Inside instance method of Outer Inner a = new Inner(); Inner b = this.new Inner();
So, in general, the link-time information is kept to the right of
new, while the run-time information is kept to the left, or is blank. The same can be said of the simpler method-reference syntax above.
Lambda syntax
Until 2011-09-08, The lambda syntax accepted the following forms:
# { params -> expr }
# { params -> statements }
# { -> expr }
# { -> statements }
Each of these forms is an expression, but it must be associated with a SAM type somehow, i.e. by assigning to a SAM-type reference variable, passing as a SAM-type argument, or explicit casting:
// assignment/initialization
Runnable action = # { -> System.out.println("hello") };
// argument (implicit Comparator<String>)
List<String> list = ...;
list.sort( # { a, b -> -a.compareTo(b) } );
My preferred syntax
A few things bother me about this syntax, and many of the proposed alternatives:
-
One is that the braces go around the whole expression. If possible, I would prefer not to have any kind of parenthesis spreading so far, simply because it is more visually challenging to keep track of. Of course, IDEs help sort this out, but you still have to understand what the IDE is complaining about.
-
Another irk is that the statement form and the expression form of the body are not distinct. In the rest of Java code, this interchangability doesn't exist, so why allow it here? Will a learner find it frustrating that it is possible in a lambda, but not elsewhere? I'd prefer that braces imply a block statement.
-
Thirdly, this syntax, and some alternatives, propose an infix punctuator such as
->. Inside the braces in the current syntax, it serves to separate the parameters from the body, and something certainly must exist to do that. But other proposals have no outer bracketing, so you end up needing them, around the parameters at least, to disambiguate the parameter separator from an argument separator when the lambda is in an argument list. I suspect that parser writers would prefer to know sooner that they are dealing with a lambda expression, especially if their parser is for something that shouldn't need to get the total meaning of the code, e.g. a syntax highlighter or formatter. Again, I'm just supposing about the needs of parsers.
Taking those into consideration, I'd much prefer to see these forms:
// nilary with expression for body
#() 6
// nilary with statement for body
#() { doSomething(); }
// binary with expression for body
#(x, y) x + y
This addresses each of my concerns:
-
The range of any bracketing is minimised. Round brackets surround the parameter list, which will probably be short anyway. Braces surround only the body, rather than the whole lambda, and only if it's a statement. No additional bracketing is required for an expression body.
-
Expression and statement forms are distinct. Expressions never use braces, while statements always use them.
-
Finally, a lambda is immediately discovered when
#(is encountered.
Looking at it another way, the parameter terminator is now a closing bracket, fulfilling the role of the arrow -> in other proposals. Since that would look odd without its opening counterpart, the counterpart is also required, and serves with the hash as an early discriminant.
Furthermore, even though the syntax looks rather open-ended (especially in expression form), there aren't many cases of ambiguity, and these are rather contrived anyway. As an argument, the following comma or closing bracket of the argument list terminates the lambda:
process(a, #(x, y) x - y, b, c);
As a direct assignment, the semicolon terminates the lambda:
Runnable action;
action = #() {
System.out.println("Hello!");
};
If you really must declare a lambda only to invoke it immediately, you need brackets, and a cast too (unless there are function types):
((Runnable) #() { System.out.println("Hello!"); }).run();
Or you might want to add a lambda to a string:
String r =
((Runnable) #() { System.out.println("Hello!"); }) + "yes";
But really, bearing in mind that such forms are still possible, and how rarely they are useful, is the little extra typing such a problem?
With minimal bracketing, there are some short currying forms:
interface Mapper<F,T> {
T map(F from);
}
Mapper<int, Mapper<int, int>> f = #(x) #(y) x + y;
An alternative syntax which I'd find acceptable would drop the parameter brackets in the unary and nilary cases, and separate the parameters from the body with the arrow ->. For example:
process(a, x -> x * x, b, c);
Runnable action = -> {
System.out.println("Hello!");
};
Mapper<int, Mapper<int, int>> f = x -> y -> x + y;
However, I'm not keen on the late discriminant, which still requires parameter brackets for 2+-ary cases when in an argument list or an initializer:
// Is x an argument of process? process(a, x, y -> x - y, b, c); // Is x assigned to cmp? Comparator<String> cmp = x, y -> x.compareToIgnoreCase(y);
Still, the currying is nice, and -> is less jarring on the eyes than #.
As of 2011-09-08, this is the chosen syntax for Java lambdas. I can live with that, but it turns out that it's not as elegant as hoped. Nilary lambdas require brackets around the empty parameter list, to avoid confusion with casts.
To compare syntaxes, have a look at my Lambda Syntax Example Generation Tool.
Full closures
Lambdas only have a few transparencies. They can see local
variables in the enclosing scopes only if those variables are
final (or possibly ‘effectively
final’), and cannot modify them:
final int comp = a + b;
int oof = 10;
int wibbly = 10;
process(() -> comp); // okay, final
process(() -> oof); // okay, effectively final
process(() -> wibbly); // error
exec.submit(() -> { wibbly++; }); // error
wibbly++;
Are these lambdas closures?
These lambas are not what you might call ‘full closures’, because a
return within the lambda would
only exit from the lambda, not the enclosing scope. Similarly, you
can't break or continue, nor throw a checked exception not
declared on the SAM-type method. And the lambda can't mutate local
variables.
This is not a terribly bad thing, because part of the drive for
lambdas is to support concurrent APIs like map/reduce, that can take advantage of multiple
processors. These take lambdas as behaviour-specifying arguments,
but do not guarantee to invoke them in the same thread that calls
them. So it would be bad if a lambda started mutating thread-unsafe
variables, or tried to force map to return twice, or
return after it had already returned to its caller.
But you do need these other transparencies for one of the other goals of closures: control abstraction (CA). Having these is at odds with the concurrency benefits of map/reduce, so a user-defined CA implementation (CAI) should promise not to execute any provided blocks, except on the stack on which the CAI is invoked, and not after returning to its caller. This means that a CAI must not store a block for later execution, for example.
<throws E>
Lambdas can exploit the new throws syntax for generics to get exception
transparency:
interface Action<throws E> {
void act() throws E;
}
static <throws E> void doStuff(Action<E> action) throws E;
final InputStream in = ...;
final Collection<Character> coll = ...;
try {
doStuff(() -> { coll.add((char) in.read()); });
} catch (IOException ex) {
...
}
Because the lambda can throw IOException, the invocation of doStuff is inferred to be able to throw it
too.
What do I think of <throws
E>?
I think that's quite neat, but can it be much use on concurrent
APIs? What does it mean if a method invokes a supplied lambda twice
in parallel, and both throw an exception? Surely that means the
method must resolve them into a single one, to be thrown back out
to the caller. But then it must have prior knowledge of those
exeception types in order to resolve them, so it can afford to
declare them explicitly, instead of using <throws E>.
It must be useful for CA, but I will show a way that makes it redundant.
Providing full transparency
So, how can we go about providing full transparency for CA? As an example to work with, we'll imagine that the for-each loop in Java was never introduced as a language feature, and we're providing it now as a method. Here's a user of that method:
int someContext(List<List<String>> dataList)
throws DodgyException {
int index = 0;
outer:
forEach(dataList) : (List<String> data) {
forEach(data) : (String datum) {
if (datum.equals("yes"))
return index;
if (datum.equals("no"))
throw new DodgyException();
if (datum.equals("comment"))
continue;
if (datum.equals("skip"))
continue outer;
index++;
}
}
return index;
}
This exercises most of the transparencies we're interested in. The
author intends that the return
statement should break out of someContext altogether, and the exception
should similarly break out too. There is a local continue which must be signalled to inner
invocations of the CAI, and a labelled continue which must be signalled to the
outer one.
The author of the forEach method
will want it to look something like this:
interface ForEachBlock<T> {
void invoke(T value);
}
static <T> void forEach(Iterable<T> coll,
ForEachBlock<T> block) {
for (Iterator<T> iter = coll.iterator(); iter.hasNext(); ) {
T elem = iter.next();
block.invoke(elem);
}
}
Somehow, the invoke call must
detect attempts to continue or
break on the loop that
forEach implements. However, it
must also pass through any attempts to continue or break on the outer loop, and
must do something similar for returned and thrown values. Bear in
mind that the author of forEach
has no prior knowledge of the blocks it will be given, so it cannot
know return type, exception types, or outer labels.
How can we get the compiler to deal with each of the required transparencies?
-
Mutable access to local variables
This one is easy. Indeed, we could use this solution to circumvent restrictions on lambdas, unwise as that would be. All we have to do is place each variable that needs to be accessed inside an object, and provide a
finalreference of that object on to the actualForEachBlockinstance submitted to the CAI. -
Long returns
Given the implementation of
forEach, there's only one way for control to exit that method without the method being coded specifically for it — throw an unchecked exception. The exception could carry the return value if necessary, or the return value could be assigned to a generated local variable hidden from the programmer. In the latter case, the exception simply serves to transfer control, but no data. -
Local
continueandbreakThese signals specifically have to be picked up by the CAI, and interpreted in a CAI-dependent way. Let's use a special exception (as we plan to for
return), and require the CAI to catch it explicitly. -
Long
continueandbreakAgain, we'll use an exception to transfer control back through the CAI and into the call site. The difference here is that there's no need for the CAI to catch this one.
-
Checked but undeclared exceptions
We could use the existing exception-transparency mechanism here, namely a
<throws E>type parameter, but that's used in conjunction with the regularthrowsclause of a method, and I'd like to save that for something else. We'll just use a special exception to transfer control as before, and we can keep a local variable to pass the real exception, just as we intend for long returns.
We have to be careful in using exceptions, as the nature of loops
is that their bodies will be invoked many times, and the cost of
creating a special exception just to emulate the behaviour of
continue could be great. Note
that the greatest cost is in creating the exception, not throwing
it, so we will try to prepare static exception objects where
possible.
Let's deal with mutable local variables first. We'll create a local class to hold the variables our closures need to access, and create an instance of it:
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
}
final $closure$1 $closure$1$inst = new $closure$1();
outer:
forEach(dataList) : (List<String> data) {
forEach(data) : (String datum) {
if (datum.equals("yes"))
return $closure$1$inst.index;
if (datum.equals("no"))
throw new DodgyException();
if (datum.equals("comment"))
continue;
if (datum.equals("skip"))
continue outer;
$closure$1$inst.index++;
}
}
return $closure$1$inst.index;
}
Now let's turn the inner loop into a normal call to a method taking
a method reference for the loop body. Since the loop body accesses
the local variable index, and
that variable has now become an instance member of our hidden
closure class, we'll change the loop body into an instance method
of the hidden closure class, and pass a reference to it:
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
void $body$1(String datum) {
if (datum.equals("yes"))
return index;
if (datum.equals("no"))
throw new DodgyException();
if (datum.equals("comment"))
continue;
if (datum.equals("skip"))
continue outer;
index++;
}
}
final $closure$1 $closure$1$inst = new $closure$1();
outer:
forEach(dataList) : (List<String> data) {
forEach(data, $closure$1$inst#$body$1(String));
}
return $closure$1$inst.index;
}
Of course, those control statements are no longer valid within that method, so each has to be translated:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
}
final $closure$1 $closure$1$inst = new $closure$1();
outer:
forEach(dataList) : (List<String> data) {
forEach(data, $closure$1$inst#$body$1(String));
}
return $closure$1$inst.index;
}
We see two new classes here. java.lang.closure.Exit has several static
methods, each throwing an unchecked exception for different
circumstances. A static instance of java.lang.closure.Label is created to act as
a reference point for the outer
label (now technically redundant), and an unchecked exception that
refers to it can also be thrown.
Note that the site-specific exception, DodgyException, has been given a placeholder
so that an instance of it can be retained when the block exits. We
have done the same for the returning of index.
The static methods of Exit are
guaranteed to throw something when invoked. Since this guarantee is
not expressed in the methods' signatures, we've thrown an
UnreachableError after each call
to help with escape analysis. UnreachableError could be defined as
follows:
package java.lang;
public class UnreachableError extends AssertionError {
public UnreachableError() { super("unreachable"); }
}
Exit.continuing() will throw an
exception of type Continue, and
Exit.breaking() will throw an
exception of type Break. These
are signals which the CAI must intercept, and are the only
additional complexity it has to deal with:
static <T> void forEach(Iterable<T> coll,
ForEachBlock<T> block) {
for (Iterator<T> iter = coll.iterator(); iter.hasNext(); ) {
T elem = iter.next();
try {
block.invoke(elem);
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
}
This complexity is appropriate. The forEach method is intended to implement an
iterative control structure, and so its author must understand
break and continue signals directed at it. These
cannot be translated automatically without pre-supposing that they
only ever translate into the form above — and the whole point of
supporting control abstraction is to leave these decisions up to
the programmer.
The other signals will pass through and reach the call site (the body of the outer loop), so we must catch them there:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
}
final $closure$1 $closure$1$inst = new $closure$1();
forEach(dataList) : (List<String> data) {
try {
forEach(data, $closure$1$inst#$body$1(String));
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
if (ex.isContinuingTo($label$1))
continue;
throw new java.lang.closure.ClosureError(ex);
}
}
return $closure$1$inst.index;
}
The thrown exception type belongs to the Exit hierarchy, and particular subclasses
exist for signalling return,
throw, break and continue. A plain Exit is used for a long jump (the
continue outer;), and we could
have a similar one for break
outer;. We have clauses only for the subset of signals that
we know our loop body can generate. If we get an unexpected
Exit instance thrown, we raise a
run-time error — which is appropriate, since it should never
happen. We don't expect a Break
or Continue, as these are always
intended as signals to the CAI, not to the call site.
Now let's apply the same transformations to the outer loop. We already have an object for local variables, so we re-use it. We just need to create another method on the local class, and move the loop body into it:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
void $body$2(List<String> data) {
try {
forEach(data, #$body$1(String));
} catch (java.lang.closure.Return ex) {
return $return$1;
} catch (java.lang.closure.Throw ex) {
throw $throw$1;
} catch (java.lang.closure.Exit ex) {
if (ex.isContinuingTo($label$1))
continue;
throw new java.lang.closure.ClosureError(ex);
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
forEach(dataList, $closure$1$inst#$body$2(List<String>));
return $closure$1$inst.index;
}
Translate control statements within the second body:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
void $body$2(List<String> data) {
try {
forEach(data, #$body$1(String));
} catch (java.lang.closure.Return ex) {
$return$1 = $return$1;
java.lang.closure.Exit.returning();
throw new UnreachableError();
} catch (java.lang.closure.Throw ex) {
$throw$1 = $throw$1;
java.lang.closure.Exit.throwing();
throw new UnreachableError();
} catch (java.lang.closure.Exit ex) {
if (ex.isContinuingTo($label$1)) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
throw new java.lang.closure.ClosureError(ex);
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
forEach(dataList, $closure$1$inst#$body$2(List<String>));
return $closure$1$inst.index;
}
For completeness, redundant assignments of return values and thrown exceptions have been included.
Finally, we need to catch the exceptions in the call site:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
void $body$2(List<String> data) {
try {
forEach(data, #$body$1(String));
} catch (java.lang.closure.Return ex) {
$return$1 = $return$1;
java.lang.closure.Exit.returning();
throw new UnreachableError();
} catch (java.lang.closure.Throw ex) {
$throw$1 = $throw$1;
java.lang.closure.Exit.throwing();
throw new UnreachableError();
} catch (java.lang.closure.Exit ex) {
if (ex.isContinuingTo($label$1)) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
throw new java.lang.closure.ClosureError(ex);
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
forEach(dataList, $closure$1$inst#$body$2(List<String>));
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return $closure$1$inst.index;
}
We can simplify by recognising that the inner closure's
try is almost redundant. Almost
everything it throws could actually be a rethrow of what it just
caught, and we know that an outer scope will catch any errant
Exits:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
void $body$2(List<String> data) {
try {
forEach(data, #$body$1(String));
} catch (java.lang.closure.Exit ex) {
if (ex.isContinuingTo($label$1)) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
throw ex;
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
forEach(dataList, $closure$1$inst#$body$2(List<String>));
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return $closure$1$inst.index;
}
We could take the run-time checking a step further, and ensure that the CAI does not call our code after it is out of scope:
private static java.lang.closure.Label $label$1 =
new java.lang.closure.Label();
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
boolean $body$1$enter;
void $body$1(String datum) {
if (!$body$1$enter)
throw new java.lang.closure.ClosureError();
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
java.lang.closure.Exit.continuingTo($label$1);
throw new UnreachableError();
}
index++;
}
boolean $body$2$enter;
void $body$2(List<String> data) {
if (!$body$2$enter)
throw new java.lang.closure.ClosureError();
try {
$body$1$enter = true;
forEach(data, #$body$1(String));
} catch (java.lang.closure.Exit ex) {
if (ex.isContinuingTo($label$1)) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
throw ex;
} finally {
$body$1$enter = false;
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
$closure$1$inst.$body$2$enter = true;
forEach(dataList, $closure$1$inst#$body$2(List<String>));
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
} finally {
$closure$1$inst.$body$2$enter = false;
}
return $closure$1$inst.index;
}
Each body gets a flag which is set just before the CAI can invoke it, and cleared just after. The method carrying the body checks the flag, and raises a run-time error if false, implying the body is out of scope. Alternatively, these could be tested as assertions, and thus be disabled most of the time. (Could that cause the boolean assignments to be optimized away too?)
Efficient throwing of exceptions
Is it efficient to throw a new exception in the body of a loop,
just to signify (say) continue?
In my experience, no. However, the overhead is mostly in the
creation of a new object each time, and hardly at all in the
throwing itself. That's why generated calls to Exit methods are used: they return re-usable
objects to be thrown. Here's a definition of Exit:
package java.lang.closure;
public abstract class Exit extends Throwable {
// Only this package can create Exits.
Exit() { }
private static final RETURN = new Return();
public static void Return returning() {
throw RETURN;
}
private static final Throw THROW = new Throw();
public static void throwing() {
throw THROW;
}
private static final Break BREAK = new Break();
public static void breaking() {
throw BREAK;
}
private static final Continue CONTINUE = new Continue();
public static void continuing() {
throw CONTINUE;
}
public static void continuingTo(Label label) {
throw label.CONTINUE;
}
public static void breakingTo(Label label) {
throw label.BREAK;
}
public boolean isContinuingTo(Label label) { return false; }
public boolean isBreakingTo(Label label) { return false; }
}
The four types Return,
Throw, Continue and Break each extend Exit, but add no new methods. Singletons are
created for each one, and are re-used.
Labelled jumps require the Label
class:
package java.lang.closure;
public class Label {
final Exit CONTINUE = new Exit() {
public boolean isContinuingTo(Label label) {
return label == Label.this;
}
};
final Exit BREAK = new Exit() {
public boolean isBreakingTo(Label label) {
return label == Label.this;
}
};
}
The Label is really just a
holder for a couple of Exit
instances. Since generated code creates only static instances of
Labels, these Exit are effectively static too, and are
re-used as often as the code for which they were created is
invoked.
Other signals to the CAI
Break and Continue are signals from the body of the
loop to the control-abstraction implementation. If the CAI was not
for a loop, such signals would not make sense, and so should not
appear in the generated code.
Could a CAI define other ways for the CA body to signal to it? There are two obvious possibilities: return values and exceptions. We'll demonstrate both in a single (contrived) example:
class ContrivedException extends Exception;
interface ContrivedBlock<T> {
int invoke(T arg, int rem) throws ContrivedException;
}
static <T> void contrivance(Iterable<T> coll,
ContrivedBlock<T> block) {
Iterator<T> iter = coll.iterator();
int delay = 0;
T current = null;
while (iter.hasNext() || delay > 0) {
if (delay > 0)
delay--;
else
current = iter.next();
try {
int amount = block.invoke(current, delay);
if (delay == 0 && amount > 0)
if (iter.hasNext())
current = iter.next();
else
current = null;
delay += amount;
} catch (ContrivedException ex) {
delay = 0;
} catch (java.lang.closure.Break ex) {
break;
} catch (java.lang.closure.Continue ex) {
continue;
}
}
}
The important thing here is that the method on the SAM type
ContrivedBlock does not return
void, and throws a specific
exception. These are both signals to the CAI, because it has been
declared to accept them, and you can see from the body of the CAI
that it indeed uses the return value, and catches the declared
exception.
How should a call site be translated now? Here's one to work with:
void caller(List<String> list) {
contrivance(list) : (String value, int rem) {
self:
if (rem == 0) {
try {
// A syntax for a local return
return:self Integer.parseInt(value);
} catch (NumberFormatException ex) {
// Fall through.
}
}
if (value.equals("skip"))
throw new ContrivedException();
if (value.equals("end"))
// Return from 'caller'.
return;
process(value);
return:self 0;
}
}
For simplicity, the loop body doesn't use any external local variables, nor does it long-return any values, nor throw any exceptions other than those declared in the SAM type. We'll forego the boolean in-scope check too, allowing us to dispense with the generated local class, and just use a static method:
static int $body$1(String value, int rem) {
if (rem == 0) {
try {
return Integer.parseInt(value);
} catch (NumberFormatException ex) {
// Fall through.
}
}
if (value.equals("skip"))
throw new ContrivedException();
if (value.equals("end")) {
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
process(value);
return 0;
}
void caller(List<String> list) {
contrivance(list, #$body$1(String, int));
}
Now, when we translate the control statements, the labelled returns
have become plain returns (because they are signals to the CAI),
and the unlabelled return has become the throwing of an
Exit. The throwing of an
exception has not been translated, because its type is declared on
the SAM's signature, so it too is a signal to the CAI.
We just need to translate the remaining call site:
static int $body$1(String value, int rem) {
if (rem == 0) {
try {
return Integer.parseInt(value);
} catch (NumberFormatException ex) {
// Fall through.
}
}
if (value.equals("skip"))
throw new ContrivedException();
if (value.equals("end")) {
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
process(value);
return 0;
}
void caller(List<String> list) {
try {
contrivance(list, #$body$1(String, int));
} catch (java.lang.closure.Return ex) {
return;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
}
So, in summary, we can return values and throw exceptions to the
CAI with very little translation. If any other checked exceptions
were thrown from the loop body, we would still have to translate
them into Exit.throwing(), so
that the CAI could not see them.
If we were instead to use <throws E> to acheive this
exception transparency, we would not be able to allow a closure to
signal exceptions to its CAI. It would then be incongruent to
forbid exceptions, yet allow return values, as signals. So I'm not
against <throws E>,
but I don't see a need for it here (and it gets in the way of
another feature), and I'm sceptical of its utility in cases where
lambdas are accepted but closures aren't.
To deal with nested closures, we'd have to be able to return a
value from the body of the inner one to the CAI of the outer one.
For that, we need to add two methods to Exit, and one field to Label:
class Exit {
public boolean isReturningTo(Label label) { return false; }
public static void returningTo(Label label) {
throw label.RETURN;
}
...
}
class Label {
final Exit RETURN = new Exit() {
public boolean isReturningTo(Label label) {
return label == Label.this;
}
};
...
}
In other words, we use the same technique as for labelled
break and continue.
In fact, it's probably simpler if we drop the Return, class, and use a synthetic label,
i.e. create a Label, every time.
In contrast to Break and
Continue, which must be handled
by user code, there's no great advantage in distinguishing a
general return signal from one
to a specific label, which are both handled by generated code.
Syntax for closures
Firstly, for a method to accept a closure, rather than a limited lambda, it must make the promise about not invoking the closure on a different stack that its caller's, and not after returning to the caller. This indicates that the choice of whether to use a closure or a lambda belongs to the author of the method accepting it (who can make the promise), not the author of the call site (who cannot). That is to say, if the method does not declare that it accepts a closure, the compiler should forbid closure syntax, and permit only lambda.
Secondly, while there seem to be very few use cases of a control-abstraction method accepting more than one closure argument, it makes sense not to forbid the possibility. A syntax such as this:
forEach(String elem : list) {
body;
}
…doesn't lend itself easily to multiple closures, which is why I've used the following syntax in the examples above:
forEach(list) : (String elem) {
body;
}
If the control abstraction accepts multiple blocks, this syntax has room for them:
foo(inputArgs) : (String elem) {
block1;
} : () {
block2;
}
It could even take named blocks, where appropriate:
foo(inputArgs) : onSuccess(String elem) {
block1;
} : onFailure.invoke() {
block2;
}
Of course, the original form could still exist, to be used as a shorthand in the most common cases.
The promise to treat a SAM object as a closure should be expressed programmatically, so the compiler can tell whether to accept closure syntax. A few annotations should suffice:
package java.lang.closure;
@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.CLASS)
public @interface Block { }
@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.CLASS)
public @interface Iteration { }
@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.CLASS)
public @interface Once {
String group() default "";
}
@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.CLASS)
public @interface Guaranteed {
String group() default "";
}
forEach would use @Iteration because it accepts the
Break and Continue signals from the block.
Consequently, the compiler needs to know whether to translate
break and continue into those signals, into some other
signals, or to reject them as errors.
import java.lang.closure.Iteration;
static <T> void forEach(Iterable<T> coll,
@Iteration ForEachBlock<T> block) {
...
}
A method that doesn't accept such signals should just use
@Block:
static void with(Lock lock, @Block Runnable block) {
lock.claim();
try {
block.run();
} finally {
lock.release();
}
}
It should also be possible to inform the compiler at the call site that a block is guaranteed to be invoked exactly once:
import java.lang.closure.Once;
static void with(Lock lock, @Once @Block Runnable block) {
lock.claim();
try {
block.run();
} finally {
lock.release();
}
}
Whether this allows the compiler of the with method to determine that the guarantee
is true is another matter, but the real aim is to allow the call
site to be checked for things like this:
final String blank;
with (myLock) : {
blank = "yes";
}
So @Once allows the compiler to
statically determine that a blank final is definitely unassigned.
To determine whether variables are definitely assigned, we need
something similar like @Guaranteed, indicating that a block will be
executed at least once. Both of these annotations should allow
parameters to be grouped, so that the caller understands the
branching possibilities:
interface SuccessBlock<T> {
void invoke(T data);
}
interface FailureBlock {
void invoke();
}
void processArg(@Once("a") @Block
SuccessBlock<String> onSuccess,
@Once("a") @Block
FailureBlock onFailure) {
if (dataAvailable)
onSuccess.invoke(data);
else
onFailure.invoke();
}
processArg() : onSuccess(String s) {
print(s);
} : onFailure() {
errorMessage("not available");
}
The checkedness of the Exit
hierarchy
I've so far declared Exit to
extend Throwable, so that makes
it a checked exception. However, it needs to be an unchecked one to
go unnoticed by CAIs. So it must really derive from Error or RuntimeException. Yet neither of these seem
very appropriate either. It's not a programming error, as the
latter suggests, nor is it an unavoidable system failure, as the
former suggests.
If there is no other option, I'd go for Error, but there is a case where even
Throwable would be
inappropriate:
try {
// Do something dodgy...
} catch (Throwable t) {
logger.log(t);
throw t;
}
Apart from Break and
Continue, user code should never
have to interact with Exits,
even to log them.
In our favour, no user code even directly generates an Exit. Suppose that the JVM does not require
that thrown objects are Throwable. That means we only need to deal
with cases where Exit is thrown
even though it does not extend Throwable. Exit would be a hierarchy distinct from
Throwable, and no existing code
would encounter it. For new code, only catch (Continue) and catch (Break) would need to be permitted. I
assume that closure translations would occur after such
compile-time checks, so generated code would not be affected.
If a class's exception classification could be controlled
programmatically, we would probably use a few annotations.
@PackageThrowable would indicate
that a type could only be thrown by methods within the same
package, @Unchecked would mark
the root of a hierarchy of unchecked exceptions, and @Catchable would make our break and continue signals catchable by any user code:
@PackageThrowable
@Unchecked
class Exit { ... }
@Catchable
class Continue extends Exit { .. }
Labelled throws
Since we can support labelled returns, i.e. return targeted at a CAI which accepts it as
a signal, and we support CAIs that accept exceptions too, we should
also support throw statements
targeted at specific CAIs, in case one closure is nested in
another, and the CAIs of both accept the same exception type as a
signal. We should be able to extend Label and Exit just as we did for supporting long
returns:
class Label {
final Exit THROW = new Exit() {
public boolean isThrowingTo(Label label) {
return label == Label.this;
}
};
...
}
class Exit {
public boolean isThrowingTo(Label label) { return false; }
public static void throwingTo(Label label) {
throw label.THROW;
}
...
}
As with the unlabelled Return
class, we may as well drop the Throw class, and just generate a synthetic
Label every time.
Efficient inlining
I wonder how well all this translation gets inlined at runtime.
Here's a recap of the code we have so far. First, the
implementation of forEach:
static <T> void forEach(Iterable<T> coll,
ForEachBlock<T> block) {
for (Iterator<T> iter = coll.iterator(); iter.hasNext(); ) {
T elem = iter.next();
try {
block.invoke(elem);
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
}
In anticipation of potential problems, we'll use a version of the
generated code already well optimized by the compiler, and we'll
use a different technique for labelled jumps. Instead of multiple
instances of Label, we'll create
a new signal class for each jump that gets compiled. Then we can
use catch to detect the jump
directly, instead of catching Exit and making an additional call to
Label. The call site is now
translated as:
static class $label$1$Continue extends Exit {
static final $label$1$Continue INSTANCE =
new $label$1$Continue();
}
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
java.lang.closure.Exit.returning();
throw new UnreachableError();
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
java.lang.closure.Exit.throwing();
throw new UnreachableError();
}
if (datum.equals("comment")) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
if (datum.equals("skip")) {
throw $label$1$Continue.INSTANCE;
}
index++;
}
void $body$2(List<String> data) {
try {
forEach(data, #$body$1(String));
} catch ($label$1$Continue ex) {
java.lang.closure.Exit.continuing();
throw new UnreachableError();
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
forEach(dataList, $closure$1$inst#$body$2(List<String>));
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return $closure$1$inst.index;
}
Of course, it's actually bytecode that gets inlined, but let's inline the source code to see if it shows any immediate problems.
We'll start by inlining the static calls to Exit. That will allow us to eliminate some
unreachable statements too:
static class $label$1$Continue extends Exit {
static final $label$1$Continue INSTANCE =
new $label$1$Continue();
}
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
throw java.lang.closure.Exit.RETURN;
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
throw java.lang.closure.Exit.THROW;
}
if (datum.equals("comment")) {
throw java.lang.closure.Exit.CONTINUE;
}
if (datum.equals("skip")) {
throw $label$1$Continue.INSTANCE;
}
index++;
}
void $body$2(List<String> data) {
try {
forEach(data, #$body$1(String));
} catch ($label$1$Continue ex) {
throw java.lang.closure.Exit.CONTINUE;
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
forEach(dataList, $closure$1$inst#$body$2(List<String>));
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return $closure$1$inst.index;
}
Now we'll inline the two calls to forEach:
static class $label$1$Continue extends Exit {
static final $label$1$Continue INSTANCE =
new $label$1$Continue();
}
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$1(String datum) {
if (datum.equals("yes")) {
$return$1 = index;
throw java.lang.closure.Exit.RETURN;
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
throw java.lang.closure.Exit.THROW;
}
if (datum.equals("comment")) {
throw java.lang.closure.Exit.CONTINUE;
}
if (datum.equals("skip")) {
throw $label$1$Continue.INSTANCE;
}
index++;
}
void $body$2(List<String> data) {
try {
for (Iterator<String> iter2 = data.iterator();
iter2.hasNext(); ) {
String elem2 = iter2.next();
try {
$body$1(elem2);
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch ($label$1$Continue ex) {
throw java.lang.closure.Exit.CONTINUE;
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
for (Iterator<List<String>> iter = dataList.iterator();
iter.hasNext(); ) {
List<String> elem = iter.next();
try {
$closure$1$inst.$body$2(elem);
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return $closure$1$inst.index;
}
There shouldn't be a problem inlining $body$1 into $body$2:
static class $label$1$Continue extends Exit {
static final $label$1$Continue INSTANCE =
new $label$1$Continue();
}
int someContext(List<List<String>> dataList)
throws DodgyException {
class $closure$1 {
int index = 0;
int $return$1;
DodgyException $throw$1;
void $body$2(List<String> data) {
try {
for (Iterator<String> iter2 = data.iterator();
iter2.hasNext(); ) {
String datum = iter2.next();
try {
if (datum.equals("yes")) {
$return$1 = index;
throw java.lang.closure.Exit.RETURN;
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
throw java.lang.closure.Exit.THROW;
}
if (datum.equals("comment")) {
throw java.lang.closure.Exit.CONTINUE;
}
if (datum.equals("skip")) {
throw $label$1$Continue.INSTANCE;
}
index++;
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch ($label$1$Continue ex) {
throw java.lang.closure.Exit.CONTINUE;
}
}
}
final $closure$1 $closure$1$inst = new $closure$1();
try {
for (Iterator<List<String>> iter = dataList.iterator();
iter.hasNext(); ) {
List<String> elem = iter.next();
try {
$closure$1$inst.$body$2(elem);
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch (java.lang.closure.Return ex) {
return $closure$1$inst.$return$1;
} catch (java.lang.closure.Throw ex) {
throw $closure$1$inst.$throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return $closure$1$inst.index;
}
The closure method doesn't expose itself, so perhaps its context can be unwound as it is inlined:
static class $label$1$Continue extends Exit {
static final $label$1$Continue INSTANCE =
new $label$1$Continue();
}
int someContext(List<List<String>> dataList)
throws DodgyException {
int index = 0;
int $return$1;
DodgyException $throw$1;
try {
for (Iterator<List<String>> iter = dataList.iterator();
iter.hasNext(); ) {
List<String> data = iter.next();
try {
try {
for (Iterator<String> iter2 = data.iterator();
iter2.hasNext(); ) {
String datum = iter2.next();
try {
if (datum.equals("yes")) {
$return$1 = index;
throw java.lang.closure.Exit.RETURN;
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
throw java.lang.closure.Exit.THROW;
}
if (datum.equals("comment")) {
throw java.lang.closure.Exit.CONTINUE;
}
if (datum.equals("skip")) {
throw $label$1$Continue.INSTANCE;
}
index++;
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch ($label$1$Continue ex) {
throw java.lang.closure.Exit.CONTINUE;
}
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch (java.lang.closure.Return ex) {
return $return$1;
} catch (java.lang.closure.Throw ex) {
throw $throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return index;
}
Most of the exception handlers do not use the exception object itself, only its class. That should mean we can inline their bodies into places where the exception type is thrown:
static class $label$1$Continue extends Exit {
static final $label$1$Continue INSTANCE =
new $label$1$Continue();
}
int someContext(List<List<String>> dataList)
throws DodgyException {
int index = 0;
int $return$1;
DodgyException $throw$1;
try {
outer:
for (Iterator<List<String>> iter = dataList.iterator();
iter.hasNext(); ) {
List<String> data = iter.next();
try {
try {
inner:
for (Iterator<String> iter2 = data.iterator();
iter2.hasNext(); ) {
String datum = iter2.next();
try {
if (datum.equals("yes")) {
$return$1 = index;
return $return$1;
}
if (datum.equals("no")) {
$throw$1 = new DodgyException();
throw $throw$1;
}
if (datum.equals("comment")) {
continue inner;
}
if (datum.equals("skip")) {
continue outer;
}
index++;
} catch (java.lang.closure.Continue ex) {
continue inner;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch ($label$1$Continue ex) {
continue outer;
}
} catch (java.lang.closure.Continue ex) {
continue outer;
} catch (java.lang.closure.Break ex) {
break;
}
}
} catch (java.lang.closure.Return ex) {
return $return$1;
} catch (java.lang.closure.Throw ex) {
throw $throw$1;
} catch (java.lang.closure.Exit ex) {
throw new java.lang.closure.ClosureError(ex);
}
return index;
}
We haven't eliminated all the exception handlers, but they should
never be invoked now anyway. Apart from that, it seems we have
inlined the control abstraction into a form similar to the original
code. Because we know that these exceptions can't happen, we can be
sure that the final code is equivalent to one with no try blocks, and we can expect some
assignments to be optimized away too, resulting in:
int someContext(List<List<String>> dataList)
throws DodgyException {
int index = 0;
outer:
for (Iterator<List<String>> iter = dataList.iterator();
iter.hasNext(); ) {
List<String> data = iter.next();
for (Iterator<String> iter2 = data.iterator();
iter2.hasNext(); ) {
String datum = iter2.next();
if (datum.equals("yes"))
return index;
if (datum.equals("no"))
throw new DodgyException();
if (datum.equals("comment"))
continue;
if (datum.equals("skip"))
continue outer;
index++;
}
}
return index;
}
…as if the source had been inlined by the compiler.
Unfortunately, I don't know enough about inlining to be sure that all of the above steps are possible, especially when the bytecode contains less information than the source.
Use cases
interface EntryAction<K, V> {
void apply(K key, V value);
}
static <K, V>
void forEach(Map<K, V> map,
@Iteration EntryAction<? super K, ? super V> action) {
for (Map.Entry<K, V> entry : map.entrySet()) {
action.apply(entry.getKey(), entry.getValue());
} catch (java.lang.closure.Continue ex) {
continue;
} catch (java.lang.closure.Break ex) {
break;
}
}
Map<P, Q> map;
forEach(map) : (P key, Q value) {
...
}
import java.util.concurrent.locks.Lock;
import java.lang.closure.Block;
static void with(Lock lock, @Block @Once Runnable action) {
lock.lock();
try {
action.run();
} finally {
lock.unlock();
}
}
package java.security;
import java.lang.closure.Block;
class AccessController {
static void doPrivileged(@Block @Once Runnable action) {
...
}
}
interface MessageBuilder {
void buildMessage(PrintWriter out);
};
static void with(Logger logger, LogLevel level,
@Block @Once MessageBuilder block) {
if (logger.isLoggable(level)) {
StringWriter sw = new StringWriter();
PrintWriter out = new PrintWriter(sw);
try {
block.buildMessage(out);
} finally {
out.flush();
logger.log(level, sw.toString());
}
} else {
block.buildMessage(new NullWriter());
}
}
Asynchronous, single-threaded APIs
A topic started on the project mailing list near the end of July 2011: Effectively final. It calls for the ability of lambdas to mutate local variables (mutable local capture), something which is forbidden now to discourage poor practices in using concurrent APIs.
The topic originator pointed out that there are asynchronous, single-threaded APIs, which don't need to be protected from mutable local capture, and require it in many cases. Should it be supported, and what can be done instead?
Co-ordinating multiple responses
Let's suppose we have an API with some methods that promise to call you back with results through some SAM types after performing an action:
class Services {
static Completion foo(Some args, FooResponseHandler action);
static Completion bar(Some args, BarResponseHandler action);
.
.
.
}
They all return Completion,
which you can use to set up another action to be called after the
results are delivered:
interface Completion {
void act(Runnable action);
}
These can be passed to a collection of utilities for handling several responses together:
class Composer {
static Completion when(Completion... parts);
.
.
.
}
The example method shown creates a new Completion which fires when all of the
supplied Completions have
happened.
Now we would like to write this:
void context() {
int result1, result2;
Composer
.when(Services.foo(args, (val) -> result1 = val),
Services.bar(args, (val) -> result2 = val))
.act(() -> sendResponse(result1 + result2));
}
In other words, get and store two values asynchronously. When both have arrived, sum them, and send the result somewhere else. Only one thread is used, so capturing the locals and assigning to them from the lambdas should be safe, but those assignments are forbidden, in general.
Instead, we have to create an object of no particular type as a container for the locals, and refer to the locals as members of it:
void context() {
class Anon {
int result1, result2;
}
Anon anon = new Anon();
Composer
.when(Services.foo(args, (val) -> anon.result1 = val),
Services.bar(args, (val) -> anon.result2 = val))
.act(() -> sendResponse(anon.result1 + anon.result2));
}
To avoid having to refer to anon
everywhere, we could change the lambdas into members of
Anon:
void context() {
class Anon {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
}
Anon anon = new Anon();
Composer
.when(Services.foo(args, anon#getResult1),
Services.bar(args, anon#getResult2)
.act(anon#done);
}
Maybe this is the right way to go about it, as it places all the code that actually shares data in one place, so you can quickly check it's safe. That makes the verbosity less of a burden, but it still contains some boiler-plate code to create an essentially anonymous object.
Anonymizing the shared data
We could try to use an anonymous type to hold the shared data:
void context() {
Object anon = new Object() {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
};
Composer
.when(Services.foo(args, anon#getResult1),
Services.bar(args, anon#getResult2)
.act(anon#done);
}
…but this doesn't work, because the members of the object which are
not inherited from Object are
not visible outside the declaration. Is it worth making an
exception here, that would break the encapsulation useful in other
scenarios? Perhaps we need another syntax to combine the object and
type declarations, without requiring the type to have a name, or be
derived from any particular type:
void context() {
class anon = {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
};
Composer
.when(Services.foo(args, anon#getResult1),
Services.bar(args, anon#getResult2)
.act(anon#done);
}
Even without a new syntax, you can push the registration phase into an anonymous object using an instance initializer:
void context() {
new Object() {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
{
Composer
.when(Services.foo(args, this#getResult1),
Services.bar(args, this#getResult2)
.act(this#done);
}
};
}
That leaks this from an
incompletely constructed object. If that's a problem, make the
object a Runnable instead:
void context() {
new Runnable() {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
public void run() {
Composer
.when(Services.foo(args, this#getResult1),
Services.bar(args, this#getResult2)
.act(this#done);
}
}.run();
}
If you need to return a value, use Callable:
int context() {
return new Callable<Integer>() {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
public Integer call() {
Composer
.when(Services.foo(args, this#getResult1),
Services.bar(args, this#getResult2)
.act(this#done);
return 0;
}
}.call();
}
Actually, the object's hidden type is still visible for a moment after its declaration:
int context() {
return new Object() {
int result1, result2;
void getResult1(int val) { result1 = val; }
void getResult2(int val) { result2 = val; }
void done() { sendResponse(result1 + result2); }
int call() {
Composer
.when(Services.foo(args, this#getResult1),
Services.bar(args, this#getResult2)
.act(this#done);
return 0;
}
}.call();
}
This looks like a fairly elegant work-around that will probably be suitable for almost all cases.
Marking shared data
We could resurrect the old @Shared annotation to mark shared data, and
the compiler would put the data into a shared object for us:
void context() {
@Shared
int result1, result2;
Composer
.when(Services.foo(args, (val) -> result1 = val),
Services.bar(args, (val) -> result2 = val))
.act(() -> sendResponse(result1 + result2));
}
But maybe that would tempt users to discard protection when using concurrent APIs. It places responsibility for thread safety on the caller, when it can only really be enforced by the callee.
Marking asynchronous SAM objects
It might be better to mark the SAM parameters to be fulfilled by data-sharing objects:
class Services {
static Completion foo(Some args,
@Callback FooResponseHandler action);
static Completion bar(Some args,
@Callback BarResponseHandler action);
.
.
.
}
Again, the compiler would put shared variables in a hidden object, but this would only happen where the promise was made to execute the lambdas in an asynchronous, single-threaded fashion, i.e. where the annotation is present.
The callbacks would still be lambdas, and (unlike closures) would not have to be defined at the point they are submitted:
int result1;
@Callback
FooResponseHandler action = (val) -> { result1 = val; };
Services.foo(args, action);
We would have to regard @Callback as a taint on the reference, so
that shared variables could not leak into concurrent APIs:
void func1(Runnable cb);
void func2(@Callback Runnable cb);
Runnable cb1 = () -> { };
@Callback Runnable cb2 = () -> { };
func1(cb1);
func2(cb2);
func1(cb2); // error
func2(cb1);
Engineering mutable local capture
The most plausible and feasible suggestions revolve around some form of boxing of the variables, so that they can, in general, live longer than normal variables. The simplest comprehensive suggestion wraps each accessed local in a length-1 array, so these variables:
int a; short b;
…become these:
final int[] a = new int[1]; final short[] b = new short[1];
As finals, a and b
are accessible to any lambdas declared in their scope. So long as
there are guarantees about thread-safety, this technique works.
However, it potentially generates a lot of objects.
When thread-safety isn't guaranteed, one suggestion is to box
variables in AtomicInteger and
friends:
final AtomicInteger a = new AtomicInteger(); final AtomicInteger b = new AtomicInteger(); // no AtomicShort
Of course, this still generates a lot of objects, and deals with thread-safety only in the simplest cases. It does not work in general, for example, when computations on multiple variables interact. As such, I don't think its prospects are good. Indeed, I don't think it's worth trying to patch thread-safety into lambda translation, and rather it should be handled by formalizing the guarantees that some APIs are able to make.
So let's assume thread-safety, and try to reduce the number of objects. All locals that need to be accessed by a lambda could be placed into a single object:
class $Anon$1 {
int a;
short b;
}
final $Anon$1 $anon$1 = new $Anon$1();
The lambda could even be translated into a method reference to a method on the class:
class $Anon$1 {
int a;
short b;
void doStuff() {
// Manipulate a and b.
}
}
final $Anon$1 $anon$1 = new $Anon$1();
setTimeout($anon$1#doStuff);
The object could be accessed by multiple lambdas, even if the sets of variables they access do not intersect. This would be very suitable in most cases, but what if one of the variables was an object reference, and it was only used by one lambda, whose instantiation has been discarded? One would expect the object to be garbage-collectable, but it won't be until all the lambdas using the shared box have been discarded too. So, when object references are captured, care has to be taken to partition variables so they are not retained longer than necessary.
Let's contrive our asynchronous example so that an object reference is assigned:
void context() {
Map<String, Integer> map;
int result1, result2;
Composer
.when(Services.foo(args, (val) -> {
result1 = val;
map = new HashMap<>();
}),
Services.bar(args, (val) -> result2 = val))
.act(() -> sendResponse(result1 + result2));
}
Now rewrite it so that a single object is used to box all the locals:
void context() {
class $Anon$1 {
Map<String, Integer> map;
int result1, result2;
}
final $Anon$1 $anon$1 = new $Anon$1();
Composer
.when(Services.foo(args, (val) -> {
$anon$1.result1 = val;
$anon$1.map = new HashMap<>();
}),
Services.bar(args, (val) -> $anon$1.result2 = val))
.act(() -> sendResponse($anon$1.result1 +
$anon$1.result2));
}
Suppose the foo lambda is
called, but the bar lambda never
is. (Of course, if that's the case, you have bigger problems. I did
say it was contrived.) Nothing else uses map, yet it must hang around because other
lambdas indirectly reference it.
If that translation had been done manually, it would not be a big problem, because the programmer can see it, and do something about it. However, if it's automatic, it produces an unexpected behaviour hidden from the programmer.
Semantic-changing annotations
I just noticed this: Nice to @Share (Neal Gafter, 2010-02-22 13:31 PST). It expresses the position that annotations should not change semantics. Does anything I've suggested do that? Perhaps, but I think they don't have any effect except to determine whether something compiles or not. Should you care about the semantic difference between a program that compiles and one that doesn't?
Summary
-
I prefer certain syntaxes for method references and lambda expressions, driven by feature orthogonality and the (imagined) needs of informal parsers.
-
Lambdas should have few transparencies, and be used with APIs supporting concurrency, because other transparencies don't make sense with concurrency.
-
Supporting all the transparencies required by full closures means that full closures only make sense with serial APIs.
-
Such transparencies should only be supported when a control-abstraction syntax is used. Although the syntax doesn't strictly enforce seriality in the implementation of the control abstraction, it can be enforced at run-time by the call site. Furthermore, if a method cannot promise to execute a SAM object serially, it can be declared such that the control-abstraction syntax is not accepted, to signal the absence of that promise.
-
Anything specified in the SAM type used by a control-abstraction implementation is intended as a signal to or from that implementation, and should not be handled transparently.
-
Short returns and throws must be supported in control abstractions, so they can be used as such signals to the control-abstraction implementation.
-
Long returns, throws, continues and breaks can be implemented by throwing static instances of a kind of exception, which possibly has to be distinct from
Throwable. -
<throws E>, as neat as it is, doesn't seem to be very useful on the SAM type for a concurrent lambda, is not necessary on the SAM type for a full closure, and even gets in the way of full closures where declared exceptions are signals to the control-abstraction implementation. -
There's room for a third kind of SAM objects, callbacks, which can capture local variables mutably. The API writer will determine whether they are permitted.
Recapitulation of classes
package java.lang;
/**
* Signals that code has been reached that should not be reachable.
*/
public class UnreachableError extends AssertionError {
public UnreachableError() { super("unreachable"); }
}
package java.lang;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Marks a class as throwable by code within its own package.
*/
@Target(ElementType.TYPE)
public @interface PackageThrowable { }
package java.lang;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Marks the root of a hierarchy of unchecked throwable types.
*/
@Target(ElementType.TYPE)
public @interface Unchecked { }
package java.lang;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Marks the root of a catchable sub-hierarchy of throwable types.
*/
@Target(ElementType.TYPE)
public @interface Catchable { }
package java.lang.closure;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Identifies a SAM-type parameter that can be fulfilled with a
* non-iterative closure block.
*/
@Target(ElementType.PARAMETER)
public @interface Block { }
package java.lang.closure;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Identifies a SAM-type parameter that can be fulfilled with an
* iterative closure block.
*/
@Target(ElementType.PARAMETER)
public @interface Iteration { }
package java.lang.closure;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Identifies a SAM-type closure parameter, whose body is guaranteed
* to be executed exactly once. A parameter can belong to a group.
* Parameters of the same method with the same group share the
* guarantee, i.e. exactly one of them will be executed.
*/
@Target(ElementType.PARAMETER)
public @interface Once {
/**
* Identifies the group that this closure parameter belongs to.
*/
String group() default "";
}
package java.lang.closure;
import java.lang.annotation.Target;
import java.lang.annotation.ElementType;
/**
* Identifies a SAM-type closure parameter, whose body is guaranteed
* to be executed at least once. A parameter can belong to a group.
* Parameters of the same method with the same group share the
* guarantee, i.e. at least one of them will be executed.
*/
@Target(ElementType.PARAMETER)
public @interface Guaranteed {
/**
* Identifies the group that this closure parameter belongs to.
*/
String group() default "";
}
package java.lang.closure;
/**
* Identifies a jump point within a closure or its enclosing context.
* This class is used internally by the compiler to support closures,
* and does not need to be accessed directly by user code.
*/
public class Label {
final Exit CONTINUE = new Exit() {
public boolean isContinuingTo(Label label) {
return label == Label.this;
}
};
final Exit BREAK = new Exit() {
public boolean isBreakingTo(Label label) {
return label == Label.this;
}
};
final Exit RETURN = new Exit() {
public boolean isReturningTo(Label label) {
return label == Label.this;
}
};
final Exit THROW = new Exit() {
public boolean isThrowingTo(Label label) {
return label == Label.this;
}
};
}
package java.lang.closure;
/**
* Signals an exit from a closure to its enclosing context. User code
* need not access this class directly, but may use its subtypes
* {@link Break} and {@link Continue}.
*/
@PackageThrowable
@Unchecked
public abstract class Exit {
// Only this package can create Exits.
Exit() { }
@Deprecated
private static final Return RETURN = new Return();
/**
* Obtain the signal object to indicate a long return.
*
* @throws Exit the singleton signal object
*
* @deprecated Just use a labelled return with a synthetic label.
*/
public static void returning() {
throw RETURN;
}
@Deprecated
private static final Throw THROW = new Throw();
/**
* Obtain the signal object to indicate a long throw.
*
* @throws Exit the singleton signal object
*
* @deprecated Just use a labelled throw with a synthetic label.
*/
public static void throwing() {
throw THROW;
}
private static final Break BREAK = new Break();
/**
* Obtain the signal object to indicate a short {@code break} within
* an iterative closure block.
*
* @throws Exit the singleton signal object
*/
public static void breaking() {
throw BREAK;
}
private static final Continue CONTINUE = new Continue();
/**
* Obtain the signal object to indicate a short {@code continue}
* within an iterative closure block.
*
* @throws Exit the singleton signal object
*/
public static void continuing() {
throw CONTINUE;
}
/**
* Obtain the signal to indicate a {@code continue} to the specified
* label.
*
* @param label the label identifying the target of the {@code
* continue}
*
* @throws Exit a signal representing a jump to the requested target
*/
public static void continuingTo(Label label) {
throw label.CONTINUE;
}
/**
* Obtain the signal to indicate a {@code break} to the specified
* label.
*
* @param label the label identifying the target of the {@code
* break}
*
* @throws Exit a signal representing a jump to the requested target
*/
public static void breakingTo(Label label) {
throw label.BREAK;
}
/**
* Obtain the signal to indicate a {@code return} to the specified
* label.
*
* @param label the label identifying the target of the {@code
* return}
*
* @throws Exit a signal representing a jump to the requested target
*/
public static void returningTo(Label label) {
throw label.RETURN;
}
/**
* Obtain the signal to indicate a {@code throw} to the specified
* label.
*
* @param label the label identifying the target of the {@code
* throw}
*
* @throws Exit a signal representing a jump to the requested target
*/
public static void throwingTo(Label label) {
throw label.THROW;
}
/**
* Check whether this signal indicates a {@code continue} to the
* specified label.
*
* @param label the label identifying the target of the {@code
* continue}
*
* @return true if this is the requested signal
*/
public boolean isContinuingTo(Label label) { return false; }
/**
* Check whether this signal indicates a {@code break} to the
* specified label.
*
* @param label the label identifying the target of the {@code
* break}
*
* @return true if this is the requested signal
*/
public boolean isBreakingTo(Label label) { return false; }
/**
* Check whether this signal indicates a {@code return} to the
* specified label.
*
* @param label the label identifying the target of the {@code
* return}
*
* @return true if this is the requested signal
*/
public boolean isReturningTo(Label label) { return false; }
/**
* Check whether this signal indicates a {@code throw} to the
* specified label.
*
* @param label the label identifying the target of the {@code
* throw}
*
* @return true if this is the requested signal
*/
public boolean isThrowingTo(Label label) { return false; }
}
package java.lang.closure;
/**
* Signals a long {@code return} from a closure. User code need not
* access this class directly.
*
* @deprecated Just use a labelled return with a synthetic label.
*/
public class Return extends Exit { }
package java.lang.closure;
/**
* Signals a long {@code throw} from a closure. User code need not
* access this class directly.
*
* @deprecated Just use a labelled throw with a synthetic label.
*/
public class Throw extends Exit { }
package java.lang.closure;
/**
* Signals a short {@code continue} from a closure. Code that invokes
* a SAM-type parameter declared with {@link Iteration} must catch
* this signal escaping from the call.
*/
@Catchable
public class Continue extends Exit { }
package java.lang.closure;
/**
* Signals a short {@code break} from a closure. Code that invokes a
* SAM-type parameter declared with {@link Iteration} must catch this
* signal escaping from the call.
*/
@Catchable
public class Break extends Exit { }
Updated: 2012-Apr-29 07:01 GMT