Lambdas, InvokeDynamic, and the Dynamic Language Runtime (DLR)

While going through [information on Java 8](http://www.infoq.com/news/2014/03/java8-launch-qa?

utm_source=infoq&utm_medium=popular_links_homepage), I was surprised to discover that Lambdas rely on invokedynamic. This bytecode instruction, introduced in Java 7, was initially intended for dynamic language implementors on the JVM, not for Java itself. Notably, Java 8 doesn’t incorporate dynamic features like C#’s dynamic keyword.

The article mentioned experimenting with various Lambda implementations on the VM, ultimately finding that invokedynamic proved beneficial, even leading to improvements within invokedynamic itself. This focus on seamlessly using Lambdas with collections resulted in the Streams API and, consequently, extension methods, necessitating language-level support.

My surprise stemmed from the fact that languages like C#/VB.Net on the CLR and Groovy/Scalar on the JVM have had lambdas long before dynamic capabilities were introduced to their platforms (DLR for CLR and invokedynamic for JVM). These languages implement Lambdas at the compiler level, generating new methods or classes (for closures requiring state). This discovery prompted me to research the rationale and benefits of Java’s approach.

A very interesting post provided some insights. The comments section featured an explanation from a Scala team member:

From that document you can see that Java 8 does create an anonymous class that implements SAM interface for for each lambda separately. The difference between Scala and Java 8 is that Java 8 will create those anonymous classes at runtime using LambdaMetaFactory. The class is named “meta factory” for two reasons: 1. LambdaMetaFactory is being called during program linkage (so at meta level). 2. LambdaMetaFactory is “a factory of factories” (hence meta factory) which means, it creates classes that have constructors and those constructors are factories for each lambda they represent. Therefore, the invokedynamic instruction is there to get the code that will create an instance of a lambda. As mentioned above, that invokedynamic instruction will get expanded to a call to a constructor of anonymous class that LambdaMetaFactory creates at runtime. This means that at runtime the bytecode you get for Scala and Java lambdas looks very similar. It also means that Java 8 doesn’t have any more efficient way of implementing lambdas if you care about runtime performance. Java 8’s strategy does result in smaller JAR files because all anonymous classes are created on the fly. However, the key thing about invokedynamic is that it’s essentially a JVM-level macro that defers lambda translation strategy to LambdaMetaFactory which is a library class. If Java 9 or 10 gets more direct (and performant) support for lambdas that won’t require classes and object allocations then it can swap implementation of LambdaMetaFactory and all _existing_ code written for Java 8 will get performance boost. That is the brilliance of using invokedynamic in context of translating lambdas. We’ll have to wait for future versions of Java to experience those benefits, though.

In essence, Java 8 also generates anonymous classes, but at runtime instead of compile time, primarily to leverage potential future JVM enhancements. This runtime class creation, similar to the DLR’s approach, piqued my curiosity.

This exploration sparked my interest in the workings of invokedynamic and its comparison to other methods. Unlike the JVM, the CLR lacks a similar bytecode-level instruction. Dynamic features in the CLR (IronPython, IronRuby, C#’s dynamic) typically rely on the DLR, introduced in .Net 4.0 without modifying the underlying VM (no new bytecode instructions). A good overview of the DLR can be found [here](http://stackoverflow.com/questions/4515135/the-dlr-boo-and-the-

jvm). The following paragraph comparing it to the then-under-development JVM counterpart is particularly interesting:

The Davinci project will add “method handles” to the JVM which the CLR has already had in the form of delegates. It will also add a new “invokedynamic” opcode which the CLR doesn’t have and didn’t gain w/ the DLR work. Instead the DLR just uses the existing primitives (delegates, reified generics) plus libraries for defining the interop protocol. Both add the concept of call sites and those may be fairly similar between the two.

One of the DLR’s components, callsites (evident in the bytecode of C# code using dynamic) and callsite caching, should resonate with anyone familiar with pre-invokedynamic Groovy’s generated code. Groovy, a highly dynamic language, has functioned well with this callsite caching approach. However, its port to use invokedynamic (the indy version) promises significant performance enhancements. Note that invokedynamic appears to be based on MethodHandles and CallSites (MethodHandles were added to Java to support invokedynamic).

A this write up, authored by a JRuby developer, argues that the JVM surpasses the CLR as an environment for dynamic languages. Lacking sufficient expertise in either platform, I’ve compiled a list of resources for future exploration:

Licensed under CC BY-NC-SA 4.0
Last updated on Aug 20, 2023 08:42 +0100