The recent release of Windows 10 includes an Ahead-of-Time (AOT) compilation feature .NET native, following similar moves by Google with ART for Android and Oracle’s AOT compiler for Java is planning. With Apple committed to AOT for Objective-C and Swift, JavaScript remains a prominent language relying on Just-In-Time (JIT) compilation.
This shift towards AOT suggests that JIT’s drawbacks, often outweighing its benefits, are finally being recognized. The industry seems to be acknowledging that JIT compilation, while theoretically appealing, struggles to meet the performance demands of modern systems, particularly in the mobile space.
Overestimated Advantages of JITs
Early claims about JIT performance, like those from IBM Systems Journal on Java Technology in 2000 
, were overly optimistic. Real-world applications, such as IBM’s San Francisco project, revealed significant performance limitations. Similarly, the Java-based TeX rewrite, NTS, faced severe performance issues, humorously adopting a snail as its logo.
However, JIT’s strength in dynamically typed languages, determining variable types at runtime, becomes less significant in statically typed languages like Java and C#. These languages readily provide type information at compile time.
The Price of JIT
JIT compilation comes with notable runtime costs. The compiler, running concurrently with the program, consumes processing time and resources. This impacts startup times, affecting short-lived programs significantly. Additionally, JIT-generated code increases memory pressure due to “dirty” code pages, which require disk writes when memory is scarce.
Addressing these issues often involves complex solutions like multiple compilers with heuristics. While these measures mitigate the problem, they increase complexity without fully resolving it.
Security Implications
JIT compilation poses security risks due to writable and executable memory regions. Consequently, platforms like iOS prohibit JITs except for Apple’s own JavaScript engine, limiting flexibility for developers.
Evolving Hardware Landscape
The rapid advancements in hardware performance have made pure interpreters viable for many tasks. Languages like Python, even with its known performance limitations, are often sufficient for numerous applications.
Early JIT implementations, such as Peter Deutsch’s PS (Portable Smalltalk), emerged decades ago. Yet, systems like Squeak demonstrated the effectiveness of combining interpreters with native primitives for performance-intensive operations.
Hybrid Models and Optimization Strategies
John Ousterhout, in his paper Scripting: Higher-Level Programming for the 21st Century, advocated for a hybrid approach where high-level scripting languages utilize high-performance code written in systems languages. This approach, evident in systems like Numpy and even supercomputing facilities using Tcl ("computational steering"), highlights the limitations of JITs for both scripting and systems-level tasks.
This aligns with Donald Knuth’s caution against premature optimization, as highlighted in his 1974 work, “Structured Programming with go to Statements” Structured Programming with go to Statements. The essence of Knuth’s argument is that optimization efforts should be directed towards the critical 3% of code where performance truly matters.
Objective-C, despite its hybrid nature, exemplifies how combining scripting-like features with compiled performance can be successful. However, Apple’s attempts to mold it into a Java-like language have, unfortunately, hindered its strengths.
The Unpredictability of JIT
A significant drawback of JIT compilation is its unpredictable performance model. The timing of optimizations and compilation, along with their effectiveness, can fluctuate, making performance analysis challenging and often statistical. This unpredictability poses problems for real-time systems where meeting deadlines consistently is paramount.
This unpredictability was evident in the shift towards pre-rendered bitmaps in pre-press workflows during the 1990s. Despite the inherent inefficiency of this approach, it guaranteed predictable worst-case performance, crucial for timely newspaper production.
Objective-C, despite its performance overhead compared to C++, often outperformed it in real-world projects. This was attributed to its simpler, more predictable performance model, making optimization and tuning more manageable.
The Illusion of “Sufficiently Smart Compilers”
The pursuit of advanced JIT optimizations, while well-intentioned, often exacerbates the problem of unpredictability. The complexity introduced by such optimizations makes it increasingly difficult to reason about and control performance.
Even AOT compilers like those used for Swift, while not as susceptible to runtime fluctuations, suffer from the same issue. The more the compiler tries to do, the less predictable the outcome becomes.
The Waning of JIT
While revolutionary in concept, JIT compilation’s applicability has been gradually diminishing due to technological advancements, shifting performance needs, and the realization that predictability often trumps average speed. The industry is gradually transitioning towards AOT compilation, favoring simplicity, speed, and predictable performance.
The notion of writing performance-agnostic high-level code and relying on “sufficiently smart compilers” to handle optimization is flawed. A more viable approach involves a hybrid model, empowering programmers to guide performance-critical sections without resorting to complete code rewrites.
This leads to the question: are JIT compilers nearing obsolescence, or is this a misjudgment? Should developers have more direct control over performance, or is trusting advanced compilers the way forward?
Update: Nov. 13th 2017
The Mono Project’s decision to add a byte-code interpreter announced highlights the realization that interpretation can sometimes outperform JIT compilation.
Update: Mar. 25th 2021
Facebook’s AOT compiler for JavaScript, Hermes, demonstrates significant performance and memory usage improvements, further validating the shift towards AOT.
Google’s efforts with Ignition (a JavaScript bytecode interpreter) and JIT-Less (a purely interpreted JavaScript engine) also highlight the performance potential of these approaches.
WebAssembly, designed for predictable high performance on the web, reinforces the demand for predictable execution.
Furthermore, the adoption of GraalVM’s AOT capabilities and the success of Apple’s AOT-based Rosetta 2 translator underscore the industry’s preference for predictability and performance.
Update: Oct. 3rd 2021
Microsoft’s exploration of “SDSM” (Super Duper Secure Mode), effectively disabling JavaScript’s JIT engine, addresses security vulnerabilities associated with JIT. Interestingly, the performance impact is less severe than anticipated, even for real-world tasks.
Update: Mar. 30th 2023
Casey Muratori’s observation about modern software performance reaching a “beyond parody” stage https://t.co/jA0jSc8rsp March 28, 2023 points to the increasing performance overhead of modern software, with JIT compilation being a contributing factor.
This observation aligns with user jigawatts’ analysis of Microsoft Teams’ startup performance, suggesting that JIT compilation significantly contributes to the lag. This lag appears to be a root cause of performance issues in Visual Studio launch time as well.
Update: Apr. 1st 2023
Even Julia, a language known for its performance, struggles with latency attributed to JIT compilation, as noted in a recent post discussing its startup and package loading times Julia’s latency: Past, present and future.
These recent developments further reinforce the argument that while JIT compilation was a groundbreaking concept, its limitations are becoming increasingly apparent in a landscape demanding predictability, security, and efficient resource utilization.