◐ Shell
clean mode source ↗

The Java HotSpot Performance Engine Architecture

Overview

The Java HotSpot Virtual Machine is Sun's VM for the Java platform. It delivers the optimal performance for Java applications using many advanced techniques, incorporating a state-of-the-art memory model, garbage collector, and adaptive optimizer. It is written in a high-level, object-oriented style, and features:

  • Uniform object model
  • Interpreted, compiled, and native frames all use the same stack
  • Preemptive multithreading based on native threads
  • Accurate generational and compacting garbage collection
  • Ultra-fast thread synchronization
  • Dynamic deoptimization and aggressive compiler optimizations
  • System-specific runtime routines generated at VM startup time
  • Compiler interface supporting parallel compilations
  • Run-time profiling focuses compilation effort only on "hot" methods

The JDK includes two flavors of the VM -- a client-side offering, and a VM tuned for server applications. These two solutions share the Java HotSpot runtime environment code base, but use different compilers that are suited to the distinctly unique performance characteristics of clients and servers. These differences include the compilation inlining policy and heap defaults.

The JDK contains both of the these systems in the distribution, so developers can choose which system they want by specifying -client or -server.

Although the Server and the Client VMs are similar, the Server VM has been specially tuned to maximize peak operating speed. It is intended for executing long-running server applications, which need the fastest possible operating speed more than a fast start-up time or smaller runtime memory footprint.

The Client VM compiler serves as an upgrade for both the Classic VM and the just-in-time (JIT) compilers used by previous versions of the JDK. The Client VM offers improved run time performance for applications and applets. The Java HotSpot Client VM has been specially tuned to reduce application start-up time and memory footprint, making it particularly well suited for client environments. In general, the client system is better for GUIs.

The Client VM compiler does not try to execute many of the more complex optimizations performed by the compiler in the Server VM, but in exchange, it requires less time to analyze and compile a piece of code. This means the Client VM can start up faster and requires a smaller memory footprint.

The Server VM contains an advanced adaptive compiler that supports many of the same types of optimizations performed by optimizing C++ compilers, as well as some optimizations that cannot be done by traditional compilers, such as aggressive inlining across virtual method invocations. This is a competitive and performance advantage over static compilers. Adaptive optimization technology is very flexible in its approach, and typically outperforms even advanced static analysis and compilation techniques.

Both solutions deliver extremely reliable, secure, and maintainable environments to meet the demands of today's enterprise customers.

Memory Model

Handleless Objects

In previous versions of the Java virtual machine, such as the Classic VM, indirect handles are used to represent object references. While this makes relocating objects easier during garbage collection, it represents a significant performance bottleneck, because accesses to the instance variables of Java programming language objects require two levels of indirection.

In the Java HotSpot VM, no handles are used by Java code. Object references are implemented as direct pointers. This provides C-speed access to instance variables. When an object is relocated during memory reclamation, the garbage collector is responsible for finding and updating all references to the object in place.

Two-Word Object Headers

The Java HotSpot VM uses a two machine-word object header, as opposed to three words in the Classic VM. Since the average Java object size is small, this has a significant impact on space consumption -- saving approximately eight percent in heap size for typical applications. The first header word contains information such as the identity hash code and GC status information. The second is a reference to the object's class. Only arrays have a third header field, for the array size.

Reflective Data are Represented as Objects

Classes, methods, and other internal reflective data are represented directly as objects on the heap (although those objects may not be directly accessible to Java technology-based programs). This not only simplifies the VM internal object model, but also allows classes to be collected by the same garbage collector used for other Java programming language objects.

Native Thread Support, Including Preemption and Multiprocessing

Per-thread method activation stacks are represented using the host operating system's stack and thread model. Both Java programming language methods and native methods share the same stack, allowing fast calls between the C and Java programming languages. Fully preemptive Java programming language threads are supported using the host operating system's thread scheduling mechanism.

A major advantage of using native OS threads and scheduling is the ability to take advantage of native OS multiprocessing support transparently. Because the Java HotSpot VM is designed to be insensitive to race conditions caused by preemption and/or multiprocessing while executing Java programming language code, the Java programming language threads will automatically take advantage of whatever scheduling and processor allocation policies the native OS provides.

Garbage Collection

The generational nature of the Java HotSpot VM's memory system provides the flexibility to use specific garbage collection algorithms suited to the needs of a diverse set of applications. The Java HotSpot VM supports several different garbage collection algorithms designed to serve different pause time and throughput requirements.

Background

A major attraction of the Java programming language for programmers is that it is the first mainstream programming language to provide built-in automatic memory management, or garbage collection (GC). In traditional languages, dynamic memory is allocated using an explicit allocate/free model. In practice, this turns out to be not only a major source of memory leaks, program bugs, and crashes in programs written in traditional languages, but also a performance bottleneck and a major impediment to modular, reusable code. (Determining free points across module boundaries is nearly impossible without explicit and hard-to-understand cooperation between modules.) In the Java programming language, garbage collection is also an important part of the "safe" execution semantics required to support the security model.

A garbage collector automatically handles freeing of unused object memory behind the scenes by reclaiming an object only when it can prove that the object is no longer accessible to the running program. Automation of this process completely eliminates not only the memory leaks caused by freeing too little, but also the program crashes and hard-to-find reference bugs caused by freeing too much.

Traditionally, garbage collection has been considered an inefficient process that impeded performance, relative to an explicit-free model. In fact, with modern garbage collection technology, performance has improved so much that overall performance is actually substantially better than that provided by explicit freeing of objects.

The Java HotSpot Garbage Collector

In addition to including the state-of-the-art features described below, the memory system is designed as a clean, object-oriented framework that can easily be instrumented, experimented with, or extended to use new garbage collection algorithms.

The major features of the Java HotSpot garbage collector are presented below. Overall, these capabilities are well-suited both for applications where the highest possible performance is needed, and for long-running applications where memory leaks and memory inaccessibility due to fragmentation are highly undesirable.

Accuracy

The Java HotSpot garbage collector is a fully accurate collector.In contrast, many other garbage collectors are conservative or partially accurate. While conservative garbage collection can be attractive because it is very easy to add to a system without garbage collection support, it has certain drawbacks. In general, conservative garbage collectors are prone to memory leaks, disallow object migration, and can cause heap fragmentation.

A conservative collector does not know for sure where all object references are located. As a result, it must be conservative by assuming that memory words that appear to refer to an object are in fact object references. This means that it can make certain kinds of mistakes, such as confusing an integer for an object pointer. Memory cells that look like a pointer are regarded as a pointer -- and GC becomes inaccurate. This has several negative impacts. First, when such mistakes are made (which in practice is not very often), memory leaks can occur unpredictably in ways that are virtually impossible for application programmers to reproduce or debug. Second, since it might have made a mistake, a conservative collector must either use handles to refer indirectly to objects -- decreasing performance -- or avoid relocating objects, because relocating handleless objects requires updating all references to the objects. This cannot be done if the collector does not know for sure whether an apparent reference is a real reference. The inability to relocate objects causes object memory fragmentation and, more importantly, prevents use of the advanced generational copying collection algorithms described below.

Because the Java HotSpot collector is fully accurate, it can make several strong design guarantees that a conservative collector cannot make:

  • All inaccessible object memory can be reclaimed reliably.
  • All objects can be relocated, allowing object memory compaction, which eliminates object memory fragmentation and increases memory locality.

    An accurate garbage collection mechanism avoids accidental memory leaks, enables object migration, and provides for full heap compaction. The GC mechanism in the Java Hotspot VM scales well to very large heaps.

Generational Copying Collection

The Java HotSpot VM employs a state-of-the-art generational copying collector, which provides two major benefits:

  • Increased allocation speed and overall garbage collection efficiency for most programs, compared to nongenerational collectors
  • Corresponding decrease in the frequency and duration of user-perceivable garbage collection pauses

A generational collector takes advantage of the fact that in most programs,the vast majority of objects (often greater than 95 percent) are very short lived (for example, they are used as temporary data structures). By segregating newly created objects into an object nursery, a generational collector can accomplish several things. First, because new objects are allocated contiguously in stack-like fashion in the object nursery, allocation becomes extremely fast, since it merely involves updating a single pointer and performing a single check for nursery overflow. Secondly, by the time the nursery overflows, most of the objects in the nursery are already dead, allowing the garbage collector to simply move the few surviving objects elsewhere, and avoid doing any reclamation work for dead objects in the nursery.

Parallel Young Generation Collector

The single-threaded copying collector described above, while suitable for many deployments, could become a bottleneck to scaling in an application that is otherwise parallelized to take advantage of multiple processors. To take full advantage of all available CPUs on a multiprocessor machine, the Java HotSpot VM offers an optional multithreaded collector for the young generation, in which the tracing and copying of live objects is accomplished by multiple threads working in parallel. The implementation has been carefully tuned to balance the collection work between all available processors, allowing the collector to scale up to large numbers of processors. This reduces the pause times for collecting young space and maximizes garbage collection throughput. The parallel collector has been tested with systems containing more than 100 CPU's and 0.5 terabytes of heap. The parallel young generation collector is the default garbage collection algorithm used with the Server VM.

When moving objects, the parallel collector tries to keep related objects together, resulting in improved memory locality and cache utilization, and leading to improved mutator performance. This is accomplished by copying objects in depth first order.

The parallel collector also uses available memory more optimally. It does not need to keep a portion of the old object space in reserve to guarantee space for copying all live objects. Instead, it uses a novel technique to speculatively attempt to copy objects. If old object space is scarce this technique allows the collector to switch smoothly to compacting the heap without the need for holding any space in reserve. This results in better utilization of the available heap space.

Finally, the parallel collector is able to dynamically adjust its tunable parameters in response to the application's heap allocation behavior, leading to improved garbage collection performance over a wide range of applications and environments. This means less hand-tuning work for customers. This capability was first introduced with the parallel collector and is now available for many of the other garbage collection algorithms.

In comparison with the default single-threaded collector, the break-even point for the parallel collector appears to be somewhere between two and four CPUs, depending on the platform and the application. This is expected to further improve in future releases.

Mark-Compact Old Object Collector

Although the generational copying collector collects most dead objects efficiently, longer-lived objects still accumulate in the old object memory area. Occasionally, based on low-memory conditions or programmatic requests, an old object garbage collection must be performed. The Java HotSpot VM by default uses a standard mark-compact collection algorithm, which traverses the entire graph of live objects from its roots, then sweeps through memory, compacting away the gaps left by dead objects. By compacting gaps in the heap, rather than collecting them into a freelist, memory fragmentation is eliminated, and old object allocation is streamlined by eliminating freelist searching.

Mostly Concurrent Mark-Sweep Collector

For applications that require large heaps, collection pauses induced by the default old generation mark-compact collector can often cause disruptions, as application threads are paused for a period that is proportional to the size of the heap. The Java HotSpot VM has implemented an optional concurrent collector for the old object space that can take advantage of spare processor cycles (or spare processors) to collect large heaps while pausing the application threads for very short periods. This is accomplished by doing the bulk of the tracing and sweeping work while the application threads are executing. In some cases, there may be a small decline in peak application throughput as some processor cycles are devoted to concurrent collection activity; however, both average-and worst-case garbage collection pause times are often reduced by one or two orders of magnitude, allowing much smoother application response without the burstiness sometimes seen when the default synchronous mark-compact algorithm operates on large heaps.

Parallel Old Generation Collector

The current version of the Java HotSpot VM introduces a parallel mark-compact collector for the old generation designed to improve scalability for applications with large heaps. Where the concurrent mark-sweep collector focuses on decreasing pause times, the parallel old collector focuses on increasing throughput by using many threads simultaneously to collect the old generation during stop-the-world pauses. The parallel old collector uses many novel techniques and data structures internally to achieve high scalability while retaining the benefits of exact garbage collection and with a minimum of bookkeeping overhead during collection cycles.

For More Information

For more information on the garbage collection algorithms supported in the Java HotSpot VM, please refer to the memory management white paper (PDF)

Ultra-Fast Thread Synchronization

The Java programming language allows for use of multiple, concurrent paths of program execution -- threads. The Java programming language provides language-level thread synchronization, which makes it easy to express multithreaded programs with fine-grained locking. Previous synchronization implementations such as that in the Classic VM were highly inefficient relative to other micro-operations in the Java programming language, making use of fine-grain synchronization a major performance bottleneck.

The Java HotSpot VM incorporates leading-edge techniques for both uncontended and contended synchronization operations which boost synchronization performance by a large factor. Uncontended synchronization operations, which dynamically comprise the majority of synchronizations, are implemented with ultra-fast, constant-time techniques. With the latest optimizations, in the best case these operations are essentially free of cost even on multiprocessor computers. Contended synchronization operations use advanced adaptive spinning techniques to improve throughput even for applications with significant amounts of lock contention. As a result, synchronization performance becomes so fast that it is not a significant performance issue for the vast majority of real-world programs.

64-bit Architecture

Early releases of the Java HotSpot VM were limited to addressing four gigabytes of memory -- even on 64-bit operating systems such as the Solaris OE. While four gigabytes is a lot for a desktop system, modern servers can contain far more memory. For example, the Sun Fire E25K server supports up to 1.15 terabytes of memory per domain. With a 64-bit JVM, Java technology-based applications can now utilize the full memory of such a system.

There are several classes of applications where using 64-bit addressing can be useful. For example, those that store very large data sets in memory. Applications can now avoid the overhead of paging data from disk or extracting it from a RDBMS. This can lead to dramatic performance improvements in applications of this type.

The Java HotSpot VM is now 64-bit safe, and the Server VM includes support for both 32-bit and 64-bit operations. Users can select either 32-bit or 64-bit operation by using commandline flags -d32 or -d64 respectively. Users of the Java Native Interface will need to recompile their code to run it on the 64-bit VM.

Object Packing

Object packing functionality has been added to minimize the wasted space between data types of different sizes. This is primarily a benefit in 64-bit environments, but offers a small advantage even in 32-bit VMs.

For example: 



  public class Button {
  char shape;
  String label;
  int xposition;
  int yposition;
  char color;
  int joe;
  object mike;
  char armed;
}

This would waste space between: color and joe (three bytes to pad to an int boundary) joe and mike (four bytes on a 64-bit VM to pad to a pointer boundary) Now, the fields are reordered to look like this:



  ...
  object mike;
  int joe;
  char color;
  char armed;
...

In this example, no memory space is wasted.