9.3 Popular Threading Implementations - Java Threads, Third Edition

9.3 Popular Threading Implementations

We'll now look at how all of this plays out in the implementation of the Java virtual machine on several popular platforms. In many ways, this is a section that we'd rather not have to write: Java is a platform-independent language and to have to provide platform-specific details of its implementations certainly violates that precept. But we stress that these details actually matter in very few cases. This section is strictly for informational purposes.

9.3.1 Green Threads

The first model that we'll look at is the simplest. In this model, the operating system doesn't know anything about Java threads at all; it is up to the virtual machine to handle all the details of the threading API. From the perspective of the operating system, there is a single process and a single thread.

Each thread in this model is an abstraction within the virtual machine: the virtual machine must hold within the thread object all information related to that thread. This includes the thread's stack, a program counter that indicates which Java instruction the thread is executing, and other bookkeeping information about the thread. The virtual machine is then responsible for switching thread contexts: that is, saving this information for one particular thread, loading it from another thread, and then executing the new thread. As far as the operating system is concerned, the virtual machine is just executing arbitrary code; the fact that the code is emulating many different threads is unknown outside of the virtual machine.

This model is known in Java as the green thread model. In other circles, these threads are often called user-level threads because they exist only within the user level of the application: no calls into the operating system are required to handle any thread details.

User- and System-Level Threads In most operating systems, the operating system is logically divided into two pieces: user and system level. The operating system itself梩hat is, the operating system kernel條ies at the system level. The kernel is responsible for handling system calls on behalf of programs run at the user level. When a program running at user level wants to read a file; for example, it must call (or trap) into the operating system kernel, which reads the file and returns the data to the program. This separation has many advantages, not the least of which is that it allows for a more robust system: if a program performs an illegal operation, it can be terminated without affecting other programs or the kernel itself. Only when the kernel executes an illegal operation does the entire machine crash. Because of this separation, it is possible to have support for threads at the user level, the system level, or at both levels independently.

In the early days of Java, the green thread model was fairly common, particularly on most Unix platforms. Some specialized operating systems today use this model, but most computers use a native, system-level model.

The green thread model is completely deterministic with respect to scheduling. Running our priority calculation above, we see this output:

Starting task Task 5 at 07:23:12:074

Ending task Task 5 at 07:23:12:995 after 921 milliseconds

Starting task Task 4 at 07:23:13:111

Starting task Task 6 at 07:23:13:281

Ending task Task 6 at 07:23:14:256 after 975 milliseconds

Starting task Task 7 at 07:23:14:386

Ending task Task 7 at 07:23:15:398 after 1012 milliseconds

Starting task Task 8 at 07:23:15:504

Ending task Task 8 at 07:23:16:567 after 963 milliseconds

Starting task Task 9 at 07:23:16:624

Ending task Task 9 at 07:23:17:699 after 1075 milliseconds

Ending task Task 4 at 07:23:18:912 after 5801 milliseconds

Starting task Task 3 at 07:23:19:114

Ending task Task 3 at 07:23:20:177 after 1063 milliseconds

Starting task Task 2 at 07:23:20:301

Ending task Task 2 at 07:23:21:305 after 1004 milliseconds

Starting task Task 1 at 07:23:21:486

Ending task Task 1 at 07:23:22:449 after 963 milliseconds

As soon as the thread with priority 6 (task 5) becomes runnable, the green thread scheduler runs it, and all threads must wait. That includes the main thread, which cannot go on to create a higher-priority thread. This is why the priority 9 thread runs after the priority 6-8 threads have finished: the main thread cannot create the priority 9 thread because it runs at a priority of 5 and is blocked by the threads at priority 6-8. Task 4 gets to run occasionally when the main thread is blocked, and it eventually completes after very high-priority task 9.

9.3.2 Windows Native Threads

In the native-threading model used on 32-bit Windows operating systems, the OS is fully cognizant of the multiple threads that the virtual machine uses, and there is a one-to-one mapping between Java threads and operating system threads. Therefore, the scheduling of Java threads is subject to the underlying scheduling of threads by the operating system.

This model is usually simple to understand because every thread can be thought of as a process. The OS scheduler makes no real distinction in this case between a process and a thread: it treats each thread like a process. Of course, there are still other differences in the OS between a thread and a process, but not as far as the scheduler is concerned.

Windows operating systems use a complex priority calculation to determine which thread should be the currently running thread. That calculation takes into account the Windows thread priority. This is very similar to the Java-level thread priority between 0 and 10, except that Windows provides only 7 priorities. Therefore, some overlap occurs as Java's 11 logical priorities are mapped to Windows 7 actual priorities. Different implementations of the virtual machine do this differently, but one common implementation performs the mapping listed in Table 9-1.

Table 9-1. Mapping of Java thread priorities on Win32 platforms

Java priority	Win32 priority
0	THREAD_PRIORITY_IDLE
1 (Thread.MIN_PRIORITY)	THREAD_PRIORITY_LOWEST
2	THREAD_PRIORITY_LOWEST
3	THREAD_PRIORITY_BELOW_NORMAL
4	THREAD_PRIORITY_BELOW_NORMAL
5 (Thread.NORM_PRIORITY)	THREAD_PRIORITY_NORMAL
6	THREAD_PRIORITY_ABOVE_NORMAL
7	THREAD_PRIORITY_ABOVE_NORMAL
8	THREAD_PRIORITY_HIGHEST
9	THREAD_PRIORITY_HIGHEST
10 (Thread.MAX_PRIORITY)	THREAD_PRIORITY_TIME_CRITICAL

On this implementation, a thread with a Java priority of 3 and one with a Java priority of 4 have the same effective priority.

In addition to 7 priority levels, Windows operating systems also have 5 scheduling classes, and a thread is actually scheduled as a combination of its priority and its scheduling class. However, scheduling classes are not easy to change, so they do not factor into a discussion of Java threads.

Windows operating systems also use a complex priority calculation that includes the following:

• Threads are subject to priority inheritance.

• The actual priority of a thread is based on its programmed (or inverted) priority minus a value that indicates how recently the thread has actually run. This value is subject to continual adjustment: the more time passes, the closer to zero that value becomes. This primarily distinguishes between threads of the same priority, and it leads to round-robin scheduling of threads of the same priority.

• On another level, a thread that has not run for a very long time is given a temporary priority boost. The value of this boost decays over time as the thread has a chance to run. This prevents threads from absolute starvation while still giving preference to higher-priority threads over lower-priority threads. The effect of this priority boost depends on the original priority of the thread.

• Threads running in a program that has keyboard and mouse focus are given a priority boost over threads in other programs.

The upshot of all this is that it's very difficult to guarantee explicitly ordered thread execution on Windows platforms, but the complex priority calculation ensures that threads do not starve.

On Windows operating systems, the output of our priority-based thread calculation looks something like this:

Starting task Task 9 at 21:19:23:590

Starting task Task 8 at 21:19:23:590

Starting task Task 7 at 21:19:23:590

Ending task Task 9 at 21:19:28:750 after 5160 milliseconds

Starting task Task 4 at 21:19:29:470

Ending task Task 8 at 21:19:30:180 after 6590 milliseconds

Starting task Task 2 at 21:19:30:180

Starting task Task 0 at 21:19:30:460

Ending task Task 7 at 21:19:32:050 after 8460 milliseconds

Starting task Task 6 at 21:19:23:590

Starting task Task 5 at 21:19:23:590

Starting task Task 3 at 21:19:30:180

Ending task Task 5 at 21:19:35:950 after 12360 milliseconds

Ending task Task 6 at 21:19:35:950 after 12360 milliseconds

Starting task Task 1 at 21:19:30:180

Ending task Task 4 at 21:19:37:820 after 8350 milliseconds

Ending task Task 2 at 21:19:41:610 after 11430 milliseconds

Ending task Task 3 at 21:19:41:720 after 11540 milliseconds

Ending task Task 0 at 21:19:45:120 after 14660 milliseconds

Ending task Task 1 at 21:19:45:120 after 14940 milliseconds

On this platform, the complex priority calculation places a great deal of emphasis on the Java priority level. In fact, the highest priority tasks finish before some of the lower-priority tasks even have a chance to start. Note also that several Java priority levels map to the same Windows priority level: priorities 6 and 7 (tasks 5 and 6) are given the same priority by the operating system, as are priorities 1 and 2 (tasks 0 and 1).

9.3.3 Solaris Native Threads

Recent versions of the Solaris Operating Environment have had two different threading models. Solaris 7 featured a complex, two-level threading system, with user-level threads and system-level lightweight processes (LWPs). Java threads were equivalent to Solaris user-level threads, and there is an M-to-N mapping between the user-level threads and LWPs. Much of the flexibility of this model is lost on the Java developer, who can directly influence only the priority (and number) of the user-level threads but not the underlying LWPs.

In Solaris 9, a new one-to-one threading model is used. That makes it conceptually similar to the models on Windows operating systems, though its implementation details are quite different.

In Solaris 8, both models are available, and the user picks a model when the Java program (or any other program) is executed.

For Java programs, the one-to-one model is highly preferable, particularly when the machine has multiple CPUs and the Java threads are CPU-intensive. In other cases, the one-to-one threading model is still preferred, though the difference in threading models is not as significant. For this reason, many Java programs run better on Solaris 9 than on Solaris 7. On Solaris 8, you specify the new threading model by setting the environment variable LD_LIBRARY_PATH=/usr/lib/lwp in the shell (or script) in which the Java executable is started.

On Solaris 7, you can mimic some of the benefits of Solaris' new threading model by including these two flags in your Java command line: -Xboundthreads -XX:+UseLWPSynchronization.

The complex priority of a Solaris thread is determined by the following:

• Solaris native threads are subject to priority inheritance.

• The actual priority of a thread is a value from 0 to 59. That value is primarily determined by how long it's been since the thread has run. Though the calculation includes the Java-level priority, other factors dominate the calculation.

• Solaris also includes a variety of scheduling classes. All threads in a single program belong to the same scheduling class, so there is no variability in scheduling among them.

Running our priority-based calculator on Solaris produces this sort of output:

Starting task Task 7 at 21:26:33:040

Starting task Task 0 at 21:26:33:040

Starting task Task 6 at 21:26:33:039

Starting task Task 9 at 21:26:33:039

Starting task Task 4 at 21:26:33:039

Starting task Task 2 at 21:26:33:040

Starting task Task 5 at 21:26:33:039

Starting task Task 3 at 21:26:33:039

Starting task Task 8 at 21:26:33:039

Starting task Task 1 at 21:26:33:039

Ending task Task 6 at 21:27:02:580 after 29541 milliseconds

Ending task Task 1 at 21:27:02:802 after 29763 milliseconds

Ending task Task 4 at 21:27:03:618 after 30579 milliseconds

Ending task Task 7 at 21:27:04:173 after 31133 milliseconds

Ending task Task 0 at 21:27:04:259 after 31219 milliseconds

Ending task Task 9 at 21:27:04:375 after 31336 milliseconds

Ending task Task 3 at 21:27:04:457 after 31418 milliseconds

Ending task Task 5 at 21:27:05:050 after 32011 milliseconds

Ending task Task 8 at 21:27:05:159 after 32120 milliseconds

Ending task Task 2 at 21:27:05:287 after 32247 milliseconds

The lower-priority threads tend to start later than the higher-priority threads, but priority is no assurance of more CPU time: the thread at priority 8 finishes later than almost any other thread. The complex priority calculation being performed by the operating system ensures that all threads get adequate amounts of CPU time.

At an application level, threads on Solaris can have any of 128 priorities (though, as we mentioned, that priority is factored into a complex equation that yields 60 different runnable priorities). These priorities run from 0 to 127, and in C and C++ programs, the default priority for a thread is 127. In Java versions up to and including JDK 1.4, Java thread priorities were mapped to the full range of 128 priorities (0, 12, 24, and so on). This meant that the default priority for a Java thread was in the middle of this range and hence less than the default priority for a C or C++ thread. When a Solaris machine ran a CPU-intensive C program along with a CPU-intensive Java program, the Java program was at a disadvantage and received less than 50% of the available CPU time.

In J2SE 5.0, the mapping was changed and all Java threads with a priority of NORM_PRIORITY and higher are now mapped to a Solaris thread priority of 127. This allows Java and C programs to run at parity.

9.3.4 Linux Native Threads

Until JDK 1.3, Linux-based virtual machines tended to use a green thread model. Some used Linux's native threads, but the kernel support for those threads did not support a large number of concurrent threads. JDK 1.3 added support for Linux native threads. However, the Linux kernel at the time was not optimal for threaded applications; in particular, the ps command listed all threads as if they were different processes.

New Linux kernels use the Native Posix Thread Library (NPTL), which provides the same one-to-one mapping of Java threads to kernel threads that we've seen in other operating systems. The complex priority calculation for those threads is similar to what we saw on Solaris, where the Java priority is only a small factor in the calculation. JDK 1.4.2 is the first version of Java to support this new kernel.