Section 20.5.  Troubleshooting Classloading Problems - Eclipse Rich Client Platform: Designing, Coding, and Packaging Java Applications

20.5. Troubleshooting Classloading Problems

Most code libraries are quite straightforward to bundle and then use in Eclipse-based systems. You've seen that the wizard to create plug-ins from existing JARs does most of the work for you. But what happens if there are problems after bundling? If you remember from Chapter 10, after bundling Smack, the next thing we did was write a test plug-in to see if Smack could be referenced as a plug-in. At this point, there are two main problems that could occur. The first is at compile timeclasses in the bundled JAR may not be visible. This is easily fixed by ensuring that the plug-in containing the bundled JAR exports all the necessary packages from the JAR. But what if something goes wrong at runtime? The classic symptom is ClassNotFoundExceptions and NoClassDefFoundErrors showing up in the console or the log file.

This entire section is devoted to helping you understand and troubleshoot these runtime errors. Typically, these relate to the classloading structure inherent in Eclipse and OSGi. The OSGi classloading strategy and mechanism is discussed in Chapter 26, but here we detail some standard library coding patterns and how to handle them in Eclipse.

20.5.1. Issues with Class.forName()

Let's start with the classic example of ClassNotFoundExceptions, which occurs while using a bundled code library. Consider adding logging using log4j, a popular library for managing and logging events (http://logging.apache.org/), in Hyperbola. Using the techniques described earlier, you can bundle log4j and add it to either your workspace or target and continue development. At runtime, however, log4j throws a number of ClassNotFoundExceptions when trying to configure its appenders.

log4j is extensible in that it allows clients to supply log appenderseffectively log event handlers. Appenders are configured by naming their implementation classes in metadata files, much like Eclipse plug-ins define extensions. log4j then reads these files and loads the named classes using a code pattern similar to the snippet below:

public class AppenderHelper {
  private Appender createAppender(appenderName) {
    Class appenderClass = Class.forName(appenderName);
    return appenderClass.newInstance();
  }
}

Class.forName(String) is the classic Java mechanism for dynamic class discovery and loading. It uses the current classloader to look for and load the requested class, in this case, an appender. The current classloader is the classloader that loaded the class containing the method executing the forName(String) call. In the snippet above, the current classloader is the one that loaded AppenderHelper. The net result is the same as if a reference to the appender class was compiled into createAppender(). This is exactly what using Class.forName(String) is trying to work around.

In Eclipse, this is problematic because the log4j plug-in typically does not depend on the plug-ins providing the appenders. This is actually the pointappenders are log4j's way of allowing its function to be extended, but the log4j plug-in cannot load these appenders because it does not have the proper visibility.

If log4j was written as a standard Eclipse plug-in, it could, for example, use the Eclipse extension registry and define an appenders extension point. Plug-ins wanting to provide extenders would then contribute executable extensions that name their appender classes and log4j would use createExecutableExtension() (see Chapter 23, "RCP Everywhere") rather than the code in createAppender() above. Unfortunately, log4j is not written as an Eclipse plug-in and this technique is not available.

Buddy classloading offers an alternative integration strategy that does not require code modification. The mechanism works as follows:

• Plug-ins declare that they need the help of other plug-ins to load classes.

• They also identify the kind of help they want by specifying a buddy policy. The policy defines what kinds of plug-ins are to be considered to be buddies as well as how (e.g., in what order) they are consulted.

• When a plug-in fails to find a desired class through all the normal routes as outlined in Section 26.10, "Classloading" (i.e., Import-Package, Require-Bundle, and local classes), its buddy policy is invoked.

• The invoked policy discovers a set of buddies and consults each one in turn until either the class is found or the list is exhausted.

Let's apply the built-in registered buddy policy to the log4j case and see how it helps. In the log4j scenario, there are a relatively large number of potential clients of the logging API and a small number of clients supplying appenders. For performance and simplicity, it makes sense to limit the buddy search scope to just those supplying appenders. The simplest approach is to make those plug-ins explicitly register as buddies of log4j.

To set this up, you first mark the log4j plug-in as needing classloading help and identify the "registered" policy as the policy to use. The following line added to log4j's MANIFEST.MF makes that declaration:

Eclipse-BuddyPolicy: registered

Then in each plug-in that supplies appenders, add the following line to the MANIFEST.MF to register the plug-in as a buddy of log4j (i.e., org.apache.log4j):

Eclipse-RegisterBuddy: org.apache.log4j

At runtime, when log4j goes to instantiate an appender using Class.forName(String), it first tries all its normal prerequisites. Then, when it fails to find the appender class, each of its registered buddy plug-ins is asked to load the class. If all the appender plug-ins are registered, the appender class is sure to be found.

20.5.1.1. Built-in Buddy Policies

Eclipse supplies a number of built-in policies, as summarized in Table 20-1:

Table 20-1. Built-in Buddy Policies

Name	Description
boot	Indicates that the standard Java boot classloader is a buddy.
ext	Indicates that the standard Java extension classloader is a buddy. This policy is a superset of the boot policy.
app	Indicates that the standard Java application classloader is a buddy. This policy is a superset of the ext policy.
parent	Indicates that the plug-in's parent classloader is a buddy. By default, the parent classloader is the standard Java boot classloader. Plug-in classloader parentage is controlled on a global basis by setting the osgi.parentClassloader system property.
dependent	Consults all plug-ins that directly or indirectly depend on the current plug-in. Note that this casts a rather wide net and may introduce performance problems as the number of plug-ins increases.
registered	Is similar to the dependent policy, but only dependent plug-ins that have explicitly registered themselves as buddies of the current plug-in are consulted.

One plug-in can apply several policies simply by listing them on the Eclipse-BuddyPolicy line in the MANIFEST.MF. Eclipse invokes each policy in turn until either the class is found or all policies have been consulted.

20.5.1.2. Buddy Classloading Considerations

As powerful and useful as buddy classloading is, it is still a mechanism of last resort. There are a number of issues that you should consider carefully before using buddies in your system:

• Buddy classloading runs counter to the notions of component that Eclipse attempts to maintain and is not particularly well-suited to dynamic environmentsparticularly ones where buddies can be uninstalled.

• Buddy classloading also incurs various performance costs. For example, when a normal classload fails in a plug-in using buddy loading, the buddy policy is invoked. Typical Java resource bundle loading causes up to three classload failures and some number of resource load failures before finally getting the desired resource. Each of these failures repeats the buddy search.

• Buddy loading is relatively undirected. Normally, the OSGi classloading infrastructure knows exactly where to go to find any given packagethis is the information gleaned from the MANIFEST.MF files. Typical buddy loading policies simply search successive buddies.

• It is possible that the buddy search will find the wrong class with the right name. If two buddies contain the same class, depending on the policy, it may be ambiguous as to which buddy ultimately supplies the class.

20.5.1.3. Dynamic-ImportPackage vs. Buddy Classloading

Readers familiar with OSGi may be scratching their heads asking, "What about Dynamic-ImportPackage?" For readers who are not familiar with OSGi, Dynamic-ImportPackage is a mechanism that allows a bundle to state its need to use a given set of packages but not force an early binding to the exporters of those packages. Rather, the binding to package exporters is done at runtime when the bundle tries to load from a dynamically imported package.

So, some Class.forName() problems can be alleviated simply by adding

DynamicImport-Package: <list of packages or *>

to the MANIFEST.MF for the plug-in using Class.forName(String). This has the following drawbacks compared to the buddy loading described here:

• Dynamic importing is unscoped. That is, all bundles exporting packages are considered. As such, the search may include many irrelevant and unrelated bundles. By contrast, the buddy loading mechanism allows for policies that use dynamic information such as the plug-in dependency graph to drive the search for classes.

• Dynamic importing implies inter-bundle constraints. That is, when a bundle A loads a class from a bundle B using dynamic importing, A is then considered to be dependent on B. If B is refreshed or uninstalled, A is refreshed. This behavior is valuable for maintaining consistency when A actually uses and retains references to B's classes. However, several serialization scenarios have A simply using B's classes temporarily (e.g., to load some object stream)there should be no lasting dependency.

• Dynamic import considers only packages explicitly exported by other bundles. Again this can be a desirable characteristic, but in various use cases such as serialization, the importing bundle potentially needs access to all classes in the system, for example, to load instances from an object stream.

This is not to say that Dynamic-ImportPackage should never be used, just that it should be used appropriately. For example, when the set of packages needed is well-known and the importing bundle has a lasting dependency on the imported packages.

20.5.2. Issues with Context Classloaders

Since Java 1.2, the Class.forName(String) mechanism has been largely superseded by context classloading. As such, most modern class libraries use a context classloader. In the discussion below, we show how Eclipse transparently converts the use of context classloaders into something equivalent to Class.forName (String). Doing this allows the buddy loading and Dynamic-Import mechanisms described above to be used to eliminate ClassNotFoundExceptions and NoClassDefFoundErrors.

Each Java Thread has an associated context classloader field that contains a classloader. The classloader in this field is set, typically by the application container, to match the context of this current execution. That is, the field contains a classloader that has access to the classes related to the current execution (e.g., Web request being processed). Libraries such as log4j access and use the context classloader with the updated AppenderHelper code pattern below:

public class AppenderHelper {
  private Appender createAppender(String appenderName) {
    ClassLoader loader = Thread.currentThread().getContextClassLoader();
    Class appenderClass = loader.loadClass(appenderName);
    return (Appender)appenderClass.newInstance();
  }
}

By default, the context classloader is set to be the normal Java application classloader. That is, the use of the context classloader in normal Java application scenarios is equivalent to using Class.forName(String) and there is only one classloader, the application classloader. When running inside Eclipse, however, the code pattern outlined above fails because:

• By default, Eclipse does not consult the application classloader. Eclipse-based applications put their code on dynamic plug-in classpaths rather than on the normal Java application classpath.

• Eclipse cannot detect plug-in context switches and set the context classloader as required. That is, there is no way to tell when execution context shifts from one plug-in to the next as is done in Web application servers.

These characteristics, combined with the compositional nature of Eclipse, mean that the value of the context classloader field is seldom useful.

Clients can, however, explicitly set the context classloader before calling libraries that use the context classloader. The snippet below shows an example of calling log4j using this approach:

Thread thread = Thread.currentThread();
ClassLoader loader = thread.getContextClassLoader();
thread.setContextClassLoader(this.getClass().getClassLoader());
try {
  ... log4j library call that calls AppenderHelper.createAppender() ...
} finally {
  thread.setContextClassLoader(loader);
}

First the current context classloader is saved. The context classloader is then set to an appropriate value for the current execution and log4j is called. log4j's AppenderHelper uses the context classloader, so in this case, it uses the client's classloader (e.g., this.getClass().getClassLoader()). When the operation is finished, the original context classloader is restored.

The assumption here is that the client's classloader is able to load all required classes. This may or may not be true. Even if it can, the coding pattern is cumbersome to use and hard to maintain for any significant number of library calls. Ideally, log4j would be able to dynamically discover the context relevant to a particular classloading operation. Eclipse enables this using the context finder.

The context finder is a kind of ClassLoader that is installed by Eclipse as the default context classloader. When invoked, it searches down the Java execution stack for a classloader other than the system classloader. In the AppenderHelper example above, it finds the log4j plug-in's classloaderthe one that loaded AppenderHelper. The context finder then delegates the load request to the discovered classloader.

This mechanism transforms log4j's call to getContextClassLoader(). loadClass(String) to the equivalent Class.forName(String) call using log4j's classloader to load the given class. Now the buddy classloading techniques discussed in Section 20.5.1 can be applied to help log4j load the needed appender classes.

The net effect is that clients of log4j do not have to use the cumbersome coding pattern outlined above even though the libraries they call use the context classloader. This approach generalizes to other context classloading situations.

20.5.3. Managing JRE Classes

For various reasons, some libraries include packages that are normally found in the JRE. For example, version 2.6 of Xalan, the XML transformation engine, comes with types from the org.w3c.dom.xpath package in xalan.jar. These types are also included as part of typical JRE distributions. When xalan.jar is used as part of a normal Java application, it is added to the classpath, but its xpath classes are obscured by those in the JRE. Everything is fine.

When you bundle Xalan, the tooling produces Export-Package enTRies for all packages in xalan.jar. However, the tooling cannot know that it should add imports for the org.w3c.dom.xpath package found in the JRE. Without the import, Xalan uses its own copies of the xpath types and may conflict with those supplied by the JRE.

This happens because Eclipse 3.1 plug-in classloading is highly optimized. These optimizations depend on the plug-in manifest information to know which packages come from which plug-ins. Except for the use of certain buddy policies, the classloaders never search for classesthey always knows exactly where to find them.

For the JRE packages, only java.* packages are assumed to come from the boot classloader. All others must be imported in the consuming plug-in's MANIFEST.MF. The API packages included in the JRE are typically exported by the system bundle (i.e., org.eclipse.osgi or system.bundle). This list is captured in a JRE profile. Eclipse includes a number of profiles for common JREs and automatically detects the appropriate one to use. You can control this further by setting the osgi.java.profile property to the URL of a profile file to use.

So, if the Xalan plug-in fails to import the xpath packages, its local copies are used. This may result in ClassCastExceptions because the plug-in's copy of the type is not interchangeable with the copy supplied by the JRE. Changing the plug-in to import the packages tells Eclipse to use the external copy. Alternatively, the offending packages can be removed from Xalan.

20.5.4. Serialization

Serialization of objects occurs in many different situations. Some libraries use the built-in java.io.Serializable mechanism directly. Some use it indirectly as a consequence of using Remote Method Invocation (RMI). Others serialize objects using their own marshalling strategies (e.g., Hibernate stores/loads objects to/from relational databases). Regardless of the technique used, these plug-ins have the following characteristics:

• They are typically generic utilities and do not have access to, or knowledge of, your domain classes.

• They do not hold onto the classes they request, but rather use them to load objects and then discard their references.

• They need access to internal classes if instances of internal classes have been serialized.

Buddy classloading and context classloading solve all these problems. In effect, loading a serialized object is equivalent to the log4j appender problem. Appender classes are identified by name to log4j. Classes to load are identified to the serialization plug-in by name in the object stream. In both cases, the loading plug-in needs to search beyond its prerequisite plug-ins to find the desired classes.