Per-Instance ClassLoading

A component developer may wish to add additional resources to the component's classpath at runtime. For example, you may want to provide the location of a JDBC driver to a processor that interacts with a relational database, thus allowing the processor to work with any driver rather than trying to bundle a driver into the NAR.

This may be accomplished by declaring one or more PropertyDescriptor instances with dynamicallyModifiesClasspath set to true. For example:

         PropertyDescriptor EXTRA_RESOURCE = new PropertyDescriptor.Builder()
   .name("Extra Resources")
   .description("The path to one or more resources to add to the classpath.")
   .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
   .expressionLanguageSupported(true)
   .dynamicallyModifiesClasspath(true)
   .build();
      

When these properties are set on a component, the framework identifies all properties where dynamicallyModifiesClasspath is set to true. For each of these properties, the framework attempts to resolve filesystem resources from the value of the property. The value may be a comma-separated list of one or more directories or files, where any paths that do not exist are skipped. If the resource represents a directory, the directory is listed, and all of the files in that directory are added to the classpath individually. These directories also will be scanned for native libraries. If a library is found in one of these directories, an OS-handled temporary copy is created and cached before loading it to maintain consistency and classloader isolation.

Each property may impose further restrictions on the format of the value through the validators. For example, using StandardValidators.FILE_EXISTS_VALIDATOR restricts the property to accepting a single file. Using StandardValidators.NON_EMPTY_VALIDATOR allows any combination of comma-separated files or directories.

Resources are added to the instance ClassLoader by adding them to an inner ClassLoader that is always checked first. Anytime the value of these properties change, the inner ClassLoader is closed and re-created with the new resources.

NiFi provides the @RequiresInstanceClassLoading annotation to further expand and isolate the libraries available on a component's classpath. You can annotate a component with @RequiresInstanceClassLoading to indicate that the instance ClassLoader for the component requires a copy of all the resources in the component's NAR ClassLoader. When @RequiresInstanceClassLoading is not present, the instance ClassLoader simply has its parent ClassLoader set to the NAR ClassLoader, rather than copying resources.

The @RequiresInstanceClassLoading annotation also provides an optional flag `cloneAncestorResources'. If set to true, the instance ClassLoader will include ancestor resources up to the first ClassLoader containing a controller service API referenced by the component, or up to the Jetty NAR. If set to false, or not specified, only the resources from the component's NAR will be included.

Because @RequiresInstanceClassLoading copies resources from the NAR ClassLoader for each instance of the component, use this capability judiciously. If ten instances of one component are created, all classes from the component's NAR ClassLoader are loaded into memory ten times. This could eventually increase the memory footprint significantly when enough instances of the component are created.

Additionally, there are some restrictions when using @RequiresInstanceClassLoading when using Controller Services. Processors, Reporting Tasks, and Controller Services can reference a Controller Service API in one of its Property Descriptors. An issue may arise when the Controller Service API is bundled in the same NAR with a component that references it or with the Controller Service implementation. If either of these cases are encountered and the extension requires instance classloading, the extension will be skipped and an appropriate ERROR will be logged. To address this issue, the Controller Service API should be bundled in a parent NAR. The service implementation and extensions that reference that service should depend on the Controller Service API NAR. Please refer to the Controller Service NAR Layout in the NiFi Archives (NARs) section. Anytime a Controller Service API is bundled with an extension that requires it, even if @RequiresInstanceClassLoading isn't used, a WARNING will be logged to help avoid this bad practice.