Sqoop jobs fail with unnamed module exception after JDK 17 upgrade

After upgrading to JDK 17, Sqoop jobs may fail due to strict enforcement of the Java Platform Module System (JPMS).

Condition

After upgrading to Cloudera Runtime 7.1.9 SP1 (or later) cluster to JDK 17, Sqoop jobs that previously worked on JDK 8 or JDK 11 begin to fail.

The failures can impact both Sqoop import and export jobs and might disrupt critical workflows.

Depending on the job type, the YARN application logs may show one or more of the following errors:

IllegalAccessError
InaccessibleObjectException
Module access errors involving an unnamed module
Failures referencing restricted Java packages such as:
- java.lang
- java.net
- sun.net.dns
- sun.net.util
- com.sun.jmx.mbeanserver

In some cases:

Sqoop import jobs fail immediately after the JDK 17 upgrade.
Sqoop export jobs continue to fail even after the YARN Application Master JVM options are configured.

Cause

JDK 17 strictly enforces the Java Platform Module System (JPMS), which restricts access to internal JDK APIs that Sqoop and the underlying MapReduce framework rely on.

Sqoop was originally built for earlier Java versions and uses reflection to access these internal APIs. Under JDK 17, this access is blocked unless explicitly allowed using JVM options such as --add-opens and --add-exports.

Additionally:

YARN Application Master JVM options (yarn.app.mapreduce.am.command-opts) do not automatically propagate to MapReduce task JVMs.
For Sqoop export jobs, the map task JVM requires additional module access (for example, java.net) to interact with external databases.
The YARN configuration Add add-opens flags to MR containers is disabled by default, regardless of the JDK or Cloudera version. When using JDK 17, required module flags must be explicitly provided to the relevant JVMs.

As a result, the import jobs fail due to restricted module access in the Application Master, and export jobs fail because the map task JVM does not receive all required module permissions.

Solution

Configure YARN Application Master JVM options (required for all Sqoop jobs).
- Open Cloudera Manager. Cloudera Manager > YARN > Configuration
- Search for the property yarn.app.mapreduce.am.command-opts.
- Add the following JVM arguments:
```
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
--add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
--add-exports=java.base/sun.net.dns=ALL-UNNAMED
--add-exports=java.base/sun.net.util=ALL-UNNAMED
 
```
- Click Save Changes.
- Restart the YARN services.
Provide additional JVM options for Sqoop export jobs.
Sqoop export jobs require additional module access in the map task JVM, which is not covered by the Application Master configuration alone.
When running Sqoop export jobs, explicitly pass the required JVM options using the mapreduce.map.java.opts property, for example:
```
sqoop export \
  -D mapreduce.map.java.opts="--add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED \
  --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED \
  --add-exports=java.base/sun.net.dns=ALL-UNNAMED \
  --add-exports=java.base/sun.net.util=ALL-UNNAMED \
  --add-opens=java.base/java.net=ALL-UNNAMED" \
  --connect ... \
  --table ... \
  --export-dir ...
                            
```
This ensures that the map task JVM can access the required internal Java packages needed for database connectivity during export operations.
Verify the solution.
Re-run the Sqoop import or export job and confirm that the job completes successfully without module-related exceptions.
Verify in YARN logs that no IllegalAccessError or InaccessibleObjectException messages are present.