OpenJPA upgrade

Explore the updates introduced in Apache Oozie following the upgrade of OpenJPA from version 2.x to 3.x. This upgrade includes new configuration properties, deprecated configuration properties, and enhanced error handling.

Following third-party libraries were upgraded in the Apache Oozie:


Third-party library name	Upgraded from	Upgraded to
OpenJPA	2.4.2	3.2 and later
commons-dbcp	2.4.2	2.12.0 and later
commons-pool	1.6	2.12.0 and later

For more information about the exact versions that the libraries were upgraded to, see Cloudera runtime parcel's Oozie directory that is, {CDH_PARCELS_DIR}/lib/oozie/lib.

Upgrading these third-party libraries ensures that Oozie works with the latest versions of these libraries, allowing you to have a more stable and secure experience. Also, new configurations are introduced in Oozie to support this upgrade.

Changes in existing Oozie configuration

The default value of the Oozie configuration oozie.service.JPAService.connection.data.source is updated from org.apache.oozie.util.db.BasicDataSourceWrapper to org.apache.commons.dbcp2.BasicDataSource. This modification was made in response to a bug found in the earlier version of commons-dbcp, which led to the implementation of the BasicDataSourceWrapper in Oozie. With the latest commons-dbcp update resolving the underlying issue, the workaround has now been removed from Oozie.

New Oozie configurations

Following new Apache Oozie configurations are introduced to support the OpenJPA upgrade. The following properties can be overridden in the Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml.

oozie.service.JPAService.pool.total.conn

Default value: 10. The default value of this property is the same as oozie.service.JPAService.pool.max.active.conn.

Description: This property enables you to configure the connection pool's capacity to limit the number of open connections. If the number of connections in the pool is equal to this value, then new queries must wait for an existing connection to be released. If the capacity is set to a negative value (unlimited), then the new connections are created as needed, regardless of the number of currently open connections. This property replaces the oozie.service.JPAService.pool.max.active.conn property.

oozie.service.JPAService.pool.min.idle.conn

Default value: 2

Description: This property enables you to set the minimum number of idle connections to be maintained in the pool. If the number of idle connections is less than this value, then new connections are created. If this property is set to zero, then no idle connections are maintained, and new connections are only created as needed. The combined number of idle and active connections must not be more than the total number of allowed connections.

oozie.service.JPAService.validate.db.query

Default value: SELECT COUNT(*) FROM VALIDATE_CONN

Description: SQL query to validate DB connection. This property is ignored if oozie.service.JPAService.validate.db.connection is set to false.

oozie.service.JPAService.custom.<***OPENJPA.CONFIGURATION***>

Description: This property enables you to specify any OpenJPA property. For more information about OpenJPA properties, see OpenJPA documentation. Properties set in Oozie's persistence.xml can also be overridden using this configuration. This property can be repeated, and to set multiple openjpa.* properties, you must define an oozie.service.JPAService.custom.<***OPENJPA.CONFIGURATION***> configuration for each one. These properties are not available by default, but you can create them on-demand if you wish to customize additional OpenJPA configurations.

Following example steps provide more information about how you can override any OpenJPA properties:

You can configure multiple openjpa.* properties in Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml as displayed in the following screenshot.

Example on how to override the transaction isolation in Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml:

oozie.service.JPAService.custom.openjpa.jdbc.TransactionIsolation=serializable

To enable OpenJPA debug logging through log4j, which logs into the Oozie server log files, set the following properties in Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml:

oozie.service.JPAService.custom.openjpa.Log=log4j
oozie.service.JPAService.custom.openjpa.ConnectionFactoryProperties=PrintParameters=true

Add the following log4j properties in Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml:

log4j.category.openjpa.Tool=TRACE
log4j.category.openjpa.Query=TRACE
log4j.category.openjpa.jdbc.SQL=TRACE
log4j.category.openjpa.Runtime=INFO
log4j.category.openjpa.Remote=WARN
log4j.category.openjpa.DataCache=WARN
log4j.category.openjpa.MetaData=WARN
log4j.category.openjpa.Enhance=WARN
log4j.category.openjpa.jdbc.SQLDiag=WARN
log4j.category.openjpa.jdbc.JDBC=WARN
log4j.category.openjpa.jdbc.Schema=WARN

Enhanced database connectivity issue handling

When Oozie is configured to manage the database connectivity issues and if the same operation fails multiple times, then Oozie executes a validation query to check if the database connection is still open. Oozie does not force close the connection if the current retry count is less than half of the maximum retries. For example, if the maximum retries limit is set to 5, Oozie starts validating the connection on the 2nd retry attempt (floor(5/2) = 2). This validation continues until the retries either succeed or reach the maximum limit of 5.

If the connection is found to be dead or closed on the DBMS side, Oozie closes it. Then, OpenJPA automatically requests a new connection from the connection pool. This process ensures that the connection pool remains free of dead connections.