Known issues and limitations

Known issues

Cloudera DataFlow Functions: DEFAULT_PARAM_CONTEXT variable no longer works alone

This issue only occurs in AWS environments and affects configurations where there are no PARAM_CONTEXT_ variables defined, only a DEFAULT_PARAM_CONTEXT.

The DEFAULT_PARAM_CONTEXT configuration variable instructs Cloudera DataFlow Functions which default secret to use when there is no secret that matches the parameter contexts in the flow. This variable is now ignored.

Create an environment variable in Configuration called PARAM_CONTEXT_[***NAME***] where [***NAME***] is the user-defined name of the parameter context. Specify the name of the AWS Secret you want to use as the value of this variable.

NiFi 2.0 [Technical Preview] deployments fail to obtain authentication token in RAZ enabled AWS environments

NiFi 2.0 [Technical Preview] deployments in RAZ enabled AWS environments are not able to obtain an authentication token and therefore fail to read/write to RAZ protected sources/destinations. NiFi 2.0 [Technical Preview] deployments without any Ranger RAZ dependencies work as expected.

There is no workaround for this issue.

IAM Policy Simulator preflight check fails with resource policy validation

With all cross account policies in place, IAM Policy Simulator preflight check still fails with the following error message:

IAM Resource Policy validation failed on AWS. CrossAccount role does not have permissions for these operations : : ssm:GetParameter, ssm:GetParameters, ssm:GetParameterHistory, ssm:GetParametersByPath

This happens because even if a given cross account role is allowed to perform a certain action (granted through IAM Policies), an attached Service Control Policy (SCP) may override that capability if it enforces a Deny on that action. SCP takes precedence over IAM Policies. SCPs are either applied at the root of an organization, or can be applied to individual accounts. A permission can be blocked at any level above the account, either implicitly or explicitly (by including it in a Deny policy statement).

As the IAM Simulator SDK does not have an option to include or exclude an organization’s SCP policy, the preflight check will fail if an SCP policy is denying an action, even though the IAM role has the necessary permissions.

This is a known issue in AWS.

Do not select the Skip Validations option when enabling Cloudera DataFlow to bypass this issue. This bypasses all preflight validation checks. Instead, submit a request to add the LIFTIE_DISABLE_IAM_PREFLIGHT_CHECK entitlement to your account which ensures only the IAM Policy preflight validation check is skipped.

Limitations

Parameter groups that have referencing flow drafts, and flow drafts referencing a parameter group cannot be reassigned to another project

A flow draft referencing any parameter group cannot be reassigned to another project.

Project reassignment does not move assets. When reassigning a parameter group that includes a FILE type parameter (asset) reference to another project, that asset needs to be re-uploaded to the new project.

There is no workaround for this issue.

Duplication of parameter groups referencing assets does not result in asset duplication

After duplicating a parameter group, assets have to be re-uploaded manually. This means that if you reassign a parameter group to another project, duplicate a parameter group, or export a parameter group, the asset will not be moved or duplicated.

Duplication can only happen inside a given project. If the newly created group is to be assigned to another project, that step can happen only after the duplication concludes. (Another project cannot be targeted for duplication.)

There is no workaround for this issue.

Diagnostic Bundle collection through the Management Console is available on the US Control Plane only

There is no workaround for this issue.

Data Lineage information is not automatically reported to Atlas in the Data Catalog

Flow deployments created by Cloudera DataFlow do not come with a pre-configured ReportLineageToAtlas Reporting Task.

If you have been assigned the DFFlowAdmin role, you can manually create and configure the ReportLineageToAtlas Reporting Task in the NiFi canvas after a deployment is completed.

PowerUsers are not able to create flow deployments without additional Cloudera DataFlow roles

While the PowerUser role gives you the ability to view flow deployments in the Dashboard, view flow definitions in the Catalog, and allows the user to initiate flow deployments, the Deployment Wizard fails after selecting an environment for which the user does not have the DFFlowAdmin resource role assigned.

Assign the DFFlowAdmin role to the user of the environment to which they want to deploy flow definitions.

Cloudera DataFlow reports "Bad Health" during Data Lake upgrade

Cloudera DataFlow monitors the state of the associated Cloudera Public Cloud environment to decide which actions Cloudera DataFlow users can take. Cloudera DataFlow detects Data Lake upgrades of the associated Cloudera Public Cloud environment and puts the Cloudera DataFlow service into Bad Health for the duration of the upgrade blocking new deployments.

To work around this issue, wait for the Data Lake upgrade to complete before creating new flow deployments.

Deployments and Cloudera DataFlow Services are no longer visible in the Cloudera DataFlow Dashboard or Environments page when the associated Cloudera Public Cloud Environment has been deleted

If the associated Cloudera Public Cloud Environment is deleted while a Cloudera DataFlow Service is enabled, it will become orphaned. Orphaned resources are no longer visible to users without the PowerUser role.

To work around this issue, open the Environments or Dashboard page with a user who has been assigned the PowerUser role. PowerUsers are able to view orphaned deployments and Cloudera DataFlow services.

Non-transparent proxies are not supported on Azure

There is no workaround for this issue.

NiFi 2.0 [Technical preview] limitations