Known issues and limitations

Known issues

Upgrade rollback from DFX 2.6.1 is disabled

Rollback after a service upgrade to DFX version 2.6.1 is not possible. This is caused by numerous breaking changes in chart and image upgrades that are included in the DFX 2.6.1 release.

IAM Policy Simulator preflight check fails with resource policy validation

With all cross account policies in place, IAM Policy Simulator preflight check still fails with the following error message:

IAM Resource Policy validation failed on AWS. CrossAccount role does not have permissions for these operations : : ssm:GetParameter, ssm:GetParameters, ssm:GetParameterHistory, ssm:GetParametersByPath

This happens because even if a given cross account role is allowed to perform a certain action (granted through IAM Policies), an attached Service Control Policy (SCP) may override that capability if it enforces a Deny on that action. SCP takes precedence over IAM Policies. SCPs are either applied at the root of an organization, or can be applied to individual accounts. A permission can be blocked at any level above the account, either implicitly or explicitly (by including it in a Deny policy statement).

As the IAM Simulator SDK does not have an option to include or exclude an organization’s SCP policy, the preflight check will fail if an SCP policy is denying an action, even though the IAM role has the necessary permissions.

This is a known issue in AWS.

Do not select the Skip Validations option when enabling DataFlow to bypass this issue. This bypasses all preflight validation checks. Instead, submit a request to add the LIFTIE_DISABLE_IAM_PREFLIGHT_CHECK entitlement to your account which ensures only the IAM Policy preflight validation check is skipped.

Limitations

Diagnostic Bundle collection through the Management Console is available on the US Control Plane only

There is no workaround for this issue.

Data Lineage information is not automatically reported to Atlas in the Data Catalog

Flow deployments created by DataFlow do not come with a pre-configured ReportLineageToAtlas Reporting Task.

If you have been assigned the FlowAdmin role, you can manually create and configure the ReportLineageToAtlas Reporting Task in the NiFi canvas after a deployment is completed.

PowerUsers are not able to create flow deployments without additional DataFlow roles

While the PowerUser role gives you the ability to view flow deployments in the Dashboard, view flow definitions in the Catalog, and allows you to initiate flow deployments, the Deployment Wizard fails after selecting an environment for which the user does not have the DFFlowAdmin resource role assigned.

Assign the DFFlowAdmin role to the user of the environment to which they want to deploy flow definitions.

CDF reports "Bad Health" during Data Lake upgrade

CDF monitors the state of the associated CDP environment to decide which actions CDF users can take. CDF detects Data Lake upgrades of the associated CDP environment and puts the CDF service into Bad Health for the duration of the upgrade blocking new deployments.

To work around this issue, wait for the Data Lake upgrade to complete before creating new flow deployments.

Deployments and CDF Services are no longer visible in the DataFlow Dashboard or Environments page when the associated CDP Environment has been deleted

If the associated CDP Environment is deleted while a DataFlow Services is enabled, it will become orphaned. Orphaned resources are no longer visible to users without the PowerUser role.

To work around this issue, open the Environments or Dashboard page with a user who has been assigned the PowerUser role. PowerUsers are able to view orphaned deployments and DataFlow services.

Non-transparent proxies are not supported on Azure

There is no workaround for this issue.

Technical Service Bulletins

TSB 2022-588: Kubeconfig and new version of aws-iam-authenticator

Regenerate Kubeconfig and in conjunction use a newer version of aws-iam-authenticator on AWS. Kubeconfig in Cloudera Data Platform (CDP) Public Cloud Data Services needs to be regenerated because the Kubeconfig generated before June 15, 2022 uses an old APIVersion (client.authentication.k8s.io/v1alpha1) which is no longer supported. This causes compatibility issues with aws-iam-authenticator starting from v0.5.7. To be able to use the new aws-iam-authenticator, the Kubeconfig needs to be regenerated.

Knowledge article

For the latest update on this issue see the corresponding Knowledge article: TSB 2022-588: Kubeconfig and new version of aws-iam-authenticator.