ClouderaOnCloudParameterProvider 2.11.0.4.13.0.0-948

Bundle
com.cloudera | nifi-cdf-cdp-nar
Description
Fetches parameters from Cloudera on Cloud.
Tags
cloud, cloudera, parameters
Input Requirement
Supports Sensitive Dynamic Properties
false
  • Additional Details for ClouderaOnCloudParameterProvider 2.11.0.4.13.0.0-948

    ClouderaOnCloudParameterProvider

    Description

    The ClouderaOnCloudParameterProvider connects to the Cloudera Data Platform (CDP) Control Plane and fetches connection endpoints for the data services running inside a single CDP environment. It exposes those endpoints to NiFi as Parameter Groups, so that dataflows can reference remote service URLs and JDBC connection strings through parameters instead of hard-coding them.

    Each time the parameters are fetched, the provider queries the following data services for the configured environment and turns the endpoints it discovers into provided parameters:

    • Data Hub – the service endpoints exposed by each Data Hub cluster.
    • Data Warehouse – the Hive and Impala JDBC connection strings of each Virtual Warehouse.
    • Operational Database – the JDBC connection strings of each database.

    All fetched parameters are marked as provided.

    After clicking the Fetch Parameters button, it might take several seconds for the pop-up window to appear with the fetched parameters.

    Data Fetched and Naming

    The provider organizes the fetched data into one Parameter Group per cluster / warehouse / database. Within each group, individual endpoints become parameters. The naming conventions differ per data service and are described below. In the naming patterns, <Environment Name> is the value of the Environment Name property.

    Data Hub

    For every Data Hub cluster in the environment, all of the cluster’s service endpoints are fetched.

    • Parameter Group name: DataHub <Environment Name> <Cluster Name>
    • Parameter name: derived from the endpoint’s service name. To keep names unique within a group, the endpoint’s mode is appended when present, followed by a numeric occurrence suffix. The resulting pattern is <Service Name>[-<Mode>]-<N>. For example, for the CM-API endpoint whose mode is PAM, the resulting parameter name is CM-API-PAM-1. For the HIVESERVER2 endpoint that does not have a mode value, the resulting parameter name is HIVESERVER2-1.
    • Parameter value: the endpoint’s service URL.
    • Parameter description: the endpoint’s display name.

    Clusters that expose no endpoints are skipped.

    Data Warehouse

    For every Virtual Warehouse belonging to each Data Warehouse cluster, the JDBC connection strings are fetched.

    • Parameter Group name: Data Warehouse <Environment Name> <Cluster Name> <Virtual Warehouse Name>
    • Parameter names: a fixed set of names is used, and a parameter is created only for the connection strings that the Virtual Warehouse actually exposes:
      Parameter name Value
      HIVE-JDBC Hive JDBC connection string
      HIVE-KERBEROS-JDBC Hive Kerberos JDBC connection string
      IMPALA-JDBC Impala JDBC connection string
      IMPALA-KERBEROS-JDBC Impala Kerberos JDBC connection string

    A Parameter Group is still created for a Virtual Warehouse that has no endpoints provisioned yet; in that case the group simply contains no parameters.

    Operational Database

    For every Operational Database in the environment, the JDBC connectors are fetched. Only connectors of kind JDBC that have a JDBC URL configured are included.

    • Parameter Group name: OpDB <Environment Name> <Database Name>
    • Parameter name: the connector’s name (for example the connector name reported by the database).
    • Parameter value: the connector’s JDBC URL.
    • Parameter description: JDBC connection string for <Connector Name>.

    Databases whose connectors are not yet available (for example while the database is not in a running state) are skipped without failing the fetch.

Properties