Create Atlas entity type definitions
As an administrator, you must create the Atlas entity type definitions before submitting a Flink job, and enable Atlas in Cloudera Manager to use Atlas for metadata management.
-
Copy and paste the following command:
curl -k -u <atlas_admin>:<atlas_admin_pwd> --location --request
- Provide your workload username and password as atlas admin.
- Navigate to Management Console > Environments, and select the environment where you have created your cluster.
- Click Data Lake.
- Click Endpoints.
- Click on the copy icon beside the Atlas endpoint.
-
Replace the atlas server URL with the copied Atlas endpoint.
POST '<atlas_endpoint_url>/v2/types/typedefs'
-
Copy and paste the entity type definitions.
--header 'Content-Type: application/json' \ --data-raw '{ "enumDefs": [], "structDefs": [], "classificationDefs": [], "entityDefs": [ { "name": "flink_application", "superTypes": [ "Process" ], "serviceType": "flink", "typeVersion": "1.0", "attributeDefs": [ { "name": "id", "typeName": "string", "cardinality": "SINGLE", "isIndexable": true, "isOptional": false, "isUnique": true }, { "name": "startTime", "typeName": "date", "cardinality": "SINGLE", "isIndexable": false, "isOptional": true, "isUnique": false }, { "name": "endTime", "typeName": "date", "cardinality": "SINGLE", "isIndexable": false, "isOptional": true, "isUnique": false }, { "name": "conf", "typeName": "map<string,string>", "cardinality": "SINGLE", "isIndexable": false, "isOptional": true, "isUnique": false }, { "name": "inputs", "typeName": "array<string>", "cardinality": "LIST", "isIndexable": false, "isOptional": false, "isUnique": false }, { "name": "outputs", "typeName": "array<string>", "cardinality": "LIST", "isIndexable": false, "isOptional": false, "isUnique": false } ] } ], "relationshipDefs": [] }'
The following short video also details the procedure how to create the Atlas entity type definitons.
- Navigate to Management Console > Environments, and select the environment where you have created your cluster.
- Click on the Streaming Analytics cluster.
- Select Cloudera Manager UI from the Services.
- Select Flink from the list of clusters.
- Click Configuration.
- Search for enable atlas in the search bar.
-
Make sure that Atlas is enabled in the configurations.
The following short video also details the procedure how to enable Atlas for Flink in Cloudera Manager.