Applying Metadata to HDFS and Hive Entities using the API

• File - The path of the metadata file must be .filename.navigator. For example, to apply properties to the file /user/test/file1.txt, the metadata file path is /user/test/.file1.txt.navigator.
• Directory - The path of the metadata file must be dirpath/.navigator. For example, to apply properties to the directory /user, the metadata path must be /user/.navigator.

• If an entity has been extracted when the API is called, the metadata is applied immediately.
• If the entity has not been extracted, you can preregister metadata, which is then applied once the entity is extracted.

• Identity
• Source ID
• Metadata fields (name, description, tags, properties)
• Fields relevant to the identifier

http://fqdn-n.example.com:port/api/APIversion/entities/ 

• PUT

  curl http://fqdn-n.example.com:port/api/APIversion/entities/identity \
  -u username:password \
  -X PUT \
  -H "Content-Type: application/json" \
  -d '{properties}'

  where identity is an entity ID and properties are:
  • name - Name metadata.
  • description - Description metadata.
  • tags - Tag metadata.
  • properties - Custom metadata properties. The format is {key: value}.
  • customProperties - Managed metadata properties. The format is {namespace: {key: value}}. If a property is assigned a value that does not conform to type constraints, an error is returned.
  All existing naming rules apply, and if any value is invalid, the entire request is denied.
• POST

  curl http://fqdn-n.example.com:port/api/APIversion/entities/ \
  -u username:password \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{properties}'

  where properties are:
  • sourceId (required) - An existing source ID. After the first extraction, you can retrieve source IDs using the call:

    curl http://fqdn-n.example.com:port/api/APIversion/entities/?query=type:SOURCE \
    -u username:password -X GET

    For example:

    [ ...
    {  {
    "identity": "61cfefd303d4284b7f5014b701f2c76d",
    "originalName": "source.listing",
    "originalDescription": null,
    "sourceId": "012437f9eeb3c23dc69e679ac94a7fa2",
    "firstClassParentId": null,
    "parentPath": "/user/hdfs/.cm/distcp/2016-02-03_487",
    ...
    "properties": {
    "__cloudera_internal__hueLink":
    "http://fqdn-2.example.com:8888/filebrowser/#/user/hdfs/.cm/distcp/2016-02-03_487/source.listing"
     },
    "technicalProperties": null,
    "fileSystemPath": "/user/hdfs/.cm/distcp/2016-02-03_487/source.listing",
    "type": "FILE",
    "size": 92682,
    "created": "2016-02-03T21:12:16.587Z",
    "lastModified": "2016-02-03T21:12:16.587Z",
    "lastAccessed": "2016-02-03T21:12:16.587Z",
    "permissions": "rw-r--r--",
    "owner": "hdfs",
    "group": "supergroup",
    "blockSize": 134217728,
    "mimeType": "application/octet-stream",
    "replication": 3,
    "userEntity": false,
    "deleted": false,
    "sourceType": "HDFS",
    "metaClassName": "fselement",
    "packageName": "nav",
    "internalType": "fselement"
    }, ...

    If you have multiple services of a given type, you must specify the source ID that contains the entity you expect it to match.
  • parentPath - The path of the parent entity, defined as:
    • HDFS file or directory - fileSystemPath of the parent directory. (Do not provide this field if the entity affected is the root directory.) Example parentPath for /user/admin/input_dir: /user/admin. If you add metadata to a directory, the metadata does not propagate to any files or folders in that directory.
    • Hive database - If you are updating database metadata, do not specify this field.
    • Hive table or view - The name of database containing the table or view. Example for a table in the default database: default.
    • Hive column - database name/table name/view name. Example for a column in the sample_07 table: default/sample_07.
  • originalName (required) - The name as defined by the source system.
    • HDFS file or directory- Name of file or directory (ROOT if the entity is the root directory). Example originalName for /user/admin/input_dir: input_dir.
    • Hive database, table, view, or column - The name of the database, table, view, or column.
      • Example for default database: default
      • Example for sample_07 table: sample_07
  • identity
  • Metadata fields (name, description, tags, properties)
  • Fields relevant to the identifier