Defining Managed Metadata
Minimum Required Role: Metadata Administrator (also provided by Full Administrator)
You can use managed metadata to add typed metadata to classes of entities. You can add namespaces and properties.
- nav for Navigator metadata classes (for example, fselement and user-defined custom fields)
- up (custom metadata)
- tp (technical properties)
- xt (partner applications)
The combination of namespace and property name must be unique.
- Boolean
- date
- integer
- long
- float
- double
- text (with optional maximum length and regular expression validation criteria)
- enum (of string)
Once you have created properties and assigned property values to specific entities, you can create search filters for property values.
Creating Custom Properties Using the Metadata UI
To create custom properties in Cloudera Navigator:
- Start and log into the Cloudera
Navigator data management component UI with the credentials of a user having one of the following user roles:
- Cloudera Manager Full Administrator
- Cloudera Manager Navigator Administrator
- Cloudera Navigator Full Administrator
- Cloudera Navigator Metadata Administrator
- Click the Administration link in the upper right. The Managed Metadata tab displays the list of namespaces and the properties defined in the namespaces.
- Click the New Property... button.
- In the Classes field, click or type the beginning of a Navigator entity classname.
- Select the class of entities to which the property applies. To clear the field, hover over the field and click the that displays at the right of the field.
- Click the Namespace field and select a namespace. If the Namespace drop-down list is empty, click Create Namespace....
- Specify a namespace name and optional description.
- Click Continue.
- Add the name for the property.
The name can contain letters, numbers, underscores, and hyphens and can be up to 50 characters long.
- Specify an optional description.
- Select the Multivalued Enable checkbox if the property can have more than one value. For example, an emailFrom property should accept only one value, but an emailTo property could accept more than one value.
- In the Type drop-down list, select the property type and specify constraints on the value.
- Boolean - Boolean: true or false.
- Date - Date and time.
- Enumeration - A set of values. In the Enumeration field, type valid enumeration values and press Enter or Tab.
- Number - A number. In the Number Type field, select the type of the number: Integer, Long, Float, Double.
- Text - A string.
- Maximum Length - The maximum length of the string.
- Regular Expression - A regular expression that determines whether a string is a valid value. You can test the
expression by clicking Show regex tester, entering input that you expect to match the expression, and clicking Execute test. In the
following example, the expression tester indicates that test@example.com matches the defined expression.
- Click Continue to Review. The Review screen displays.
- Click Create to create the property, Cancel to return to the Properties page, or Back to Edit Property to continue editing the property.
Example Properties
The following figure shows two properties in the namespace MailAnnotation that apply to entities of the HDFS Entity class (HDFS files and directories). The emailFrom property is of type TEXT and can be assigned a single value. The MailTo property is also of type TEXT but can have multiple values.Managing Custom Properties Using the Metadata UI
You can view managed metadata property summary details by clicking property name in the Properties table, or by clicking the Actions box in the property row and then clicking View in the dropdown.
You can also edit some aspects of a property, delete and restore a property, and purge a deleted property.
Editing a Property
- Add classes to which the property applies
- Add and remove enumeration values
- Change the description
- Change the maximum length
- Change the regex pattern
- Start and log into the Cloudera Navigator data management component
UI with the credentials of a user having one of the following roles:
- Cloudera Manager Full Administrator
- Cloudera Manager Navigator Administrator
- Cloudera Navigator Full Administrator
- Cloudera Navigator Metadata Administrator
- Click the Administration link in the upper right. The Managed Metadata tab displays the list of namespaces and the properties defined in the namespaces.
- Open the property Edit page by clicking the Actions box in the property row and then clicking Edit in the dropdown.
- In the Additional Class field, click or type the beginning of a Navigator entity classname.
- Select the class of entities to which the property applies. To clear the field, hover over the field and click the that displays at the right of the field.
- In the Description field, add a description or edit an existing description.
- If the property is of the Enumeration type, you can add or remove values in the Enumeration field.
- For Text properties:
- In the Maximum Length field, add or change the value for the maximum length.
- In the Regular Expression field, edit the expression. Click Show regex tester to test input against any changes you make.
- Click Continue to Review. The Review screen displays.
- Click Update to commit the change or Back to Edit Property to continue editing the property, or Cancel to return to the Properties page.
Deleting, Restoring, and Purging Managed Metadata Properties
After a property is deleted, it cannot be used in a filter or in value assignments for entities. The deleted property is visible only from the Admin interface; it is labeled as Deleted
in the Status column of the Properties table. In the following example, the EmailFrom property has been deleted.
For non-Admin users, a deleted property is hidden. However, the property still exists, and the values assigned to entities using this property are not affected until the deleted property is purged. Purging permanently removes the properties and any values from all entities. Any policies using these purged properties for metadata assignments are not modified by the purge and will fail the next time they run.
Because properties are not completely removed until purge, you cannot create a new property with the same name until the deleted property has been purged.
Deleting a Property
- In the Properties table, for the property that you are deleting, click the Actions button, and then click Delete in the drop-down menu.
- In the Delete Property dialog box, review the property deletion information. If any entities are affected, you see a View affected entities link; click to see all entities that use the property. Click Confirm Delete to delete the property, or click Cancel.
Restoring a Property
If you have not yet purged a deleted property, you can restore it.
- In the Properties table, for the property that you are restoring, click the Actions button, and then click Restore in the drop-down menu.
Purging a Property
- In the Properties table, click Purge Deleted Properties. The Purge all Deleted Properties dialog box opens, describing the effects of the purge and reporting the number of entities that use the property.
- In the Purge all Deleted Properties dialog box, click Confirm Purge to permanently remove all deleted properties, or click Cancel to return to the Properties page.
Navigator Built-in Classes
Class | Description |
---|---|
HDFS Dataset | Logical dataset backed by a path in HDFS. |
HDFS Dataset Field | Field in an HDFS dataset. |
HDFS Entity | HDFS file or directory. |
Hive Column | Column in a Hive table. |
Hive Database | Hive database. |
Hive Partition | Partition of a Hive table. |
Hive Query | Hive query template. |
Hive Query Execution | Instance of a Hive query. |
Hive Query Part | Component of a Hive query that maps specific input columns to output columns. |
Hive Table | A Hive table. |
Hive View | View on one or more Hive tables. |
Impala Query | Impala query template. |
Impala Query Execution | Instance of an Impala query. |
Impala Query Part | Component of an Impala query that maps specific input columns to output columns. |
Job Instance | Instance of a MapReduce, YARN, or Spark job. |
Job Template | Template for a MapReduce, YARN, or Spark job. |
Oozie Workflow | Template for an Oozie workflow. |
Oozie Workflow Instance | Instance of an Oozie workflow. |
Pig Field | Field for a relation in Pig; similar to a column in a Hive table. |
Pig Operation | Template for a Pig transformation. |
Pig Operation Execution | Instance of a Pig transformation. |
Pig Relation | Pig relation; similar to a Hive table. |
S3 Bucket | A bucket in S3. |
S3 Object | A file or directory in an S3 bucket. |
Sqoop Export Sub-operation | Sqoop export component that connects specific columns. |
Sqoop Import Query | Sqoop import job with query options. |
Sqoop Import Sub-operation | Sqoop import component that connects specific columns. |
Sqoop Operation Execution | Instance of a Sqoop job. |
Sqoop Table Export | Sqoop table export operation template. |
Sqoop Table Import | Sqoop table import operation template. |
User Sub-operation | User-specified sub-operation of a MapReduce or YARN job; used for specifying custom column-level lineage. |
Defining Metadata with the Navigator API and Navigator SDK
In addition to defining metadata using features provided by the Navigator Metadata UI, you can also define metadata using the Navigator API and Navigator SDK.
For information on the Navigator API, see Cloudera Navigator Data Management API.
For information on the SDK, see the Navigator SDK documentation.