Cloudera Navigator Metadata Server

Adding the Navigator Metadata Server Role

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Before adding the Navigator Metadata Server role, configure the database where policies, roles, and audit report metadata is stored.

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Instances tab.
3. Click the Add Role Instances button. The Customize Role Assignments page displays.
4. Assign the Navigator role to a host.
   1. Customize the assignment of role instances to hosts. The wizard evaluates the hardware configurations of the hosts to determine the best hosts for each role. The wizard assigns all worker roles to the same set of hosts to which the HDFS DataNode role is assigned. These assignments are typically acceptable, but you can reassign them if necessary.

      Click a field below a role to display a dialog containing a list of hosts. If you click a field containing multiple hosts, you can also select All Hosts to assign the role to all hosts or Custom to display the pageable hosts dialog.

      The following shortcuts for specifying hostname patterns are supported:

      • Range of hostnames (without the domain portion)

        Range Definition	Matching Hosts
        10.1.1.[1-4]	10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
        host[1-3].company.com	host1.company.com, host2.company.com, host3.company.com
        host[07-10].company.com	host07.company.com, host08.company.com, host09.company.com, host10.company.com

      • IP addresses
      • Rack name

      Click the View By Host button for an overview of the role assignment by hostname ranges.

5. When you are satisfied with the assignments, click Continue. The Database Setup screen displays.
6. Configure database settings:
   1. Choose the database type:
      • Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure required databases. Make a note of the auto-generated passwords.
        
        
        
        
      • Select Use Custom Databases to specify external databases.
        1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database.
   2. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.) The Review Changes screen displays.
7. Click Finish.

Starting, Stopping, and Restarting the Navigator Metadata Server

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Instances tab.
3. Do one of the following depending on your role:

   • Minimum Required Role: Full Administrator

     1. Check the checkbox next to the Navigator Metadata Server role.
     2. Select Actions for Selected > Action. Click Action to confirm the action, where Action is Start, Stop, or Restart.

   • Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

     1. Click the Navigator Metadata Server role link.
     2. Select Actions > Action this Navigator Metadata Server. Click Action this Navigator Metadata Server, where Action is Start, Stop, or Restart, to confirm the action.

Configuring the Navigator Metadata Server Storage Directory

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Describes how to configure where the Navigator Metadata Server stores extracted data. The default is /var/lib/cloudera-scm-navigator.

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Click the Navigator Metadata Server Default Group.
4. Specify the directory in the Navigator Metadata Server Storage Dir property.
5. Click Save Changes.
6. Click the Instances tab.
7. Check the checkbox next to the Navigator Metadata Server role.
8. Select Actions for Selected > Restart.

Configuring the Navigator Metadata Server Port

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Describes how to configure the port on which the Navigator UI is accessed. The default is 7187.

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Select Navigator Metadata Server Default Group > Ports and Addresses.
4. Specify the port in the Navigator Metadata Server Port property.
5. Click Save Changes.
6. Click the Instances tab.
7. Check the checkbox next to the Navigator Metadata Server role.
8. Select Actions for Selected > Restart.

Navigator Metadata Server Sizing and Performance Recommendations

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The Navigator Metadata Server uses Solr to store, index, and query metadata. Indexing happens during extraction. Querying is fast and efficient because the data is indexed.

Memory and CPU requirements are based on amount of data that is stored and indexed. With 6 GB of RAM and 8-10 cores Solr can process 6 million entities in 25-30 minutes or 80 million entities in 8 to 9 hours. Any less RAM than 6GB and will result in excessive garbage collection and possibly out-of-memory exceptions. For large clusters, Cloudera advises at least 8 GB of RAM and 8 cores. The Solr instance runs in process with Navigator, so the Java heap for the Navigator Metadata Server should be set according to the size of cluster.

By default, during the Cloudera Manager first run installation wizard the Navigator Audit Server and Navigator Metadata Server are assigned to the same host as the Cloudera Management Service monitoring roles. This configuration works for a small cluster, but should be updated before the cluster grows. You can either change the configuration at installation time or move the Navigator Metadata Server if necessary.

Moving a Navigator Metadata Server Role

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Stop the Navigator Metadata Server role, delete it from existing host, and add it to a new host.
2. If the Solr data path is not on NFS/SAN, move the data to the same path on the new host.
3. Start the Navigator Metadata Server role.

Enabling Hive Metadata Extraction in a Secure Cluster

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The Navigator Metadata Server uses the hue user to connect to the Hive Metastore. The hue user is able to connect to the Hive Metastore by default. However, if the Hive service Hive Metastore Access Control and Proxy User Groups Override property and/or the HDFS service Hive Proxy User Groups property have been changed from their default values to settings that prevent the hue user from connecting to the Hive Metastore, Navigator Metadata Server will be unable to extract metadata from Hive. If this is the case, modify the Hive service Hive Metastore Access Control and Proxy User Groups Override property and/or the HDFS service Hive Proxy User Groups property so that the hue user can connect as follows:

1. Go to the Hive or HDFS service.
2. Click the Configuration tab.
3. Expand the Service-Wide > Proxy category.
4. In the Hive service Hive Metastore Access Control and Proxy User Groups Override field or the HDFS service Hive Proxy User Groups field, click the Value column, and click to add a new row.
5. Type hue.
6. Click Save Changes to commit the changes.
7. Restart the service.

Configuring the Metadata Server to Mask Personally Identifiable Information

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Expand the Navigator Metadata Server Default Group category.
4. Click the Advanced category.
5. Configure the PII Masking Regular Expression property with a regular expression that matches the credit card number formats to be masked. The default expression is:

   (4[0-9]{12}(?:[0-9]{3})?)|(5[1-5][0-9]{14})|(3[47][0-9]{13})
   |(3(?:0[0-5]|[68][0-9])[0-9]{11})|(6(?:011|5[0-9]{2})[0-9]{12})|((?:2131|1800|35\\d{3})\\d{11})

   which is constructed from the following subexpressions:
   • Visa - (4[0-9]{12}(?:[0-9]{3})?)
   • MasterCard - (5[1-5][0-9]{14})
   • American Express - (3[47][0-9]{13})
   • Diners Club - (3(?:0[0-5]|[68][0-9])[0-9]{11})
   • Discover - (6(?:011|5[0-9]{2})[0-9]{12})
   • JCB - ((?:2131|1800|35\\d{3})\\d{11})
   If the property is left blank, PII information is not masked.
6. Click Save Changes to commit the changes.

Enabling Policies

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Expand the Navigator Metadata Server Default Group category.
4. Click the Advanced category.
5. In the Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for cloudera-navigator.properties property, set the following properties:

   Property	Description
   navms.policy.enable	Indicate whether to enable policies. Default: false.

6. Click Save Changes to commit the changes.
7. Restart the Navigator Metadata Server.

Configuring a JMS Server for Policy Messages

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Expand the Navigator Metadata Server Default Group category.
4. In the Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for cloudera-navigator.properties property, set the following properties:

   Property	Description
   navms.jms.url	The URL of the JMS server to which notifications of changes to entities affected by policies are sent.
   navms.jms.user	The JMS user to which notifications of changes to entities affected by policies are sent.
   navms.jms.password	The password of the JMS user to which notifications of changes to entities affected by policies are sent.
   navms.jms.queue	The JMS queue to which notifications of changes to entities affected by policies are sent.

5. Click Save Changes to commit the changes.
6. Restart the Navigator Metadata Server.