Cloudera Navigator Metadata Server

Adding the Navigator Metadata Server

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Configure the database where policies, roles, and audit report metadata is stored.
2. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
3. Click the Instances tab.
4. Click the Add Role Instances button. The Customize Role Assignments page displays.
5. Assign the Navigator role to a host.
   1. Customize the assignment of role instances to hosts. The wizard evaluates the hardware configurations of the hosts to determine the best hosts for each role. The wizard assigns all worker roles to the same set of hosts to which the HDFS DataNode role is assigned. You can reassign role instances if necessary.

      Click a field below a role to display a dialog box containing a list of hosts. If you click a field containing multiple hosts, you can also select All Hosts to assign the role to all hosts, or Custom to display the pageable hosts dialog box.

      The following shortcuts for specifying hostname patterns are supported:

      • Range of hostnames (without the domain portion)

        Range Definition	Matching Hosts
        10.1.1.[1-4]	10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
        host[1-3].company.com	host1.company.com, host2.company.com, host3.company.com
        host[07-10].company.com	host07.company.com, host08.company.com, host09.company.com, host10.company.com

      • IP addresses
      • Rack name

      Click the View By Host button for an overview of the role assignment by hostname ranges.

6. When you are satisfied with the assignments, click Continue.
7. Configure database settings:
   1. Choose the database type:
      • Keep the default setting of Use Embedded Database to have Cloudera Manager create and configure required databases. Record the auto-generated passwords.
        
        
        
        
      • Select Use Custom Databases to specify external database host, enter the database type, database name, username, and password for the database that you created when you set up the database.
      • If you are adding the Oozie service, you can change your Oozie configuration to control when data is purged in order to improve performance, cut down on database disk usage, improve upgrade performance, or to keep the history for a longer period of time. See Configuring Oozie Data Purge Settings Using Cloudera Manager.
   2. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise, check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.)

      The Cluster Setup Review Changes screen displays.

8. Click Finish.

Starting, Stopping, and Restarting the Navigator Metadata Server

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Instances tab.
3. Do one of the following depending on your role:

   • Minimum Required Role: Full Administrator

     1. Select the checkbox next to the Navigator Metadata Server role.
     2. Select Actions for Selected > Action. Click Action to confirm the action, where Action is Start, Stop, or Restart.

   • Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

     1. Click the Navigator Metadata Server role link.
     2. Select Actions > Action this Navigator Metadata Server. Click Action this Navigator Metadata Server, where Action is Start, Stop, or Restart, to confirm the action.

Navigator Metadata Server Memory Sizing Recommendations

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The Navigator Metadata Server uses Solr to store, index, and query metadata. Indexing happens during extraction. Querying is fast and efficient because the data is indexed. The Navigator Metadata Server memory requirements are based on amount of data that is stored and indexed. The Solr instance runs in process with Navigator, so you should set Java heap for the Navigator Metadata Server accordingly.

When the Navigator Metadata Server starts up it logs the number of documents contained in Solr. For example:

2015-11-11 09:24:58,013 INFO com.cloudera.nav.server.NavServerUtil: Found 68813088 documents in solr core nav_elements
2015-11-11 09:24:58,705 INFO com.cloudera.nav.server.NavServerUtil: Found 78813930 documents in solr core nav_relations

To compute the memory required by the Metadata Server during normal operation, use the number of documents in nav_elements * 200. So for the above example, the recommended amount of memory would be (68813088 * 200) or about 14 GB.

For upgrade, use the number of documents in nav_elements + nav_relations. If you use the number in the above example, for upgrade you would need ((68813088 + 78813930) * 200) or about 30 GB.

By default, during the Cloudera Manager Installation wizard the Navigator Audit Server and Navigator Metadata Server are assigned to the same host as the Cloudera Management Service monitoring roles. This configuration works for a small cluster, but should be updated before the cluster grows. You can either change the configuration at installation time or move the Navigator Metadata Server if necessary.

Setting the Navigator Metadata Server Java Heap Size

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Select Scope > Navigator Metadata Server.
4. Specify the memory value in the Java Heap Size of Navigator Metadata Server in Bytes property.
5. Click Save Changes to commit the changes.
6. Click the Instances tab.
7. Restart the role.

Moving the Navigator Metadata Server to a New Host

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

1. Stop the Navigator Metadata Server, delete it from existing host, and add it to a new host.
2. If the storage directory is not on NFS/SAN, move the data to the same path on the new host.
3. Start the Navigator Metadata Server.

Configuring the Navigator Metadata Server Storage Directory

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The section describes how to configure where the Navigator Metadata Server stores extracted and indexed data. The default is /var/lib/cloudera-scm-navigator.

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Select Scope > Navigator Metadata Server.
4. Specify the directory in the Navigator Metadata Server Storage Dir property.

   If more than one role group applies to this configuration, edit the value for the appropriate role group. See Modifying Configuration Properties Using Cloudera Manager.

5. Click Save Changes to commit the changes.
6. Click the Instances tab.
7. Restart the role.

Configuring the Navigator Metadata Server Port

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Describes how to configure the port on which the Navigator UI is accessed. The default is 7187.

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. In the Search box, type ports.
4. Specify the port in the Navigator Metadata Server Port property.

   If more than one role group applies to this configuration, edit the value for the appropriate role group. See Modifying Configuration Properties Using Cloudera Manager.

5. Click Save Changes to commit the changes.
6. Click the Instances tab.
7. Restart the role.

Configuring the Navigator Metadata Server Log Directory

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

You can configure the location of the Navigator Metadata Server logs (by default, /var/log/cloudera-scm-navigator) as follows:

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Select Scope > Navigator Metadata Server.
4. Select Category > Logs.
5. Set the Navigator Metadata Server Log Directory property.

   If more than one role group applies to this configuration, edit the value for the appropriate role group. See Modifying Configuration Properties Using Cloudera Manager.

6. Click Save Changes to commit the changes.
7. Restart the role.

Configuring the Navigator Metadata Server Session Timeout

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

The default session timeout is 30 minutes. To change the timeout period:

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Select Scope > Navigator Metadata Server.
4. In Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for cloudera-navigator.properties, set the property

   nav.max_inactive_interval=period (s)

   If more than one role group applies to this configuration, edit the value for the appropriate role group. See Modifying Configuration Properties Using Cloudera Manager.

5. Click Save Changes to commit the changes.
6. Restart the role.

Managing Anonymous Usage Data Collection

Minimum Required Role: Navigator Administrator (also provided by Full Administrator)

Cloudera Navigator sends anonymous usage information using Google Analytics to Cloudera. The information helps Cloudera improve Cloudera Navigator. By default anonymous usage data collection is enabled. To enable and disable data collection:

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > Cloudera Management Service.
   • On the Status tab of the Home > Status tab, in Cloudera Management Service table, click the Cloudera Management Service link.
2. Click the Configuration tab.
3. Select Scope > Navigator Metadata Server.
4. Select or deselect the Allow Usage Data Collection checkbox.

   If more than one role group applies to this configuration, edit the value for the appropriate role group. See Modifying Configuration Properties Using Cloudera Manager.

5. Click Save Changes to commit the changes.
6. Restart the role.