Using External Databases

After external database servers are defined, the databases on them can be defined. Cloudera Director can use databases that already exist on those servers, or it can create them while bootstrapping new Cloudera Manager installations or CDH clusters.

The following parts of an existing database must be defined:
  • type - The type of database, “MYSQL” or “POSTGRESQL.”
  • hostname - The name of the server host.
  • port - The listening port of the server.
  • name - The name of the database on the server.
  • username - The name of the user account having full access to the database.
  • password - The password for the user account.
The parts of an external database template are:
  • name - A unique name for the template within the deployment or cluster template.
  • databaseServerName - The name of the external database server where the new database is to reside.
  • databaseNamePrefix - The string prefix for the name of the new database server.
  • usernamePrefix - The string prefix for the name of the new user account that will have full access to the database.
The database server name in a database server template must refer to an external database server that is already defined.
When Cloudera Director creates the new database, it names the database by starting with the prefix in the template and then appends a random string. This prevents name duplication issues when sharing a database server across many deployments and clusters. Likewise, Cloudera Director creates new user accounts by starting with the prefix in the template and appending a random string.

If Cloudera Director creates new external databases during the bootstrap of a deployment or cluster, then it also drops them, and their associated user accounts, when terminating the deployment or cluster. Be sure to back up those databases before beginning termination.

API

Define external databases in the templates for new Cloudera Manager installations (“deployments”) or new clusters. You cannot define both existing databases, and new databases that need to be created, in the same template.

Defining External Databases in the Configuration File

For Cloudera Manager

Define external databases used by Cloudera Manager in the cloudera-manager section of a configuration file. The following example defines existing external databases.

cloudera-manager {
   # ...
   databases {
      CLOUDERA_MANAGER {
         name: scm1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: scmuser
         password: scmpassword
      }
      ACTIVITYMONITOR {
         name: am1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: amuser
         password: ampassword
      }
      REPORTSMANAGER {
         name: rm1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: rmuser
         password: rmpassword
      }
      NAVIGATOR {
         name: nav1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: navuser
         password: navpassword
      }
      NAVIGATORMETASERVER {
         name: navmeta1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: navmetauser
         password: navmetapassword
      }
   }

The following example defines new external databases that Cloudera Director must create while bootstrapping the deployment.

cloudera-manager {
   # ... 
   databaseTemplates { 
      CLOUDERA_MANAGER { 
         name: cmtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: scm 
         usernamePrefix: cmadmin 
      } 
      ACTIVITYMONITOR {
         name: cmamtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: am 
         usernamePrefix: cmamadmin 
      } 
      REPORTSMANAGER {
         name: cmrmtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: rm 
         usernamePrefix: cmrmadmin 
      } 
      NAVIGATOR { 
         name: cmnavtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: nav 
         user: cmnavadmin 
      } 
      NAVIGATORMETASERVER {
         name: cmnavmetatemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: navmeta 
         usernamePrefix: cmnavmetaadmin 
      } 
   }

Each template must refer to a database server defined elsewhere in the configuration file. The database server template can be for a server that does not yet exist; in that case, Cloudera Director starts creating the server, and then waits while bootstrapping the deployment until the server is available.

A deployment must use either all existing databases or all non-existing databases for the different Cloudera Manager components; they cannot be mixed.

For Cluster Services

Define external databases used by cluster services such as Hive in the cluster section of a configuration file. The following example defines existing external databases.

cluster {
   #...
   databaseTemplates: {
      HIVE {
         name: hive1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: hiveuser
         password: hivepassword
      }
   }

The following example defines new external databases that Cloudera Director must create while bootstrapping the cluster.

cluster {
   #... 
   databaseTemplates: {
      HIVE {
      name: hivetemplate 
      databaseServerName: mysql1 
      databaseNamePrefix: hivemetastore 
      usernamePrefix: hive 
   } 
}

Each template must refer to a database server defined elsewhere in the configuration file. The database server template can be for a server that does not yet exist; in that case, Cloudera Director starts creating the server, and then waits while bootstrapping the cluster until the server is available.

A deployment must use either all existing databases or all non-existing databases for the different cluster services; they cannot be mixed.