Using External Databases

After external database servers are defined, the databases on them can be defined. Cloudera Director can use databases that already exist on those servers, or it can create them while bootstrapping new Cloudera Manager installations or CDH clusters.

When Cloudera Director creates the new database, it names the database by starting with the prefix in the template and then appends a random string. This prevents name duplication issues when sharing a database server across many deployments and clusters. Likewise, Cloudera Director creates new user accounts by starting with the prefix in the template and appending a random string.

If Cloudera Director creates new external databases during the bootstrap of a deployment or cluster, then it also drops them, and their associated user accounts, when terminating the deployment or cluster. Be sure to back up those databases before beginning termination.

API

Define external databases in the templates for new Cloudera Manager installations (“deployments”) or new clusters. You cannot define both existing databases, and new databases that need to be created, in the same template.

Defining External Databases in the Configuration File

For Cloudera Manager

Define external databases used by Cloudera Manager in the cloudera-manager section of a configuration file. The following example defines existing external databases.

cloudera-manager {
   # ...
   databases {
      CLOUDERA_MANAGER {
         name: scm1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: scmuser
         password: scmpassword
      }
      ACTIVITYMONITOR {
         name: am1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: amuser
         password: ampassword
      }
      REPORTSMANAGER {
         name: rm1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: rmuser
         password: rmpassword
      }
      NAVIGATOR {
         name: nav1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: navuser
         password: navpassword
      }
      NAVIGATORMETASERVER {
         name: navmeta1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: navmetauser
         password: navmetapassword
      }
   }

The following example defines new external databases that Cloudera Director must create while bootstrapping the deployment.

cloudera-manager {
   # ... 
   databaseTemplates { 
      CLOUDERA_MANAGER { 
         name: cmtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: scm 
         usernamePrefix: cmadmin 
      } 
      ACTIVITYMONITOR {
         name: cmamtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: am 
         usernamePrefix: cmamadmin 
      } 
      REPORTSMANAGER {
         name: cmrmtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: rm 
         usernamePrefix: cmrmadmin 
      } 
      NAVIGATOR { 
         name: cmnavtemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: nav 
         user: cmnavadmin 
      } 
      NAVIGATORMETASERVER {
         name: cmnavmetatemplate 
         databaseServerName: mysql1 
         databaseNamePrefix: navmeta 
         usernamePrefix: cmnavmetaadmin 
      } 
   } 

Each template must refer to a database server defined elsewhere in the configuration file. The database server template can be for a server that does not yet exist; in that case, Cloudera Director starts creating the server, and then waits while bootstrapping the deployment until the server is available.

A deployment must use either all existing databases or all non-existing databases for the different Cloudera Manager components; they cannot be mixed.

For Cluster Services

Define external databases used by cluster services such as Hive in the cluster section of a configuration file. The following example defines existing external databases.

cluster {
   #...
   databaseTemplates: {
      HIVE {
         name: hive1
         type: mysql
         host: 1.2.3.4
         port: 3306
         user: hiveuser
         password: hivepassword
      }
   }

The following example defines new external databases that Cloudera Director must create while bootstrapping the cluster.

cluster {
   #... 
   databaseTemplates: {
      HIVE {
      name: hivetemplate 
      databaseServerName: mysql1 
      databaseNamePrefix: hivemetastore 
      usernamePrefix: hive 
   } 
}

Each template must refer to a database server defined elsewhere in the configuration file. The database server template can be for a server that does not yet exist; in that case, Cloudera Director starts creating the server, and then waits while bootstrapping the cluster until the server is available.

A deployment must use either all existing databases or all non-existing databases for the different cluster services; they cannot be mixed.