Hortonworks Docs
»
Cloudbreak 2.9.2
»
Advanced Cluster Options
Advanced Cluster Options
Also available as:
Release Notes
Release Notes
About this release
New features
Technical preview features
Behavioral changes
Fixed issues
Known issues
Image catalog updates
Legal information
Concepts
Introduction to Cloudbreak
What is Cloudbreak
Primary use cases
Interfaces
Core concepts
Architecture
Cloudbreak deployer architecture
Cloudbreak application architecture
Cloudbreak Security Overview
Security overview
Virtual networks
Network security
Cloudbreak instance security group
Cluster security groups
Identity management
Authentication with AWS
Authentication with Azure
Authentication with GCP
Authentication with OpenStack
Planning
Cloudbreak Deployment Options
Cloudbreak deployment options
Deployment option cheatsheet
Quickstart option for AWS
Quickstart option for Azure
Quickstart option for GCP
Production deployment option
Cluster deployment options
Default cluster configurations
Deploying on AWS GovCloud
Using Cloudbreak with special Azure regions
Installation & Upgrade
Quickstart on AWS
Prerequisites on AWS
Launch Cloudbreak from the quickstart template on AWS
Access Cloudbreak web UI on AWS
Next steps
Delete Cloudbreak on AWS
Quickstart on Azure
Prerequisites on Azure
Launch Cloudbreak from the quickstart template on Azure
Access Cloudbreak web UI on Azure
Next steps
Delete Cloudbreak on Azure
Quickstart on GCP
Prerequisites on GCP
Cloud SDK
GCP APIs
Service account
Browser
Launch Cloudbreak from the quickstart template on GCP
Next steps
Delete Cloudbreak on GCP
Installing Cloudbreak on AWS
Prerequisites on AWS
AWS account
AWS region
Virtual network
Security group
SSH key pair
Browser
Preparing the VM
System requirements
Root access
System updates
Iptables
Disable SELINUX
Docker
Configuring authentication with AWS
Create CloudbreakRole
Attach CloudbreakRole to the VM
Install Cloudbreak on a VM
Access Cloudbreak web UI
Next steps
Installing Cloudbreak on Azure
Prerequisites on Azure
Azure account
Azure region
Virtual network
Security group
SSH key pair
Browser
Preparing the VM
System requirements
Root access
System updates
Iptables
Disable SELINUX
Docker
Install Cloudbreak on a VM
Access Cloudbreak web UI
Next steps
Installing Cloudbreak on GCP
Prerequisites on GCP
GCP account and project
Region and zone
Virtual network
Security group
SSH key pair
Browser
Preparing the VM
System requirements
Root access
System updates
Iptables
Disable SELINUX
Docker
Install Cloudbreak on a VM
Access Cloudbreak web UI
Next steps
Installing Cloudbreak on OpenStack
Prerequisites on OpenStack
System requirements
Supported OpenStack distributions
Standard modules
Virtual network
Security group
SSH key pair
Browser
Preparing the VM
System requirements
Root access
System updates
Iptables
Disable SELINUX
Docker
Install Cloudbreak on a VM
Configure a self-signed certificate
Access Cloudbreak web UI
Next steps
Upgrading Cloudbreak
Upgrade Cloudbreak to the latest version
Upgrade Cloudbreak with no internet access
Update Cloudbreak to a version different than the latest
Installing CLI
Installing Cloudbreak CLI
Install Cloudbreak CLI
Configure Cloudbreak CLI
Advanced CLI configuration
Configure CLI for use with workspaces
Add multiple configurations
Configure default output
Configure CLI autocomplete
Web UI options useful for CLI
Obtain cluster JSON template
Obtain CLI command
Obtain cluster blueprint
Get help
How To
Creating a Credential on AWS
Credential options on AWS
Creating a key-based credential
Prerequisites for key-based credential
Create a key-based credential
Creating a role based credential
Create CredentialRole
Create a role-based credential
Creating a Credential on Azure
Credential options on Azure
Required Azure roles
Create an interactive credential
Create an app-based credential
Creating a Credential on GCP
Overview of creating a credential on GCP
Create a service account
Create a credential on GCP
Creating a Credential on OpenStack
Create a credential on OpenStack
Creating a Cluster on AWS
Create a cluster on AWS
Default cluster configurations
Cluster security groups
Guidelines for creating HDF clusters
Creating HDF Flow Management clusters
Creating HDF Messaging Management clusters
Advanced cluster options
Availability zone
Enable lifetime management
Tags
Image settings
Ambari repo specification
HDP/HDF repo specification
Use spot instances
Cloud storage
Recipes
Management packs
Custom properties
External sources
Ambari server master key
Enable Kerberos security
Gateway configuration
Services available via gateway
Configure the gateway
Configure single sign-on (TP)
Obtain gateway URLs
Creating a Cluster on Azure
Create a cluster on Azure
Default cluster configurations
Cluster security groups
Guidelines for creating HDF clusters
Creating HDF Flow Management clusters
Creating HDF Messaging Management clusters
Advanced cluster options
Resource group name
Enable lifetime management
Tags
Image settings
Ambari repo specification
HDP/HDF repo specification
Availability sets
Cloud storage
Recipes
Management packs
Custom properties
External sources
Don't create public IP
Don't create new firewall rules
Ambari server master key
Enable Kerberos security
Gateway configuration
Services available via gateway
Configure the gateway
Configure single sign-on (TP)
Obtain gateway URLs
Creating a Cluster on GCP
Create a cluster on GCP
Default cluster configurations
Cluster security groups
Guidelines for creating HDF clusters
Creating HDF Flow Management clusters
Creating HDF Messaging Management clusters
Advanced cluster options
Availability zone
Enable lifetime management
Tags
Image settings
Ambari repo specification
HDP/HDF repo specification
Use preemptible instances
Shared networks
Cloud storage
Recipes
Management packs
Custom properties
External sources
Don't create public IP
Don't create new firewall rules
Ambari server master key
Enable Kerberos security
Gateway configuration
Services available via gateway
Configure the gateway
Configure single sign-on (TP)
Obtain gateway URLs
Creating a Cluster on OpenStack
Create a cluster on OpenStack
Default cluster configurations
Cluster security groups
Guidelines for creating HDF clusters
Creating HDF Flow Management clusters
Creating HDF Messaging Management clusters
Advanced cluster options
Availability zone
Enable lifetime management
Tags
Image settings
Ambari repo specification
HDP/HDF repo specification
Recipes
Management packs
Custom properties
External sources
Ambari server master key
Enable Kerberos security
Gateway configuration
Services available via gateway
Configure the gateway
Configure single sign-on (TP)
Obtain gateway URLs
Accessing Clusters
Cloudbreak user accounts
Finding cluster information in the web UI
Cluster summary
Cluster information
Event history
Access cluster via SSH
Access Ambari
Access Hive via JDBC
Download SSL certificate
Example: SQL Workbench/J
Example: Tableau
Managing Clusters
Managing and monitoring clusters
Retry a cluster
Resize a cluster
Synchronize a cluster
Stop a cluster
Restart a cluster
Terminate a cluster
Force terminate a cluster
Deleting clusters when termination fails
Repairing a cluster
Enabling auto or manual repair
Performing manual repair
When repair fails on NAMENODE in an HA cluster
Configure autoscaling
Enable autoscaling
Define an alert
Create a scaling policy
Configure autoscaling settings
Add SSL certificate for Knox Gateway
Add SSL certificate for Ambari
Updating OS and tools on long-running clusters
Updating Ambari, HDP, and HDF on long-running clusters
Advanced Cluster Options
Custom images
Build custom images
Prepare image catalog
Structure of the image catalog
Example image catalog
Register image catalog
Set CB_ENABLED_LINUX_TYPES
Create clusters with custom images
Custom blueprints
Creating blueprints
Creating dynamic blueprints
External authentication source
External database
Upload blueprints
Recipes
Writing recipes
Recipe parameters
Add recipes
Reusable recipes
Install mysql connector recipe
Management packs
Add management pack
Kerberos security
Using existing KDC
Using test KDC
EBS encryption on AWS
Permissions for using EBS encryption
Encryption key requirements
Create a cluster with encrypted EBS volumes
Disk encryption on GCP
Encryption key requirements
Permissions required for key encryption
Create a cluster with key encryption
External databases for cluster components
Supported databases
External database options
Example 1: Built-in type Hive
Example 2: Other type
Creating a template blueprint for RDMBS
Register an external database
External authentication source for clusters
Preparing the blueprint for LDAP/AD
Register an authentication source
Custom internal hostnames for cluster hosts
Custom hostnames based on DNS on AWS
Configure DNS using Route53
Configure DNS using custom DNS server
Configuring Access to Cloud Data
Configuring access to Amazon S3
Create an IAM role for S3 access
Configure access to S3
Test access from HDP to S3
Configure S3 storage locations
Configuring access to ADLS Gen1
Prerequisites
Configure access to ADLS Gen1
Test access to ADLS Gen1
Configure ADLS Gen1 storage locations
Configuring access to ADLS Gen2
Prerequisites
Configure access to ADLS Gen2
Test access to ADLS Gen2
Configure ADLS Gen2 storage locations
Configuring access to WASB
Prerequisites
Configure access to WASB
Test access to WASB
Configure WASB storage locations
Configuring access to GCS
Prerequisites
Configure access to GCS
Test access to GCS
Configuring GCS storage locations
Working with Data Lakes
Data lake overview
Data lake blueprints
Setting up a data lake
Prerequisites
Register databases and LDAP
Create a data lake
Create attached workloads
Performing manual HA data lake recovery
Advanced Cloudbreak Configuration
External Cloudbreak database
Supported databases
Configure external Cloudbreak database
Configure an SSL certificate for an external Cloudbreak database
LDAP/AD for Cloudbreak
LDAP/AD information
Configuring Cloudbreak for LDAP/AD
Configure user authentication
Configure group authorization
Outbound internet access and proxy
Outbound network access destinations
Using a proxy
Configure Cloudbreak to use a proxy
Configuring clusters to use a proxy
Advanced proxy setup scenarios
Restrict inbound access to clusters
Use SSL certificate for Cloudbreak
Access from custom domains
Moving Cloudbreak instance
Back up Cloudbreak database
Populate database with dump from original Cloudbreak instance
Modify Cloudbreak Profile
Disable providers
Modify default Cloudbreak ports
Modify Cloudbreak credential
Set default Cloudbreak credential
Set default cluster wizard view
Set up SMTP email notifications
Import HDP and HDF images to OpenStack
Cloudbreak Profile
Secure the Profile file
Check available Profile variables
Set Profile variables
Create environment-specific Profiles
Add tags in Profile (AWS)
Modifying UAA_DEFAULT_SECRET
SmartSense telemetry
Disable bundle upload for Cloudbreak and new clusters
Disable bundle upload for an existing cluster
Managing and Monitoring Cloudbreak
Workspaces
Prerequisites for using workspaces
Manage workspaces
Access a workspace
Operations audit logging
Enable audit logging output
Audit resource events
Troubleshooting
Troubleshooting Cloudbreak
Migrate existing clusters to access new packages behind the paywall
Getting help
HCC
Flex subscription
Configure SmartSense
Register and manage Flex subscriptions
Use Flex subscription for a cluster
Use Flex subscription for Cloudbreak node
More Cloudbreak resources
Create a troubleshooting bundle
Checking Cloudbreak logs
Cloudbreak logs
Saltstack logs
Ambari logs
Recipe logs
Troubleshooting Cloudbreak
Invalid PUBLIC_IP in CBD Profile
Cbd cannot get VM's public IP
Docker does not start and returns an error
Permission or connection problems
Creating cbreak_sultans_1 … Error
Copy the JSON button does not work in Firefox
UnicodeEncodeError when starting Cloudbreak
Cloudbreak not working after VM restart
Troubleshooting Cloudbreak on AWS
User is not authorized to perform: sts:AssumeRole
Troubleshooting Cloudbreak on Azure
Resource was not found
You don't have permissions to assign roles on Azure
Credential creation errors on Azure
Troubleshooting cluster creation
Configure communication via private IPs on AWS
Cannot access Oozie web UI
Failed to retrieve the server's certificate
Quota limitations
Connection timeout when ports are not open
Blueprint errors
Recipe errors
Troubleshooting Cloudbreak CLI
Reference
CLI Reference
Cloudbreak CLI reference
Command structure
Command output
Commands
audit describe
audit list
blueprint create
blueprint delete
blueprint describe
blueprint list
cloud availability-zones
cloud regions
cloud volumes
cloud instances
cluster change-ambari-password
cluster change-image
cluster create
cluster delete
cluster describe
cluster generate-attached-cluster-template
cluster generate-template
cluster generate-reinstall-template
cluster list
cluster maintenance-mode
cluster reinstall
cluster repair
cluster retry
cluster scale
cluster start
cluster stop
cluster sync
completion
configure
credential create
credential delete
credential describe
credential list
credential modify
database create
database delete
database list
database test
imagecatalog create
imagecatalog delete
imagecatalog describe
imagecatalog images
imagecatalog list
imagecatalog set-default
ldap create
ldap delete
ldap list
mpack create
mpack delete
mpack list
proxy create
proxy delete
proxy list
recipe create
recipe delete
recipe describe
recipe list
user list
workspace add-user
workspace create
workspace delete
workspace describe
workspace list
workspace remove-user
Debugging
Checking CLI Version
Developer Documentation
Developer documentation links
Reusable recipes
The following section includes recipes for running common tasks.
Install mysql connector recipe
This recipe can be used to manually install and register the ‘mysql-connector-java.jar’.
Parent topic:
Recipes
© 2012–2019, Hortonworks, Inc.
Document licensed under the
Creative Commons Attribution ShareAlike 4.0 License
.
Hortonworks.com
|
Documentation
|
Support
|
Community