Planning Your Deployment
Also available as:
PDF

Chapter 1. Deployment Scenarios

Your deployment scenario for installing, configuring, or upgrading your Hortonworks DataFlow (HDF) components depends on your particular use case.

Table 1.1. Installation Scenarios

ScenarioInstallation ScenarioSteps
Installing an HDF Cluster

This scenario applies if you want to install the entire HDF platform, consisting of all flow management and stream processing components on a new cluster.

The stream processing components include the new Streaming Analytics Manager (SAM) modules that are in GA (General Availability). This includes the SAM Stream Builder and Stream Operations modules but does not include installing the technical preview version of SAM Stream Insight, which is powered by Druid and Superset.

This scenario requires that you install an HDF cluster.

  1. Install Ambari.

  2. Install databases.

  3. Install the HDF management pack.

  4. Install an HDF cluster using Ambari.

Installing HDF Services on a New HDP Cluster

This scenario applies to you if you are both an Hortonworks Data Platform (HDP) and HDF customer and you want to install a fresh cluster of HDP and add HDF services.

The stream processing components include the new (SAM) and all of its modules. This includes installing the technical preview version of the SAM Stream Insight module, which is powered by Druid and Apache Superset.

This scenario requires that you install both an HDF cluster and an HDP cluster.

[Important]Important

You cannot install SAM and Schema Registry for HDF 3.1 on an HDP 2.6.4 cluster, and you cannot upgrade these services from a previous HDP cluster.

[Important]Important

You cannot upgrade your HDF Storm and Kafka versions if they exist on an HDP cluster.

  1. Install Ambari.

  2. Install databases.

  3. Install an HDP cluster using Ambari.

  4. Install the HDF management pack.

  5. Update the HDF base URL.

  6. Add HDF services to an HDP cluster

Installing HDF Services on an Existing HDP Cluster

You have an existing HDP cluster with Apache Storm and or Apache Kafka services and want to install Apache NiFi or NiFi Registry modules on that cluster.

This requires that you upgrade to the latest version of Apache Ambari and HDP, and then use Ambari to add HDF services to the upgraded HDP cluster.

[Important]Important

You cannot install SAM and Schema Registry for HDF 3.1 on an HDP 2.6.4 cluster, and you cannot upgrade these services from a previous HDP cluster.

[Important]Important

You cannot upgrade your HDF Storm and Kafka versions if they exist on an HDP cluster.

  1. Upgrade Ambari

  2. Upgrade HDP

  3. Install Databases

  4. Install HDF Management Pack

  5. Update HDF Base URL

  6. Add HDF Services to HDP cluster

Performing any of the previous deployments by using a local repository

See Using Local Repositories in the instructions appropriate for your scenario.

Local repositories are frequently used in enterprise clusters that have limited outbound internet access. In these scenarios, having packages available locally provides more governance and better installation performance.

This requires that you perform several steps to create a local repository and update the Ambari repository configuration file.

  1. Obtain the public repositories.

  2. Configure the local repository.

  3. Prepare the Ambari repository configuration file.


Table 1.2. Upgrade Scenarios

ScenarioUpgrade Scenario Steps
Upgrading an HDF cluster

You have an existing Ambari-managed HDF cluster and want to upgrade it using either Rolling or Express upgrade.

[Note]Note

Rolling Upgrade is not supported for NiFi. During the Rolling Upgrade, each NiFi instance is stopped, upgraded, and restarted.

  1. Upgrade Ambari and your HDF management pack.

  2. Upgrade HDF.

Upgrading an HDP cluster with only NiFi installedYou have an existing Ambari-managed HDP cluster with NiFi installed. You want to upgrade the NiFi service installed on your HDP cluster.
  1. Upgrade Ambari and your HDF management pack.

  2. Upgrade NiFi.

Upgrading an HDP cluster with all HDF services installed You have an existing Ambari-managed HDF cluster with all your HDF services installed and you want to upgrade the NiFi service, and migrate SAM and Schema Registry to an HDF cluster.
  1. Upgrade Ambari and your HDF management pack.

  2. Upgrade NiFi.

  3. Migrate SAM and Schema Registry to an HDF cluster.