The Hortonworks SmartSense Tool (HST) is used to collect cluster diagnostic data, both for assisting with support case troubleshooting, and for SmartSense analysis. HST can be installed as a standalone component and manually installed on all nodes of the cluster, or can be integrated with Apache Ambari for automatic installation and service management.

HST uses a central server daemon and a distributed set of passive HST agents. The HST Agent processes are not long-lived services, and are only started when specific data capture tasks are required. In order to provide the most complete picture of cluster utilization, HST agents must be installed on every node in the cluster. Once the HST Agent has captured the requested data, the process stops. All data captured by HST agents are sent to the central HST server to coalesce into a single downloadable bundle file. These bundles can then be uploaded to Support for troubleshooting, or to the SmartSense SFTP server for SmartSense analysis.

The process of uploading bundles depends on how HST has been deployed. For many customers, outbound internet access from the HST server is not readily available. For those users, the HST Gateway has been created to simplify the process of uploading bundles to Hortonworks. The Gateway allows customers with complex network environments to deploy a single Gateway that supports multiple internal HST Server deployments. With this deployment, direct outbound internet access from the HST server is not required for uploading bundles, only access from the HST Server to the Gateway. The Gateway is the only component that requires outbound internet access and will take on the responsibility of uploading all bundles to Hortonworks Support, or the SmartSense SFTP environment for SmartSense Analysis.

This architecture is optional, and customers can continue to upload bundles manually if required. For more details on the specific ports and communication options between the SmartSense Gateway and the Hortonworks Landing Zone, see SmartSense Ports & Traffic Flow.