How to Use Antivirus Software on CDH Hosts

If you use antivirus software on your servers, consider configuring it to skip scans on certain types of Hadoop-specific resources. It can take a long time to scan large files or directories with a large number of files. In addition, if your antivirus software locks files or directories as it scans them, those resources will be unavailable to your Hadoop processes during the scan, and can cause latency or unavailability of resources in your cluster. Consider skipping scans on the following types of resources:
  • Scratch directories used by services such as Impala
  • Log directories used by various Hadoop services
  • Data directories which can grow to petabytes in size

The specific directory names and locations depend on the services your cluster uses and your configuration. Speak with your IT security team to determine which resources you should not scan. You can find the locations of the resources in the Cloudera Manager configuration settings.

In general, avoid scanning very large directories and filesystems. Instead, limit write access to these locations using security mechanisms such as access controls at the level of the operating system, HDFS, or at the service level.