Upgrading Log Rotation Configuration

Ambari 2.6.0 provides s a simplified log rotation configuration. These changes will be made automatically during your next stack upgrade, but are not automatically made during the Ambari upgrade. After upgrading Ambari from version 2.x to 2.6.0, if you want to utilize the simplified log rotation configuration, you must update configurations for all services in your cluster, using the following steps:

Steps

ZooKeeper
1. In Ambari Web, browse to ZooKeeper > Configs.
2. Scroll down to Custom zookeeper-log4j.
3. In Custom zookeeper-log4j, click Add Property.
4. In Add Property, type the following properties and values:
  zookeeper_log_max_backup_size=10
  zookeeper_log_number_of_backup_files=10
  For example:
5. Click Add.
6. Browse to Advanced zookeeper-log4j.
7. In Advanced zookeeper-log4j content section, find and replace the following properties and values:
  Find: log4j.appender.ROLLINGFILE.MaxFileSize=<value>
  Replace: log4j.appender.ROLLINGFILE.MaxFileSize={{zookeeper_log_number_of_backup_files}}MB
  Find: #log4j.appender.ROLLINGFILE.MaxBackupIndex=<value>MB
  Replace: #log4j.appender.ROLLINGFILE.MaxBackupIndex={{zookeeper_log_number_of_backup_files}}
  For example:
8. In Configs, click Save.
  For example:
9. Restart ZooKeeper, as prompted.
Kafka
1. In Ambari Web, browse to Kafka > Configs.
2. Scroll down to Custom Kafka-log4j.
3. In Custom Kafka-log4j, click Add Property.
4. In Add Property, type the following properties and values:
  kafka_log_maxfilesize=256
  kafka_log_maxbackupindex=20
  controller_log_maxfilesize=256
  controller_log_maxbackupindex=20
5. Click Add.
6. Browse to Advanced kafka-log4j.
7. In Advanced kafka-log4j content section, find and replace the following properties and values:
  Find: log4j.appender.kafkaAppender=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.kafkaAppender.MaxFileSize = {{kafka_log_maxfilesize}}MB
  Add: log4j.appender.kafkaAppender.MaxBackupIndex = {{kafka_log_maxbackupindex}}MB
  Find: log4j.appender.controllerAppender=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.controllerAppender.MaxFileSize = {{controller_log_maxfilesize}}MB
  Add: log4j.appender.controllerAppender.MaxBackupIndex = {{controller_log_maxbackupindex}}
8. In Configs, click Save.
9. Restart Kafka, as prompted.
Ranger
1. In Ambari Web, browse to Ranger > Configs > Advanced.
2. Scroll down to Custom admin-log4j.
3. In Custom admin-log4j, click Add Property.
4. In Add Property, type the following properties and values:
  ranger_xa_log_maxfilesize=256
  ranger_xa_log_maxbackupindex=20
5. Click Add.
6. Browse to Advanced admin-log4j.
7. In Advanced admin-log4j content section, find and replace the following properties and values:
  Find: log4j.appender.xa_log_appender=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.xa_log_appender.MaxFileSize={{ranger_xa_log_maxfilesize}}MB
  Add: log4j.appender.xa_log_appender.MaxBackupIndex={{ranger_xa_log_maxbackupindex}}
8. Scroll down to Custom usersync-log4j.
9. In Custom usersync-log4j, click Add Property.
10. In Add Property, type the following properties and values:
  ranger_usersync_log_maxfilesize=256
  ranger_usersync_log_number_of_backup_files=20
11. Click Add.
12. Browse to Advanced usersync-log4j.
13. In Advanced usersync-log4j content section, find and replace the following properties and values:
  Find: log4j.appender.logFile=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.logFile.MaxFileSize = {{ranger_usersync_log_maxfilesize}}MB
  Add: log4j.appender.logFile.MaxBackupIndex = {{ranger_usersync_log_number_of_backup_files}}
14. Scroll down to Custom tagsync-log4j.
15. In Custom tagsync-log4j, click Add Property.
16. In Add Property, type the following properties and values:
  ranger_tagsync_log_maxfilesize=256
  ranger_tagsync_log_number_of_backup_files=20
17. Click Add.
18. Browse to Advanced tagsync-log4j.
19. In Advanced tagsync-log4j content section, find and replace the following properties and values:
  Find: log4j.appender.logFile=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.logFile.MaxFileSize = {{ranger_tagsync_log_maxfilesize}}MB
  Add: log4j.appender.logFile.MaxBackupIndex = {{ranger_tagsync_log_number_of_backup_files}}
20. In Configs, click Save.
21. Restart Ranger, as prompted.
Ranger-KMS
1. In Ambari Web, browse to Ranger-KMS > Configs > Advanced.
2. Scroll down to Custom kms-log4j.
3. In Custom kms-log4j, click Add Property.
4. In Add Property, type the following properties and values:
  ranger_kms_log_maxfilesize=256
  ranger_kms_log_maxbackupindex=20
  ranger_kms_audit_log_maxfilesize=256
  ranger_kms_audit_log_maxbackupindex=20
5. Click Add.
6. Browse to Advanced kms-log4j .
7. In Advanced kms-log4j content section, find and replace the following properties and values:
  Find: log4j.appender.kms=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.kms.MaxFileSize = {{ranger_kms_log_maxfilesize}}MB
  Add: log4j.appender.kms.MaxBackupIndex = {{ranger_kms_log_maxbackupindex}}
  Find: log4j.appender.kms-audit=org.apache.log4j.DailyRollingFileAppender
  Add: log4j.appender.kms-audit.MaxFileSize={{ranger_kms_audit_log_maxfilesize}}MB
  Add: log4j.appender.kms-audit.MaxBackupIndex = {{ranger_kms_audit_log_maxbackupindex}}
8. In Configs, click Save.
9. Restart Ranger-KMS.
Storm
1. In Ambari Web, browse to Storm > Configs.
2. Scroll down to Custom cluster-log4j property.
3. In Custom cluster-log4j property, click Add Property.
4. In Add Property, type the following properties and values:
  storm_a1_maxfilesize=100
  storm_a1_maxbackupindex=9
5. Click Add.
6. Browse to Advanced storm-cluster-log4j .
7. In Advanced storm-cluster-log4j content section, find and replace the following properties and values:
  Find: In RollingFile="A1"<SizeBasedTriggeringPolicy size=”<value>MB"/>
  Replace: <SizeBasedTriggeringPolicy size="{{storm_a1_maxfilesize}}MB"/>
  Find: In RollingFile="A1"<DefaultRolloverStrategy max="<value>"/>
  Replace: <DefaultRolloverStrategy max="{{storm_a1_maxbackupindex}}"/>
8. Scroll down to Custom worker-log4j property.
9. In Custom worker-log4j property, click Add Property.
10. In Add Property, type the following properties and values:
  storm_wrkr_a1_maxfilesize=100
  storm_wrkr_a1_maxbackupindex=9
  storm_wrkr_out_maxfilesize=100
  storm_wrkr_out_maxbackupindex=4
  storm_wrkr_err_maxfilesize=100
  storm_wrkr_err_maxbackupindex=4
11. Click Add.
12. Browse to Advanced storm-worker-log4j .
13. In Advanced storm-worker-log4j content section, find and replace the following properties and values:
  Find: In RollingFile="A1"<SizeBasedTriggeringPolicy size="<value> MB"/>
  Replace: <SizeBasedTriggeringPolicy size="{{storm_wrkr_a1_maxfilesize}} MB"/>
  Find: In RollingFile="A1"<DefaultRolloverStrategy max="<value>"/>
  Replace: <DefaultRolloverStrategy max="{{storm_wrkr_a1_maxbackupindex}}"/>
  Find: In RollingFile="STDOUT"<SizeBasedTriggeringPolicy size="<value>" MB/>
  Replace: <SizeBasedTriggeringPolicy size="{{storm_wrkr_out_maxfilesize}} MB"/>
  Find: In RollingFile="STDOUT"<DefaultRolloverStrategy max="<value>"/>
  Replace: <DefaultRolloverStrategy max="{{storm_wrkr_out_maxbackupindex}}"/>
  Find: In RollingFile=”STDERR”<SizeBasedTriggeringPolicy size=”<value>” MB/>
  Replace: <SizeBasedTriggeringPolicy size="{{storm_wrkr_err_maxfilesize}} MB"/>
  Find: In RollingFile="STDOUT"<DefaultRolloverStrategy max="<value>"/>
  Replace: <DefaultRolloverStrategy max="{{storm_wrkr_err_maxbackupindex}}"/>
14. In Configs, click Save.
15. Restart Storm, as prompted.

​Upgrading Log Rotation Configuration

Upgrading Log Rotation Configuration