Make sure to finish all current jobs running on the system before upgrading the cluster. Client libraries will change during the HDP upgrade. Any jobs remaining active that use the older version libraries will probably fail during the upgrade.