Spark ML is one of the dominant frameworks for many major machine learning algorithms, such as the Alternating Least Squares (ALS) algorithm for recommendation systems, the Principal Component Analysis algorithm, and the Random Forest algorithm. However, frequent misconfiguration means the potential of Spark ML is seldom fully utilized. Using native math libraries for Spark ML is a method to achieve that potential.
This article discusses how to accelerate model training speed by using native libraries for Spark ML. In addition, it discusses why Spark ML benefits from native libraries, how to enable the native libraries with CDH Spark, and provides performance comparisons between Spark ML on different native libraries.