Cloudera Docs

Installing standard Python 3.8 binary on CentOS 7, CentOS 8 at a standard or custom location

Cloudera Manager agents and Hue require Python 3.8. You must install Python 3.8.12 or higher on all cluster hosts before installing Cloudera Manager and adding services to your cluster. Follow the instructions in this topic to manually download, install, and build standard Python 3.8 binaries from source on CentOS 7 and CentOS 8 at a standard or custom location.

You must manually download, install, and build Python 3.8 from source on CentOS 7 and CentOS 8 because:
  • Python 2.7.5 is preinstalled on CentOS 7, but CDP requires Python 3.8
  • Python is not distributed with CentOS 8, unlike other Linux distributions

You can install Python 3.8 in a standard location or in a custom location depending on your organization's policies. To install Python 3.8 in a custom location, replace [***CUSTOM-INSTALL-PATH***] in the instructions with the custom path.

Install the following necessary developer tools and packages on your system:
  • gcc
  • make
  • openssl-devel
  • bzip2-devel
  • libffi-devel
  • zlib-devel
yum install gcc openssl-devel bzip2-devel libffi-devel zlib-devel -y
Installing Python 3.8 from CentOS Software Collections (SCL)
  1. You can install the Python 3.8 from CentOS Software Collections (SCL) by running the following commands:
    yum install centos-release-scl ( part of the extra repository )
yum install rh-python38 rh-python38-python-pip rh-python38-python-pip-wheel
  2. Verify Python 3.8 is installed successfully.
  3. If you are unable to install Python 3.8 from CentOS Software Collections (SCL), then you can perform the following steps to build the Python 3.8 from source distribution.
  1. SSH into the host system as a root user.
  2. Change directory to /opt: 
    cd /opt
  3. Download Python 3.8 as follows: 
    curl -O https://www.python.org/ftp/python/3.8.17/Python-3.8.17.tgz
  4. Decompress the package as follows: 
    tar -zxvf Python-3.8.17.tgz
  5. Change directory to where you decompressed the Python 3.8 package: 
    cd /opt/Python-3.8.17
  6. Install Python 3.8 as follows:
    Standard location: 
    ./configure --enable-shared
    Custom location: 
    ./configure --enable-shared --prefix=[***CUSTOM-INSTALL-PATH***]
    If you are installing Python 3.8 in any non-standard location, then you must specify the path using the --prefix option. You must also create a symbolic link pointing to /usr/bin or to /usr/local/bin by running the following command:
    ln -s [***SOURCE***] [***DESTINATION***]
    For example: 
    ln -s [***CUSTOM-INSTALL-PATH***]/python3.8 /usr/bin/python3.8
    The --enabled-shared option is used to build a shared library instead of a static library.
  7. Build Python 3.8 as follows:
    1. Run the make command to compile the files: 
      make
    2. Run the following command to put the compiled files in the default location, or in the custom location that you specified using the --prefix option: 
      make altinstall
      Using altinstall prevents you from overwriting the system Python, if any.
    3. Copy the shared compiled library files (libpython3.8.so) to the /lib64/ directory: 
      cp --no-clobber ./libpython3.8.so* /lib64/
      The --no-clobber option is used to prevent overwriting files.
    4. Change the permissions of the libpython3.8.so files as follows: 
      chmod 755 /lib64/libpython3.8.so*
    If you see an error such as error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory, then run the following command: 
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/
  8. Change the permission of the Python 3.8 installation directory to 755 so that Hue and its related services can leverage the binary and the site packages, as follows: 
    chmod -R 755 /usr/local/lib/python3.8
Enable Cloudera Manager to use Python 3.8 installed in a custom location

If you have installed Python 3.8 at a custom location, then you must either create a symbolic link using the alternatives command or by adding an environment variable to the Cloudera Manager Agent in the systemd service definition files with the full path to the Python 3.8 custom-built binary.

To use the alternatives command, get help by running the command alternatives --help. You should create symbolic links from either /usr/bin/python3.8 or /usr/local/bin/python3.8 to [***CUSTOM-INSTALL-PATH***]/bin/python3.8 where you installed the Python 3.8 package.

To add an environment variable to the Cloudera Manager Agent in the systemd service definition files perform the following steps:
  1. SSH into the host system on which you installed Python 3.8 as a root user.
  2. Open the /usr/lib/systemd/system/cloudera-scm-supervisord.service file for editing.
  3. Add the following line in the cloudera-scm-supervisord.service file under the [Service] section:
    Environment=”PYTHONBIN=[***CUSTOM-INSTALL-PATH***]/bin/python3.8”
  4. Open the /usr/lib/systemd/system/cloudera-scm-agent.service file for editing.
  5. Add the following line in the cloudera-scm-agent.service file under the [Service] section:
    Environment=”PYTHONBIN=[***CUSTOM-INSTALL-PATH***]/bin/python3.8”
Enable Hue to use Python 3.8 installed in a custom location
If you have installed Python 3.8 in a custom location and you did not configure alternatives, then you must prepend the custom path to the custom bin directory in Cloudera Manager > Clusters > Hue > Configuration > Hue Service Environment Advanced Configuration Snippet (Safety Valve) separated by colon (:) as follows:

Key: PATH

Value: [***CUSTOM-INSTALL-PATH***]:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin

Install the psycopg2-binary package
Install the psycopg2-binary package after installing Python 3.8 for using PostgreSQL as a backend database for Hue.