Cloudera Docs

Installing the psycopg2 Python package for PostgreSQL database

If you are using PostgreSQL as a backend database for Hue on CDP Private Cloud Base 7, then you must install the 2.9.3 version of the psycopg2 package on all Hue hosts. The psycopg2 package is automatically installed as a dependency of Cloudera Manager Agent, but the version installed is often lower than 2.9.3.

Before you begin, you must disable the postgresql10 section from the cloudera-manager.repo file as follows:
  1. SSH in to the Cloudera Manager host as an Administrator.
  2. Change to the directory where you had downloaded the cloudera-manager.repo file. On RHEL, the file is present under the /etc/yum.repo.d directory.
  3. Open the file for editing and update the value of the enabled property to 0 as follows:
    [postgresql10]
name=Postgresql 10
baseurl=https://archive.cloudera.com/postgresql10/redhat8/
gpgkey=https://archive.cloudera.com/postgresql10/redhat8/RPM-GPG-KEY-PGDG-10
enabled=0 
gpgcheck=1
module_hotfixes=true
  4. Save the file and exit.
  1. SSH into the Hue host as a root user.
  2. Install the psycopg2 package dependencies for CentOS 7 by running the following commands:
    yum install -y centos-release
yum install -y centos-release-scl
yum install -y xmlsec1  xmlsec1-openssl
  3. Install the rh-postgresql-postgresql-devel package corresponding to your database version by running the following command:
    yum install -y rh-postgresql[***DB-VERSION***]-postgresql-devel
  4. Add the location of the installed rh-postgresql-postgresql-devel package to the PATH environment variable by running the following command:
    export PATH=/opt/rh/rh-postgresql[***DB-VERSION***]/root/usr/bin:$PATH
  5. Verify the database version by running the following command: 
    psql -V
  6. Install the psycopg2 package by running the following command: 
    pip3.8 install psycopg2==2.9.3
If you see an error such as ImportError: libpq.so.rh-postgresql10-5: cannot open shared object file: No such file or directory, then you can do one of the following:
  • Add an enviornment variable in the Hue Service Environment Advanced Configuration Snippet (Safety Valve) field as follows:

    key: LD_LIBRARY_PATH

    value: /opt/rh/rh-postgresql[***DB-VERSION***]/root/usr/lib64:$LD_LIBRARY_PATH

    OR

  • Create a symlink of libpq.so.rh-postgresql10-5 from /opt/rh/rh-postgresql[***DB-VERSION***]/root/usr/lib64/libpq.so.rh-postgresql10-5.10 to /usr/lib64/
To install the centos7-extras repo, perform the following steps:
  1. Obtain the release number of the operating system and input it into the OS_RELEASE_MAJOR variable by running the following commands:
    OS_RELEASE=$(rpm -q --qf "%{VERSION}" $(rpm -q --whatprovides redhat-release))
    OS_RELEASE_MAJOR=$(echo $OS_RELEASE | cut -d. -f1)
  2. Download the GPG key and save it locally by running the following command:
    curl -s https://www.centos.org/keys/RPM-GPG-KEY-CentOS-$OS_RELEASE_MAJOR | sudo tee /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-$OS_RELEASE_MAJOR
  3. Create the repo file named Centos-Extras.repo in the /etc/yum.repos.d directory by running the following command:
    vi /etc/yum.repos.d/Centos-Extras.repo
  4. Add the following lines in the Centos-Extras.repo file, save the file and exit:
    [extras] 
name=CentOS-$releasever - Extras baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ 
#baseurl=http://mirror.infra.cloudera.com/repos/centos/7/extras/\$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
priority=1
  1. SSH into the Hue host as a root user.
  2. Install the psycopg2 package dependencies for RHEL 8 by running the following commands:
    dnf install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm
    yum install -y xmlsec1  xmlsec1-openssl
  3. Disable the postgresql stream module in the AppStream by running the following command:
    dnf -qy module disable postgresql
  4. Install the PostgreSQL database server corresponding to your database version by running the following command:
    dnf install -y postgresql[***DB-VERSION***]-server
  5. Install the postgresql-devel package corresponding to your database version by running the following command:
    yum install -y postgresql[***DB-VERSION***]-devel
  6. Add the location of the installed postgresql-devel package to the PATH environment variable by running the following command:
    export PATH=/usr/pgsql-[***DB-VERSION***]/bin:$PATH
  7. Install the psycopg2 package by running the following command: 
    pip3.8 install psycopg2==2.9.3
  1. SSH into the Hue host as a root user.
  2. Install the psycopg2 package dependencies for SLES 12 by running the following commands:
    zypper install xmlsec1
zypper install xmlsec1-devel
Zypper install xmlsec1-openssl-devel
  3. Install the postgresql-devel package corresponding to your database version by running the following command:
    zypper -n postgresql[***DB-VERSION***]-devel
  4. Add the location of the installed postgresql-devel package to the PATH environment variable by running the following command:
    export PATH=$PATH:/usr/local/bin
  5. Install the psycopg2 package by running the following command: 
    pip3.8 install psycopg2==2.9.3 --ignore-installed
  1. SSH into the Hue host as a root user.
  2. Install the psycopg2 package dependencies for Ubuntu by running the following commands:
    apt-get install -y xmlsec1
apt-get install libxmlsec1-openssl
apt-get install libpq-dev python3-pip -y
  3. Install the python3-dev and libpq-dev packages by running the following command:
    apt install python3-dev libpq-dev
  4. Add the location of the installed postgresql-devel package to the PATH environment variable by running the following command:
    export PATH=$PATH:/usr/local/bin
  5. Install the psycopg2 package by running the following command: 
    pip3.8 install psycopg2==2.9.3 --ignore-installed