Setting up Python for PyFlink

Before you can use Flink with the Python API, you need to create and initialize a Python virtual environment.

  1. Connect to the Flink Gateway node using CLI.
    ssh <[***WORKLOAD USERNAME***]>@[***FLINK MANAGER NODE***]
    Provide your workload password when prompted.
  2. Create a Python virtual environment using the following command:
    conda create --copy -y -n flink_venv python=3.8
  3. Activate the newly created virtual environment:
    conda activate flink_venv
  4. Install PyFlink to the flink_venv virtual environment using the following command:
    python -m pip install apache-flink==1.18.0
  5. Create a ZIP archive from the flink_venv virtual environment so it can be deployed with a Flink job:
    cd path/to/flink_venv && zip -r venv.zip .

When the Python installation is complete, you can submit Flink application that were created using the Python API.