Prerequisites for downloading and uploading Model artifacts in air-gapped environment

Before downloading or uploading models, ensure you have the following tools and configurations installed on the host that is connected to the airgap setup. This might be your bastion host.

• Required tools:

  • Hugging Face CLI: pip install -U "huggingface_hub[cli]"
  • AWS CLI: pip install awscli==1.35.0 is required for on premises setups.
  • PyYAML: pip install pyyaml

  • Click pip dependency: pip install -U click==8.2.1

  • NVIDIA NGC CLI:https://org.ngc.nvidia.com/setup/installers/cli for NVIDIA NGC catalog Models.

    Make sure you configure the NVIDIA NGC client with the credentials provided by Cloudera.

  • Python: Make sure the Python version is 3.10.12 or higher and lower than version 3.11.
• Configuration details: Configure the NGC client with credentials provided by Cloudera during onboarding. Contact Cloudera support or raise a ticket to get the credentials. Use the following commands to add your key and organization to your ~/.bashrc file.

  echo 'export NGC_CLI_API_KEY=<key>' >> ~/.bashrc 
        echo 'export NGC_API_KEY=<key>' >> ~/.bashrc 
        echo 'export NGC_CLI_ORG=<org>' >> ~/.bashrc

  If the system has ~/.bash_profile follow the above steps, but replace bashrc with bash_profile.

• Allow the URLs listed in Model Hub in air-gapped installations.

• Downloading the script

  Download the import_to_airgap.py script to enable downloading Model repositories from the Hugging Face or NVIDIA NGC catalog and uploading Models to on premises storage providers.

  • Path for Cloudera AI on premises 1.5.5 SP4
  • Path for Cloudera AI on premises 1.5.5 SP3 and lower releases

  Table 1. import_to_airgap.py download path for Cloudera AI on premises 1.5.5 SP4

  Release	Download location
  Cloudera AI on premises 1.5.5 SP4	https://raw.githubusercontent.com/cloudera/Model-Hub/refs/heads/main/airgap-scripts/pvc/1.5.5-sp4/import_to_airgap.py

  Table 2. import_to_airgap.py download path for Cloudera AI on premises 1.5.5 SP3 and lower releases

  Release	Download location
  Cloudera AI on premises 1.5.5 SP3	https://raw.githubusercontent.com/cloudera/Model-Hub/refs/heads/main/airgap-scripts/pvc/1.5.5-sp3/import_to_airgap.py
  Cloudera AI on premises 1.5.5 SP2	https://raw.githubusercontent.com/cloudera/Model-Hub/refs/heads/main/airgap-scripts/pvc/1.5.5-sp2/import_to_airgap.py
  Cloudera AI on premises 1.5.5 SP1	https://raw.githubusercontent.com/cloudera/Model-Hub/refs/heads/main/airgap-scripts/pvc/1.5.5-sp1/import_to_airgap.py
  Cloudera AI on premises 1.5.5	https://github.com/cloudera/Model-Hub/blob/main/airgap-scripts/pvc/1.5.5/import_to_airgap.py

  The script has the following parameters:

  Table 3.

  Parameter	Value	Description
  -do		Activates download mod
  -rt	hf	Repository type: hf for Hugging Face, use ngc for NVIDIA NGC catalog
  -t	hf_hVQbUsafafafafadfadfsNAynASXJoTCWHAEkj	Hugging Face API token for authentication, required for private or gated Models The Hugging Face token -t is required for accessing gated Models Models or Models that require authentication. For more information about tokens, see:https://huggingface.co/docs/hub/en/security-tokens.
  -p	$PWD/models	Local destination path where Model files are downloaded (uses current working directory)
  -ri	Nvidia/Llama-3.1-Nemotron-70B-Instruct-HF	Repository ID for the model on Hugging Face. profileId of the optimization profile in the NGC specification for the model. You can obtain the ri argument for Hugging Face as follows: Open up the Hugging Face page at: https://huggingface.co/. Search for the required model. The page of the model displays. Click the icon next to the name of the model, and copy the model ID, that is the ri argument to be used.
  -ns	ngc_spec.yaml	NGC Specification File: Required when downloading NGC models.

  To download an NGC model using the script, you must first download the NGC specification YAML file. This file will be provided as the -ns argument to the airgap script:

  • Path for Cloudera AI on premises 1.5.5 SP4
  • Path for Cloudera AI on premises 1.5.5 SP3 and lower releases

  Table 4. NGC specification YAML file download path for Cloudera AI on premises 1.5.5 SP4

  Release
  Cloudera AI on premises 1.5.5 SP4	https://github.com/cloudera/Model-Hub/blob/main/models/airgapped/private/1.60.1_concatenated.yaml

  Table 5. NGC specification YAML file download path for Cloudera AI on premises 1.5.5 SP3 and lower releases

  Release
  Cloudera AI on premises 1.5.5 SP3	https://github.com/cloudera/Model-Hub/blob/main/models/airgapped/private/1.56.0-h3000_concatenated.yaml
  Cloudera AI on premises 1.5.5 SP2	https://raw.githubusercontent.com/cloudera/Model-Hub/refs/heads/main/models/airgapped/private/1.51.0-h2000_concatenated.yaml
  Cloudera AI on premises 1.5.5 SP1	https://raw.githubusercontent.com/cloudera/Model-Hub/refs/heads/main/models/airgapped/private/1.51.0_concatenated.yaml
  Cloudera AI on premises 1.5.5	https://github.com/cloudera/Model-Hub/blob/main/models/airgapped/private/1.49.0_concatenated.yaml

• Installing the NVIDIA Inference Microservice (NIM) CLI
  Request early access for Cloudera organization accounts and install the NIM CLI.

  1. Installing NIM CLI: Once early access is granted, use the following steps to download and install the NIM CLI:
     1. Download the installer using the NVIDIA GPU Cloud (NGC) CLI:

        ngc registry resource download-version nvidia/nim-tools/nimtools_installer:0.0.8

     2. Navigate to the installer directory:

        cd nimtools_installer_v0.0.8/

     3. Run the Python installation script. Make sure you provide your NGC service key and the --nimcli-only flag:

        python3 nimtools_installer.py --ngc-api-key [***your-ngc-service-key***] --nimcli-only

        To install nimlib properly you might alternatively use the following command:

        python3 nimtools_installer.py --ngc-api-key [***your-ngc-service-key***]

     4. Add NIM CLI to the Linux PATH.

        By default, the installed NIM CLI binary is not automatically added to the Linux PATH. Include manually the NIM CLI binary directory in the PATH to make it accessible:

        1. Locate the NIM CLI binary directory.

           find / -name nimcli

           For example:

           /root/.pyenv/versions/3.10.19/bin/nimcli

        2. Add the directory to the PATH.

           export PATH=$PATH:/root/.pyenv/versions/3.10.19/bin