Downloading model repositories for an air-gapped environment

To use models from Hugging Face and NVIDIA NGC (NIM), the Administrator must download model artifacts from these sources on specially networked hosts, bastion hosts.

Downloading a HuggingFace model

Download the model from Hugging Face to your local file system by using the following example command:

python import_to_airgap.py -do -rt hf -ri hf-tiny-model-private/tiny-random-YosoModel

The download includes all model files and metadata in the destination directory.

Table 1.
Argument	Description
-do	Specifies the download-only operation.
-rf hf	Indicates that the repository type is Hugging Face (hf).
-ri hf-tiny-model-private/tiny-random-YosoModel	Specifies the repository ID (model name) to download. You can obtain the `ri` argument for Hugging Face by performing the following actions: Go to the Hugging Face website at https://huggingface.co/. Search for the required model. The page of the model displays. Click the icon next to the name of the model, and copy the model ID. Use this ID as the `ri` argument.

note

If you are still using Cloudera AI on premises 1.5.5 or 1.5.5 SP1 and have not upgraded to Cloudera AI on premises 1.5.5 SP2, use the following command:

python3 import_to_airgap.py -do -rt hf -t hf_hVQbUCkpCicZYjnqsNAfafafafafafaAEkj -p $PWD/models -ri Nvidia/Llama-3.1-Nemotron-70B-Instruct-HF

Result:

The script successfully connected to the Hugging Face model Hub and initiated the download.

Fetching information for HF repository: 
hf-tiny-model-private/tiny-random-YosoModel

The download path is: /root/.airgap/model.

If you use the configure option the default path is $HOME/.airgap/model.

The initial metadata file was successfully downloaded and saved.

finish downloading the metadata file
Saved metadata to /root/.airgap/model/hf/hf-tiny-model-private/tiny-random-YosoModel/metadata/metadata.json

The script also downloaded the model artifacts using the Hugging Face CLI tool.

start downloading hf repository: hf-tiny-model-private/tiny-random-YosoModel
Downloading repository: hf-tiny-model-private/tiny-random-YosoModel to /root/.airgap/model
Starting download with huggingface-cli...
Fetching 6 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:02<00:00,  2.06it/s]
Download complete: : 401kB [00:02, 175kB/s]              /root/.airgap/model/hf/hf-tiny-model-private/tiny-random-YosoModel/artifacts00:02<00:02,  1.05it/s]
Download complete: : 401kB [00:02, 136kB/s]

The local metadata has been updated to include the downloaded model.

Download completed successfully for hf-tiny-model-private/tiny-random-YosoModel
Metadata updated for hf/hf-tiny-model-private/tiny-random-YosoModel

Downloading an NVIDIA NGC (NIM) model

Download the NVIDIA NGC (NIM) model to your local file system using the following command:
```
python3 import_to_airgap.py -do -rt ngc -ri nim/meta/llama-3_1-70b-instruct:0.11.1+14957bf8-h100x4-fp8-throughput.1.2.18099809 -ns ngc_spec.yaml
```
note
If you are still using Cloudera AI on premises 1.5.5 or 1.5.5 SP1 and have not upgraded to Cloudera AI on premises 1.5.5 SP2, use the following command:
```
python3 import_to_airgap.py -do -rt ngc -p $PWD/models -ri nim/meta/llama-3_1-70b-instruct:0.11.1+14957bf8-h100x4-fp8-throughput.1.2.18099809 -ns ngc_spec.yaml
```
For NVIDIA NGC (NIM) catalog downloads, use the -rt ngc argument and provide an NVIDIA NGC specification file with the -ns parameter. The -ns parameter is only required when downloading NVIDIA NGC models and the NGC-specific file to use is specified. Cloudera distributes these specific files to you. For more information, contact Cloudera Support.

The download includes all model files and metadata in the specified destination directory.