Uploading Model Repositories for an air-gapped environment

The Model artifacts must be manually transferred and uploaded to the object storage utilized by the Cloudera AI Registry and Cloudera AI Inference service.

Configure AWS CLI with the appropriate Ozone credentials.
- Generate the Ozone S3 secret key and Ozone access key.
  note
  Ensure that the user for whom you are generating the Ozone credentials, has the proper permissions in Ranger for the cm_ozone resource.
  1. SSH to the Cloudera Manager host. You can find the hostname in Cloudera Manager > Environments.
  2. Check for a valid Kerberos ticket by using the klist command.
    If a valid Kerberos ticket is not listed run the following command with the configured Keytab:
```
# kinit -kt /cdep/keytabs/om.keytab om
Run klist to check if kerberos ticket is granted.
[root@cml-pvc-oldap-1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: om@CDSW-INT.CLOUDERA.COM

Valid starting   	Expires          	Service principal
05/25/2022 14:32:13  05/26/2022 14:32:13  krbtgt/CDSW-INT.CLOUDERA.COM@CDSW-INT.CLOUDERA.COM
    renew until 06/02/2022 14:32:13
```
  3. Get the Ozone S3 secret key, that is the awsSecret and the Ozone access key, that is awsAccessKey by running the following command:
```
[root@cml-pvc-oldap-1 ~]# ozone s3 getsecret --om-service-id=ozone1

awsAccessKey=om@CDSW-INT.CLOUDERA.COM
awsSecret=40d5fc02fc882d53df3758a76184eb810ee97d9e1c45e5a7f2ef715bf31e5a0a
```
- Copy the awsAccessKey and awsSecret keys for configuring AWS CLI by executing the aws configure command:
```
aws configure
AWS Access Key ID [****************PAOF]: 
AWS Secret Access Key [****************U4Fz]: 
Default region name [None]: 
Default output format [None]:
```

Run the import_to_airgap.py script.

Use the following command for Hugging Face models:

python import_to_airgap.py -rt hf -ri hf-tiny-model-private/tiny-random-YosoModel

Upload the Model artifacts to a secured location.

Start uploading hf repository: hf-tiny-model-private/tiny-random-YosoModel

Use the following script for NVIDIA NGC (NIM) models:

python3 import_to_airgap.py -rt ngc -ri nim/meta/llama-3_1-70b-instruct:0.11.1+14957bf8-h100x4-fp8-throughput.1.2.18099809

The script is used to upload the downloaded Model artifacts to a secured location in Cloudera on premises. The destination format must be s3://bucket/secured-models. This bucket location can be modified by the Administrator to have a different value, as for example, s3://newBucket/not-secured .

You can use the parameters listed in the Parameters for uploading the Models table for uploading the Models.

Table 1. Paramaters for uploading the Models
Parameter	Description	Example
`-i`	Runs in insecure mode and skips SSL verification. This parameter is valid only for on premises environments, but Cloudera does not recommend it for production use cases. You must fetch the CA certificate to be used for the on-premises environment.	`-i`
`-ca`	Specifies the certificate to be used for Ozone S3 gateway communication.	`-ca /path/to/cert.pem`
`-e`	Specifies the Ozone S3 gateway endpoint URL	`-ehttps://ccycloud-5.cml-cai.root.comops.site:9879`
`-c`	Specifies the on-premises Cloud type	`-c pvc`
`-s`	This source directory must contain the previously downloaded Model artifacts.	`-s $PWD/models`
`-d`	Specifies the Cloudera AI Registry bucket with the appropriate path. The destination format must be: s3://bucket/secured-models	`-d s3://bucket/secured-models`
`-rt`	Specifies the repository type, either Hugging Face or NVIDIA NGC	`-rt hf`
`-ri`	Specifies the repository ID of the Model downloaded to the local filesystem	`-ri nim/meta/llama-3_1-70b-instruct: 0.11.1+14957bf8-h100x4-fp8-throughput. 1.2.18099809` You can obtain the `ri` argument for Hugging Face by performing the following actions: Go to the Hugging Face website at https://huggingface.co/. Search for the required model. The page of the model displays. Click the icon next to the name of the model, and copy the model ID. Use this ID as the `ri` argument.
`--account`	Specifies the Azure storage account name	`--account $AZURE_STORAGE_ACCOUNT_NAME`
`--container`	Specifies the Azure storage container name	`--container data`

A key artifact transfer is documented in the following log:

upload: ../.airgap/model/hf/hf-tiny-model-private/tiny-random-YosoModel/artifacts/pytorch\_model.bin to s3://[***BUCKET NAME***]/secured-models /hf/hf-tiny-model-private/tiny-random-YosoModel/artifacts/pytorch\_model.bin

The final output verifies the successful upload and specifies the destination path.

Uploaded: /root/.airgap/model/hf/hf-tiny-model-private/tiny-random-YosoModel -> s3://[***BUCKET NAME***]/secured-models/hf/hf-tiny-model-private/tiny-random-YosoModel
Finish uploading hf repository: hf-tiny-model-private/tiny-random-YosoModel to aws
Upload completed for repository: hf-tiny-model-private/tiny-random-YosoModel to aws

The destination format must be: s3://bucket/secured-models:

s3://[***BUCKET NAME***]/secured-models/hf/hf-tiny-model-private/tiny-random-YosoModel

Changing the default model upload location

The default model upload location is set to s3://bucket/secured-models, where bucket corresponds to the bucket used during the setup of Cloudera AI Registry. To change this default upload location, for example, to s3://bucket1/another-secured-models, make an API call to Cloudera AI Registry.

# python import_to_airgap.py --list-local-models

=== LOCAL MODELS ===
Repo Type       Repo ID                                            Download Success     Num Retries    
====================================================================================================
hf              hf-tiny-model-private/tiny-random-YosoModel        true                 0

After the API call is successfully executed, the new upload location is ready for use.