Cloudera Docs

Configuring Cloudera Copilot

After setting up credentials, you must make configuration changes in the Cloudera AI UI before using Cloudera Copilot.

Choosing a model

Models vary in accuracy, and cost. Larger models will provide more accurate responses but will cost more. For Cloudera AI Inference service models, larger models require more expensive GPU hardware to run on, while in Amazon Bedrock, larger models will cost more per prompt.

Language models vs Embedding models

Cloudera Copilot supports the following model types:

  • Language models: These are used for code completion, debugging, and chat.
  • Embedding models: These are used for Retrieval Augmented Generation (RAG) use cases. This allows you to augment language model responses with specific information that a language model is not aware of. For example, you can provide internal company documents that map company acronyms to their definitions.

Recommended models

  • Language models:
    • Llama 3.1 Instruct 70b (AI Inference)
    • Claude v3 Sonnet (Amazon Bedrock)
  • Embedding models:
    • E5 Embedding v5 (AI Inference)
    • Titan Embed Text v2 (Amazon Bedrock)
  1. In the Cloudera Data Platform console, click the Cloudera AI tile.
    The Cloudera AI Workbenches page displays.
  2. Click on the workbench name.
    The Workbenches Home page displays.
  3. Click Site Administration in the left navigation menu.
    The Site Administration page displays.
  4. Click Settings, and select the Enable Cloudera Copilot checkbox under Feature Flags.
    A new navigation tab Cloudera Copilot appears at the top of the Site Administration page.
  5. Click the Cloudera Copilot tab.
    The Cloudera Copilot page displays.
  6. Click Add Model. button.
  7. Select a model provider from the Model Provider dropdown list.
  8. In the Model field, provide the model name:
    • For Bedrock models: Select a model name from the Model dropdown list.
    • For Cloudera AI Inference service models, provide the model endpoint and the model_id as the model name string. You can get the model endpoint and model_id information from the Model Endpoint details page.
      • Example Model Endpoint: https://caii-prod-long-running.eng-ml-l.vnu8-sqze.yourcompany.site/namespaces/serving-default/endpoints/llama-31-8b-instruct-2xa10g/v1/chat/completions
      • Example model ID: phwq-gqmd-4kos-perd
    • For Cloudera AI Inference service embedding models, provide the string of the embedding model listed in the below table.
      Embedding Model Name Model String
      Mistral Embedding V2 nvidia/nv-embedqa-mistral-7b-v2
      Snowflake Arctic Embed Large Embedding snowflake/arctic-embed-l
      E5 Embedding v5 nvidia/nv-embedqa-e5-v5
      • Example embedding model endpoint: https://caii-prod-long-running.eng-ml-l.vnu8-sqze.yourcompany.site/namespaces/serving-default/endpoints/mistral-7b-embedding-onnx/v1/embeddings
    The model that you add for the first time is selected as the default language model automatically and the deselect option is disabled. This is to enforce that there is always one default language model. When you add more models, you can choose any of them to be the default language model.
  9. Click Add.
The model appears under Models or Third Party Models depending on the model provider type you selected.