ModelsPDF version

Configuring Cloudera Copilot

After setting up credentials, you must make configuration changes in the Cloudera AI UI before using Cloudera Copilot.

Models vary in accuracy, and cost. Larger models will provide more accurate responses but will cost more. For Cloudera AI Inference service models, larger models require more expensive GPU hardware to run on, while in Amazon Bedrock, larger models will cost more per prompt.

Cloudera Copilot supports the following model types:

  • Language models: These are used for code completion, debugging, and chat.
  • Embedding models: These are used for Retrieval Augmented Generation (RAG) use cases. This allows you to augment language model responses with specific information that a language model is not aware of. For example, you can provide internal company documents that map company acronyms to their definitions.
  • Language models:
    • Llama 3.1 Instruct 70b (AI Inference)
    • Claude v3.5 Sonnet (Amazon Bedrock)
  • Embedding models:
    • E5 Embedding v5 (AI Inference)
    • Titan Embed Text v2 (Amazon Bedrock)
  1. In the Cloudera console, click the Cloudera AI tile.
    The Cloudera AI Workbenches page displays.
  2. Click on the workbench name.
    The Workbenches Home page displays.
  3. Click Site Administration in the left navigation menu.
    The Site Administration page displays.
  4. Click Settings, and select the Enable Cloudera Copilot checkbox under Feature Flags.
    A new navigation tab Cloudera Copilot appears at the top of the Site Administration page.
  5. Click the Cloudera Copilot tab.
    The Cloudera Copilot page displays.
  6. Select the type of model you want to add: Language Model or Embedding Model.
  7. Click Add Model button.
  8. Select a model provider from the Model Provider dropdown list.
  9. In the Model field, provide the model name:
    • For Bedrock models: Select a model name from the Model dropdown list.
    • For Cloudera AI Inference service models, provide the model endpoint and the model_id as the model name string. You can get the model endpoint and model_id information from the Model Endpoint details page.
      • Example Model Endpoint: https://caii-prod-long-running.eng-ml-l.vnu8-sqze.yourcompany.site/namespaces/serving-default/endpoints/llama-31-8b-instruct-2xa10g/v1/chat/completions
      • Example model ID: phwq-gqmd-4kos-perd
    The model that you add for the first time is selected as the default model automatically and the deselect option is disabled. This is to enforce that there is always one default model. When you add more models, you can choose one of them to be the default model. You can have one default language model, and one default embedding model.
  10. Click Add.
The model appears under Models or Third Party Models depending on the model provider type you selected.