After setting up credentials, you must make configuration changes in the Cloudera AI UI before using Cloudera Copilot.
Choosing a model
Models vary in accuracy, and cost. Larger models will provide more accurate responses
but will cost more. For Cloudera AI Inference service models, larger
models require more expensive GPU hardware to run on, while in Amazon Bedrock,
larger models will cost more per prompt.
Language models vs Embedding models
Cloudera Copilot supports the following model types:
- Language models: These are used for code completion, debugging, and
chat.
- Embedding models: These are used for Retrieval Augmented Generation (RAG)
use cases. This allows you to augment language model responses with specific
information that a language model is not aware of. For example, you can provide
internal company documents that map company acronyms to their definitions.
Recommended models
- Language models:
- Llama 3.1 Instruct 70b (AI Inference)
- Claude v3.5 Sonnet (Amazon Bedrock)
- Embedding models:
- E5 Embedding v5 (AI Inference)
- Titan Embed Text v2 (Amazon Bedrock)
-
In the Cloudera
console, click the Cloudera AI
tile.
The Cloudera AI Workbenches page
displays.
-
Click on the workbench name.
The Workbenches Home page
displays.
-
Click Site Administration in the left navigation menu.
The Site Administration page displays.
-
Click , and select the Enable Cloudera Copilot
checkbox under Feature Flags.
A new navigation tab Cloudera Copilot appears at the
top of the Site Administration page.
-
Click the tab.
The Cloudera Copilot page
displays.
-
Select the type of model you want to add: Language Model
or Embedding Model.
-
Click Add Model button.
-
Select a model provider from the Model Provider dropdown
list.
-
In the Model field, provide the model name:
- For Bedrock models: Select a model name from the
Model dropdown list.
- For Cloudera AI Inference service models, provide the model
endpoint and the model_id as the model name string.
You can get the model endpoint and model_id
information from the Model Endpoint details
page.
- Example Model Endpoint:
https://caii-prod-long-running.eng-ml-l.vnu8-sqze.yourcompany.site/namespaces/serving-default/endpoints/llama-31-8b-instruct-2xa10g/v1/chat/completions
- Example model ID: phwq-gqmd-4kos-perd
The model that you add for the first time is selected as the
default model automatically and the deselect option
is disabled. This is to enforce that there is always one default model. When you
add more models, you can choose one of them to be the default model. You can
have one default language model, and one default embedding model.
-
Click Add.
The model appears under
Models or Third Party Models depending on the
model provider type you selected.