Cloudera Docs

Processing the Evaluations

The Evaluations are processed in two different stages to balance speed and qualitative depth.

Automatic metrics

Upon successful completion of a workflow run, the Automatic Metrics are instantly populated. These metrics are deterministic and do not necessitate an additional LLM call. For more information on the automatic metrics, see Metrics Reference Glossary.

Qualitative analysis (LLM as a Judge)

Qualitative metrics require a manual trigger. Click the Run LLM as a judge evals button to initiate this analysis.

  • Redundancy Protection: To prevent duplicate compute costs, the button is automatically disabled if all LLM judges for that specific run context have already completed.
  • Partial Execution: If new evaluators are added or some failed, the button will only trigger the judges that haven't been completed yet.
  • Rich Context: Hover over evaluator names to see tooltips and descriptions, providing clearer definitions of what each judge is measuring.

For more information on the qualitative analysis metrics, see Metrics Reference Glossary.