Limitations

Experiments have the following limitations.

• Experiments do not store snapshots of project files. You cannot automatically restore code that was run as part of an experiment.

• Experiments will fail if your project filesystem is too large for the Git snapshot process. As a general rule, any project files (code, generated model artifacts, dependencies, etc.) larger than 50 MB must be part of your project's .gitignore file so that they are not included in snapshots for experiment builds.

• Experiments cannot be deleted. As a result, be conscious of how you use the track_metrics and track_file functions.

  • Do not track files larger than 50MB.
  • Do not track more than 100 metrics per experiment. Excessive metric calls from an experiment may cause Cloudera Data Science Workbench to stop responding.

• The Experiments table will allow you to display only three metrics at a time. You can select which metrics are displayed from the metrics dropdown. If you are tracking a large number of metrics (100 or more), you might notice some performance lag in the UI.

• Arguments are not supported with Scala experiments.

• The track_metrics and track_file functions are not supported with Scala experiments.

• The UI does not display a confirmation when you start an experiment or any alerts when experiments fail.

• There is no API (v1 or v2) available to interact with the Experiments feature.