PBJ Runtimes and Models
The PBJ (Powered by Jupyter) Runtime enables a wide variety of language kernels to be run as Cloudera AI workloads. Model workloads are currently only supported for Python and R kernels.
The library, cml, is added to Runtime based Python and R workloads
automatically and includes mostly the same functionality as the old cdsw
library.
In non-PBJ Runtimes, you could point to a Python or R function and run it as a model. PBJ
Runtimes open up the possibilities for users to create their own Runtimes and places few
restrictions on how to do that. PBJ technology utilizes the Jupyter
kernelgateway to communicate with language kernels and aims to be language
independent in the future.
Migrating from the CDSW to Cloudera Machine Learning library
PBJ Runtimes currently includes two supported languages, Python and R. There are new function wrappers or decorators, described below, that must be used in order to allow previously written model code to function in a PBJ Runtime. It is important to note that these support decorators or wrappers are transparent, that is, they have no effect, in non-PBJ workloads and non-model workloads.
| CDSW | Cloudera AI |
|---|---|
call_model |
models_v1.call_model |
| - | models_v1.cml_model |
get_auth |
utils_v1.get_auth |
track_file |
Removed |
model_metrics |
Replaced by models_v1.cml_model(metrics=True) |
read_metrics |
metrics_v1.read_metrics |
track_metric |
metrics_v1.track_metric |
track_delayed_metrics |
metrics_v1.track_delayed_metrics |
track_aggregate_metrics |
metrics_v1.track_aggregate_metrics |
launch_workers |
workers_v1.launch_workers |
list_workers |
workers_v1.list_workers |
stop_workers |
workers_v1.stop_workers |
await_workers |
workers_v1.await_workers |
| CDSW | Cloudera AI |
|---|---|
html |
Planned for future release |
iframe |
Planned for future release |
markdown |
Planned for future release |
display.help |
Planned for future release |
code |
Planned for future release |
image |
Planned for future release |
get.auth |
get.auth |
launch.workers |
launch.workers |
stop.workers |
stop.workers |
list.workers |
list.workers |
await.workers |
await.workers |
track.metric |
Removed |
track.file |
Removed |
| - | cml_model |
Library availability
In Cloudera AI, The cdsw and s libraries are available in different
runtime and engine types.
| Runtime / Engine type | Library availablility |
|---|---|
| Legacy Engine | cdsw |
| Classic Runtime | cdsw, cml |
| PBJ Runtime | cml |
