Sessions example for the CDE CLI
In this example, a Session is created using the Cloudera Data Engineering (CDE) CLI.
> cde session create --name cli-submission-6 --type spark-scala --conf "key=value" > cde session list +-------------------+-----------+-------------+-------------+----------------------+----------------------+-------------------------+ | NAME | STATE | TYPE | DESCRIPTION | CREATED | LAST UPDATED | CREATOR | +-------------------+-----------+-------------+-------------+----------------------+----------------------+-------------------------+ | cli-submission-1 | preparing | spark-scala | | 2023-02-15T08:47:41Z | 2023-02-15T08:47:41Z | csso_surya.balakrishnan | | cli-submission-19 | preparing | spark-scala | | 2023-02-15T07:50:08Z | 2023-02-15T07:50:08Z | csso_surya.balakrishnan | +-------------------+-----------+-------------+-------------+----------------------+----------------------+-------------------------+ > cde session describe --name techf-run-1 { "name": "techf-run-1", "state": "idle", "type": "pyspark", "created": "2023-02-15T13:19:22Z", "lastStateUpdated": "2023-02-15T13:19:43Z", "creator": "csso_surya.balakrishnan", "interactiveSpark": { "submitID": "95", "kind": "pyspark", "driverMemory": "1g", "driverCores": 1, "executorMemory": "1g", "executorCores": 1, "numExecutors": 1, "conf": { "spark.kubernetes.driver.annotation.sidecar.istio.io/inject": "false", "spark.kubernetes.executor.annotation.sidecar.istio.io/inject": "false", "spark.pyspark.python": "python3" } } } > cde session connect --name tech-forum Starting REPL... Connected to Cloudera Data engineering... Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 3.3.0 /_/ Using Scala version 2.12.15 (OpenJDK 64-Bit Server VM, Java 11.0.16.1) Type in expressions to have them evaluated. >>> spark.sql("show tables").show() +--------+---------+-----------+ |database|tableName|isTemporary| +--------+---------+-----------+ +--------+---------+-----------+ >>> testDF = spark.createDataFrame([(1,"111"), (2,"111"), (3,"222"), (4,"222"), (5,"222"), (6,"111"), (7,"333"), (8,"444")], ["id", "d_id"]) >>> ^C > cde session delete --name tech-forum