Cloudera Docs

Connecting to a Named Livy Session

You can connect a JDBC client to a specific, existing Livy session by referencing its name in the JDBC connection string. This allows you to create a persistent session and connect to it from multiple clients or reconnect after a client application has been restarted.

The Livy Named Sessions feature is useful for sharing temporary data, like a global temporary view, across different client connections.

  1. Create a named Livy session using the REST API by using cURL to send a REST API request to the Livy server to create a new Spark session with a specific name.

    In this example, persistent-bi-session is the name assigned to the session.

    LIVY_URL=http://[*** Livy Thrift Server Host ***]:8998
curl -X POST --data '{"kind": "spark", "name": "persistent-bi-session"}' -H "Content-Type: application/json" $LIVY_URL/sessions
  2. Create a Global Temporary View in the named Livy session by creating a DataFrame and register it as a global temporary view. 
    curl -X POST --data '{"code": "val df1 = spark.sparkContext.parallelize(Seq((1, \"John\", \"Doe\", 1986))).toDF(\"id\", \"first\", \"last\", \"year\")"}' -H "Content-Type: application/json" $LIVY_URL/sessions/name/persistent-bi-session/statements

curl -X POST --data '{"code": "df1.createGlobalTempView(\"mytempview\")"}' -H "Content-Type: application/json" $LIVY_URL/sessions/name/persistent-bi-session/statements
  3. Connect to the existing session through Beeline by using the Beeline JDBC client to connect to the Livy Thrift Server and passing the session name in the livy.session.name property in the connection URL. 
    beeline -u "jdbc:hive2://[*** Livy Thrift Server Host ***]:10090/?livy.session.name=persistent-bi-session" -n [*** username ***]
  4. Query the created global temporary view that is accessible within the Spark session by using the following command: 
    0: jdbc:hive2://[*** ... ***]> SELECT * FROM global_temp.mytempview;