Cloudera Data Science Workbench Release Notes

For a list of the current known issues and limitations in Cloudera Data Science Workbench 1.3.x, see Known Issues and Limitations in Cloudera Data Science Workbench 1.3.x.

• Added support for SUSE Linux Enterprise Server 12 SP3.

• Site administrators can now add template projects that are customized for their organization's use-cases.

• Version 1.3 introduces a new environment variable for Python 3 sessions called PYSPARK3_PYTHON. Python 2 sessions will continue to use the default PYSPARK_PYTHON variable. This will allow you to configure distinct variables for Python 2 and Python 3 applications.

• In the Cloudera Manager CDSW service, the Wildcard DNS Domain property has been renamed to Cloudera Data Science Workbench Domain.

• Output for the cdsw version command now includes the type of deployment you are running – RPM or CSD.

• Added log4j and spark-defaults sample configuration to the PySpark and Scala template projects.

• Fixed an issue where the cdsw status command failed to run all the required system checks.

  Cloudera Bug: DSE-3070

• Session lists now include additional metadata to help distinguish between different sessions.

  Cloudera Bug: DSE-2814

• Pre-install validation checks have been improved to detect issues with iptables modules and Java settings.

  Cloudera Bug: DSE-2293

• Fixed an issue with the cdsw status command output when TLS is enabled.

  Cloudera Bug: DSE-3182

• Cloudera Distribution of Spark 2.2 Release 2 fixes the issue where a PySpark application could only be run once per active Workbench session.

  Cloudera Bug: CDH-58475

• Fixed an issue that prevented Bokeh plots from rendering.

  Cloudera Bug: DSE-3134

• Fixed an issue in Cloudera Data Science Workbench 1.2.2 that prevented WebSocket re-connections and caused console hangs.

  Cloudera Bug: DSE-3085

• Improved CDSW service restart performance for CSD deployments.

  Cloudera Bug: DSE-2937

For a list of the current known issues and limitations in Cloudera Data Science Workbench 1.3.x, see Known Issues and Limitations in Cloudera Data Science Workbench 1.3.x.

For a list of known issues and limitations, refer the documentation for version 1.2.x at Cloudera Data Science Workbench 1.2.x.

Upgrading from Cloudera Data Science Workbench 1.2.0 to 1.2.1 on CSD-based deployments

After upgrading from Cloudera Data Science Workbench 1.2.0 to 1.2.1 on a CSD-based deployment, CLI commands might not work as expected due to missing binaries in the environment. Note that this issue does not affect fresh installs.

For a list of known issues and limitations, refer the documentation for version 1.2.x at Cloudera Data Science Workbench 1.2.x.

• Cloudera Data Science Workbench is now available as an add-on service for Cloudera Manager. To this end, Cloudera Data Science Workbench is now distributed in a parcel that integrates with Cloudera Manager using a Custom Service Descriptor (CSD). You can use Cloudera Manager to install, upgrade, and monitor Cloudera Data Science Workbench. Diagnostic data bundles can be generated and submitted to Cloudera through Cloudera Manager.
• Cloudera Data Science Workbench now enables secure sharing of job and session consoles. Additionally, site administrators can disable anonymous sharing from the Site Administrator dashboard (Admin > Security). See Sharing Job and Session Console Outputs.
• The Admin > Usage page now includes graphs for monitoring usage activity such as number of CPUs or GPUs used, memory usage, and total session runs, over customizable periods of time. See Monitoring Site Activity.
• Cloudera Data Science Workbench now lets you configure session, job, and idle timeouts. These can be configured using environmental variables either for the entire deployment or per-project.
• The cdsw enable and disable commands are no longer needed. The master node will now automatically detect the IP addresses of worker nodes joining or leaving Cloudera Data Science Workbench. See the revised Cloudera Data Science Workbench Command Line Reference.
• The Kudu Python client is now included in the Cloudera Data Science Workbench base engine image.
• Interactive session names can now be modified by project contributors and admins. By default, session names are set to 'Untitled Session'.
• All-numeric usernames are now accepted.
• Kubernetes has been upgraded to version 1.6.11.

Proxy Configuration Change: If you are using a proxy server, you must ensure that the IP addresses for the web and Livelog services are skipped from the proxy.

100.77.0.129
100.77.0.130

These have also been added to the installation instructions.

For a list of known issues and limitations, refer the documentation for version 1.2.x at Cloudera Data Science Workbench 1.2.x.

• Keytab Authentication - With version 1.1.1, you can now authenticate yourself to the CDH cluster by uploading your Kerberos keytab to Cloudera Data Science Workbench. To use this feature, go to the top-right dropdown menu, click Account settings > Hadoop Authentication, enter your Kerberos principal and click Upload Keytab.

For a list of known issues and limitations, refer the documentation for version 1.1.x at Cloudera Data Science Workbench 1.1.x.

• Added support for RHEL/CentOS 7.3 and Oracle Linux 7.3.

• Cloudera Data Science Workbench now allows you to run GPU-based workloads. For more details, see Using GPUs for Cloudera Data Science Workbench Workloads.

• For Cloudera Manager and CDH clusters that are not connected to the Internet, Cloudera Data Science Workbench now supports fully offline installations. See the installation guide for more details.

• Web UIs for processing frameworks such as Spark 2, Tensorflow, and Shiny, are now embedded in Cloudera Data Science Workbench and can be accessed directly from active sessions and jobs. For more details, see Accessing Web User Interfaces from Cloudera Data Science Workbench.

• Added support for a Jobs REST API that lets you orchestrate jobs from 3rd party workflow tools. See Cloudera Data Science Workbench Jobs API.

• DataFrames are now scrollable in the workbench session output pane. For examples, see the section on Grid Displays.

• Added support for rich visualizations in Scala engine using Jupyter jvm-repr. For an example, see HTML Visualizations - Scala.

• JAVA_HOME is now set in cdsw.conf, and not from the Site Administrator dashboard (Admin > Engines).

• Improved support for dynamic data visualizations in Python, including Bokeh.

• Fixed issues with the Python template project. The project now supports offline mode and will therefore work on airgapped clusters.

• Fixed issues related to cached responses in Internet Explorer 11.

• Fixed issues with Java symlinks outside of JAVA_HOME.

• The cdsw status command can now be run on worker nodes.

• Removed unauthenticated localhost access to Kubernetes.

• Fixed Kerberos authentication issues with specific enc-types and Active Directory.

• Removed restrictions on usernames with special characters for better compatibility with external authentication systems such as Active Directory.

• Fixed issues with LDAP configuration validation that caused application crashes.

• Improved LDAP test configuration form to avoid confusion on parameters being sent.

• Upgrading from version 1.0.x to 1.1.x

  During the upgrade process, you will encounter incompatibilities between the two versions of cdsw.conf. This is because even though you are installing the latest RPM, your previous configuration settings in cdsw.conf will remain unchanged. Depending on the release you are upgrading from, you will need to modify cdsw.conf to ensure it passes the validation checks run by the 1.1.x release.

  Key changes to note:

  • JAVA_HOME is now a required parameter. Make sure you add JAVA_HOME to cdsw.conf before you start Cloudera Data Science Workbench.
  • Previous versions allowed MASTER_IP to be set to a DNS hostname. If you are still using a DNS hostname, switch to an IP address.
• Python engine updated in version 1.1.x

  Version 1.1.x includes an updated base engine image for Python which no longer uses the deprecated pylab mode in Jupyter to import the numpy and matplotlib functions into the global scope. With version 1.1.x, engines will now use built-in functions like any rather than the pylab counterpart, numpy.any. As a result of this change, you might see certain behavioral changes and differences in results between the two versions.

  Also note that Python projects originally created with engine 1 will be running pandas version 0.19, and will not auto-upgrade to version 0.20 by simply selecting engine 2. You will also need to manually install version 0.20.1 of pandas when you launch a project session.

For a list of known issues and limitations, refer the documentation for version 1.1.x at Cloudera Data Science Workbench 1.1.x.

• Fixed a random port conflict that could prevent Scala engines from running.

• Improved formatting of validation, and visibility of some errors.

• Fixed an issue with Firefox that was resulting in duplicate jobs on job creation.

• Removed the Mathjax external dependency on CDN.

• Improved PATH and JAVA_HOME handling that previously broke Hadoop CLIs.

• Fixed an issue with Java security policy files that caused Kerberos issues.

• Fixed an issue that caused git clone to fail on some repositories.

• Fixed an issue where updating LDAP admin settings deactivated the local fallback login.

• Fixed an issue where bad LDAP configuration crashed the application.

• Fixed an issue where job environmental variable settings did not persist.

For a list of known issues and limitations, refer the documentation for version 1.0.x at Cloudera Data Science Workbench 1.0.x.