Launch Workers

Launches worker engines into the cluster.

Syntax

launch_workers(n, cpu, memory, nvidia_gpu=0, kernel="python3", script="", code="", env={})

Parameters

n (int) - The number of engines to launch.
cpu (float) - The number of CPU cores to allocate to the engine.
memory (float) - The number of gigabytes of memory to allocate to the engine.
nvidia_gpu (int, optional) - The number of GPU's to allocate to the engine.
kernel (str, optional) - The kernel. Can be "r", "python2", "python3" or "scala".
script (str, optional) - The name of a Python source file the worker should run as soon as it starts up.
code (str, optional) - Python code the engine should run as soon as it starts up. If a script is specified, code will be ignored.
env (dict, optional) - Environment variables to set in the engine.

Example Usage

Python

import cdsw
workers = cdsw.launch_workers(n=2, cpu=0.2, memory=0.5, code="print('Hello from a CDSW Worker')")

library("cdsw") 
workers <- launch.workers(n=2, cpu=0.2, memory=0.5, env="", code="print('Hello from a CDSW Worker')")