Customizing Tool Execution Environment

The execution environment can be customized, for tools that need specialized access beyond the default sandbox.

This is achieved by including a config.json file within the tool's directory. This configuration allows developers to:

Mount specific directories, such as those from /home/cdsw or other authorized paths.
Inject additional environment variables sourced from the host system.
Define new custom environment variables.
Increase the execution timeout for operations that require more time.

Using the config.json file

The config.json file must be utilized when your tool requires the following specialized configurations:

Accessing Specific Project Subdirectories: Mount a subdirectory from /home/cdsw/my-project that is not included in the default workflow_data area.
Defining Additional Environment Variables: Access host environment variables that are typically filtered out by the execution environment.
Specifying Custom Configuration Paths: Mount configuration directories from non-standard locations, such as /opt or /etc.
Extending Execution Time: Set a longer timeout for tools that execute long-running operations.

Create a config.json file in your tool directory with the following structure:

{
  "sandbox": {
    "timeout": 1800,
    "mounts": [
      {
        "host_path": "/home/cdsw/my-project/custom-data",
        "sandbox_path": "/home/cdsw/my-project/custom-data",
        "read_only": true,
        "required": false
      }
    ],
    "environment": {
      "variables": {
        "MY_TOOL_CONFIG": "/opt/my-tool/config",
        "MY_TOOL_LOG_LEVEL": "DEBUG"
      }
    }
  }
}

Accessing Directories from /home/cdsw

To access the /home/cdsw directory within your CML project that is not automatically included in the standard workflow_data mount, you must explicitly mount that directory for your tool.

Example: Mounting a custom data directory

{
  "sandbox": {
    "mounts": [
      {
        "host_path": "/home/cdsw/my-project/custom-models",
        "sandbox_path": "/home/cdsw/my-project/custom-models",
        "read_only": true,
        "required": false
      }
    ]
  }
}

Tool code to use the mounted directory:

import os

def run_tool(config, args):
    # ✅ Access the mounted directory
    models_dir = "/home/cdsw/my-project/custom-models"
    
    # List available model files
    if os.path.exists(models_dir):
        model_files = os.listdir(models_dir)
        print(f"Available models: {model_files}")
        
        # Load a specific model
        model_path = os.path.join(models_dir, "my-model.pkl")
        if os.path.exists(model_path):
            # Load and use the model
            pass

Setting Custom Environment Variables

Custom environment variables can be set that are specific to your tool.

Example: Custom configuration

{
  "sandbox": {
    "environment": {
      "variables": {
        "MY_TOOL_CONFIG_PATH": "/opt/my-tool/config",
        "MY_TOOL_LOG_LEVEL": "DEBUG",
        "MY_TOOL_CACHE_DIR": "/opt/my-tool/cache"
      }
    }
  }
}

Tool code to use custom environment variables:

import os

def run_tool(config, args):
    # ✅ Access custom environment variables
    config_path = os.environ.get('MY_TOOL_CONFIG_PATH')
    log_level = os.environ.get('MY_TOOL_LOG_LEVEL', 'INFO')
    cache_dir = os.environ.get('MY_TOOL_CACHE_DIR')
    
    print(f"Config path: {config_path}")
    print(f"Log level: {log_level}")

Security Restrictions

All mounts are strictly read-only, irrespective of any read_only: false configuration setting.
The following path prefixes are authorized: /usr, /lib, /bin, /sbin, /opt, /var, /sys, /etc, /home/cdsw, /runtime-addons, and /tmp. Only paths beginning with these prefixes are permitted.
Access to the following directories is restricted: /root,/proc, /dev, /home/* (with the exception of /home/cdsw).