Cloudera Docs

Create Azure Files Storage Account and File Share

Azure Files NFS v4.1 is a managed, POSIX compliant NFS service on Azure. The file share is used to store files for the CML infrastructure and ML workspaces. This is the recommended NFS service for use with CML.

Consult the Azure Files for NFS documentation, below, for detailed information on how to create an NFS share. We outline the sequence of steps here.

  1. Create a FileStorage storage account.
  2. Disable secure transfer.
  3. See Create a private endpoint, below, to create a private endpoint between the FileStorage storage account for each of the subnets that you plan to use for creating the CML workspace. Without this step, CML workspace nodes will not have connectivity to the NFS server.
  4. Create NFS file shares, one for each ML Workspace. Create each file share with an initial capacity of at least 100 GB. Ensure that the “No Root Squash” export policy is set. This is the default setting.
  5. Create a Linux VM in the subnet in which you are going to create the CML workspace, and ensure that you can successfully mount the NFS share in the VM. This step will ensure that there are no connectivity issues to the NFS server from hosts in this subnet.
  6. Create a directory within the NFS share to be used as the CML workspace’s projects folder. For example, if the NFS export path is ‘azurenfsv4.file.core.windows.net:/myshare’, and is mounted as ‘/mnt/myshare’, run ‘sudo mkdir /mnt/myshare/ml-workspace-1; sudo chown 8536:8536 /mnt/myshare/ml-workspace-1’ to set ownership to the ‘cdsw’ user ID and group ID.
  7. Go to the mount instructions page to copy the server name and export path details. Enter this into the CML control plane when prompted for an existing server. Due to a known issue in the control plane UI, the “nfs://” prefix must be specified if you are using the domain name of the file share server, e.g., “nfs://azurenfsv4.file.core.windows.net:/myshare/ml-workspace-1”. Remember to select the NFS protocol version as 4.1.
  8. The Linux VM created above can be deleted if there are no issues with mounting the NFS share.