Adding Docker registry credentials and certificates

To enable Cloudera Machine Learning to fetch custom ML Runtimes from a secure repository, as an Administrator you need to add Docker registry credentials used to log into the registry. You may also need to add the Docker registry TLS certificate if you are using a custom or self-signed certificate for the registry.

Adding Docker Registry Credentials to Cloudera Machine Learning

Create a kubectl secret named regcred for your secured Docker registry. Create the secret in your Kubernetes cluster:

kubectl create secret docker-registry regcred 
    --docker-server=<server host> 
    --docker-username=<username> 
    --docker-password=<password> 
    -n <compute namespace eg. mlx>

Next time the ML Runtime image is pulled, the new secret will be picked up.

note

Consult the documentation of your chosen docker registry to understand what docker login credentials it expects. The expected credentials might be different from your own username and password. Many registries require docker authentication using tokens or API keys.

kubectl create secret docker-registry regcred 
    --docker-server=<server host> 
    --docker-username=AWS 
    --docker-password=$(aws ecr get-login-password --region <region>)
    -n <compute namespace eg. mlx>

Adding Docker Registry certificates to Cloudera Machine Learning

You may need to copy your registry's TLS certificates into Cloudera Machine Learning:

Pull the existing certificates:

kubectl get configmap private-cloud-ca-certs-pem -n [***CML workspace namespace***] -o jsonpath="{.binaryData.cacerts}" | base64 -d > /tmp/oldcerts.pem

Append your docker registry certificates to the existing certificates and convert them to base64:

cat /tmp/dockercerts.crt /tmp/oldcerts.pem > /tmp/newcerts.pem
cat /tmp/newcerts.pem | base64 -w 0 > /tmp/newcerts.b64

Pull the certificates' configuration map and update it with this new certificate file:

kubectl get configmap private-cloud-ca-certs-pem -n [***CML worspace namespace***] -o yaml > /tmp/oldconfigmap.yaml

cat /tmp/oldconfigmap.yaml | sed "s/cacerts.*/cacerts: $(cat /tmp/newcerts.b64)/" > /tmp/newconfigmap.yaml

kubectl apply -f /tmp/newconfigmap.yaml

Delete the runtime-manager pod to restart it and see these new certificates:

kubectl delete pod runtime-manager-xxxxxxx-xxxxx -n [***CML worspace namespace***]

Adding Docker registry certificates to Cloudera

If the Custom Runtime repository uses self-signed TLS certificates, those certificates must be trusted by Clouderaprior to creating the Cloudera Machine Learning Workspace. The certificates must be uploaded to Miscellaneous category, as uploading them to other categories might intervene into other processes. See the necessary steps to make Cloudera trust the self-signed certificates in Updating TLS certificates.