Cloudera AI Inference cannot discover model registry

Learn about scenarios or issues that may be resolved by using the steps in the Manually Updating Model Registry Configuration solution mentioned below.

ML Serving Installed Before Model Registry
If the model registry has not been created yet, when ML Serving is installed it will not be able to generate the ConfigMap.

Model Registry Upgraded After ML Serving Installed
When the Model Registry is upgraded, it will require an updated ConfigMap.

Model Registry Certificate Expired
After 90 days, Model Registry will have a certificate update, it will require an updated ConfigMap.

Manually Updating Model Registry Configuration

If you upgrade the CML Registry after creating your Cloudera AI Inference service cluster, the model registry configuration stored by the AI Inference service will get out of sync. Follow the steps below to manually reconcile the configuration, so that AI Inference service will be able to connect to the model registry.

Get KubeConfig for ML Serving Cluster.
1. In a terminal where CDP CLI is installed, run the cdp ml list-ml-serving-apps command.
2. Find the appCrn of the ML Serving instance you wish to update.
3. Retrieve the KubeConfig with this command:
```
cdp ml get-ml-serving-app-kubeconfig --app-crn <YOUR-APP-CRN> |jq .kubeconfig|yq --prettyPrint
```
  Here:
  - cdp ml get-ml-serving-app-kubeconfig --app-crn <YOUR-APP-CRN> : returns raw json response containing kubeconfig.
  - |jq .kubeconfig: returns only the actual kubeconfig (removes json wrapper).
  - |yq --prettyPrint: prints the kubeconfig response into a format both machine and user-readable.
4. Direct the output to the standard kubeconfig location:
```
cdp ml get-ml-serving-app-kubeconfig --app-crn <YOUR-APP-CRN> |jq .kubeconfig|yq --prettyPrint>~/.kube/config
```
  Make sure this is the active kubeconfig by running the kubectl config view command. The output of the above command should now shows the kubeconfig data returned above.
Grant your user ML Serving App Access
Add your cloud provider user identifier to the ML Serving App access control list. Use the following command, with your ARN, or other cloud provider user identifier.
```
cdp ml grant-ml-serving-app-access --resource-crn crn:cdp:ml:us-west-1:9d74eee4-1cad-45d7-b645-7ccf9edbb73d:mlserving:2c982e74-c60e-43b9-be85-357cffb1d419 --identifier <Your Cloud Provider User ID>
```
Get New ConfigMap Data
1. Find the new model registry ConfigMap. Use the grant-model-registry-access API in CDP CLI to add your user name to the new model registry, or use the UI, as shown:
2. After your user ARN has been granted access to the Model Registry, get the ConfigMap data in the following way:
  - Download the Model Registry Kubeconfig
  - Set the Model Registry Kubeconfig (can save it to ~/.kube/config)
  - kubectl get cm -n mlx: lists all the available ConfigMaps
  - kubectl describe cm jwks-rootca -n mlx: Returns the TLS Certificate for Model Registry
  - cdp ml list-model-registries: The response will contain the domain for the updated Model Registry
3. Copy the entire rootca.pem output from the command
```
kubectl describe cm jwks-rootca -n mlx 
```
  The domain of the new model registry is contained in the list-model-registries response:
Apply ConfigMap Update
1. Update the KUBECONFIG back to the ML Serving kubeconfig.
2. Edit the ConfigMap of the ML Serving Cluster:
  - kubectl edit cm modelregistry-config-controlplane -n serving: Update tls.crt with the data from above.
  - kubectl describe cm api-config -n serving: Update model.registry.url with the data from above
3. Restart the deployment to force the cm changes to take effect
  - kubectl scale deployment api -n serving --replicas=0
  - kubectl scale deployment api -n serving --replicas=1