Configuring model request payload size
Model metrics have a configuration that restricts model request payload to 100 KB. You can increase the payload size if required.
-
Convert the payload size to bytes.
For example, 100 KB (Kilobytes) = 100 * 1024 bytes = 102400 bytes.
-
Encode the values into the Base64 format.
For example, 20 MB in Bytes is 20000000, to convert it to Base64 value, run the following command on the terminal:
echo -n "20000000" | base64
The output would be
MjAwMDAwMDA=.
Here,MjAwMDAwMDA=
is the Base64 encoded value. -
Edit the existing Secret object to specify the model request
payload size.
kubectl edit secret model-metrics-config -n mlx
This opens your default editor and allows you to update the Base64 encoded values.
-
Locate the
max.request.size.bytes
field and update it with the Base64 encoded value.max.request.size.bytes: [***Base64-encoded-value***]
For example, substitute the above 20 MB Base64 encoded value:max.request.size.bytes: MjAwMDAwMDA=
- Save and close the editor.
-
Restart the model metrics pod by deleting the pod.
kubectl delete pod modelmetrics [***pod nam-ext***] -n mlx
This will force Kubernetes to restart the pod with the updated configuration.