Supported models and servers in Cloudera AI Inference service

Hugging Face model server

Triton Inference server

NVIDIA NIM models

Hugging Face supported model architectures

Text generation models (139)

afmoeforcausallm, apertusforcausallm, aquilaforcausallm, aquilamodel, arceeforcausallm, arcticforcausallm, axk1forcausallm, baichuanforcausallm, bailingmoeforcausallm, bailingmoev2_5forcausallm, bailingmoev2forcausallm, bambaforcausallm, bloomforcausallm, chatglmforconditionalgeneration, chatglmmodel, cohere2forcausallm, cohereforcausallm, cwmforcausallm, dbrxforcausallm, decilmforcausallm, deepseekforcausallm, deepseekv2forcausallm, deepseekv32forcausallm, deepseekv3forcausallm, deepseekv4forcausallm, dots1forcausallm, ernie4_5_moeforcausallm, ernie4_5forcausallm, exaone4forcausallm, exaoneforcausallm, exaonemoeforcausallm, fairseq2llamaforcausallm, falconforcausallm, falconh1forcausallm, falconmambaforcausallm, flexolmoforcausallm, gemma2forcausallm, gemma3forcausallm, gemma3nforcausallm, gemma4forcausallm, gemmaforcausallm, glm4forcausallm, glm4moeforcausallm, glm4moeliteforcausallm, glmforcausallm, glmmoedsaforcausallm, gpt2lmheadmodel, gptbigcodeforcausallm, gptjforcausallm, gptneoxforcausallm, gptossforcausallm, graniteforcausallm, granitemoeforcausallm, granitemoehybridforcausallm, granitemoesharedforcausallm, gritlm, grok1forcausallm, grok1modelforcausallm, hcxvisionforcausallm, hcxvisionv2forcausallm, hunyuandensev1forcausallm, hunyuanmoev1forcausallm, hyperclovaxforcausallm, hyv3forcausallm, internlm2forcausallm, internlm2veforcausallm, internlm3forcausallm, internlmforcausallm, iquestcoderforcausallm, iquestloopcoderforcausallm, jais2forcausallm, jaislmheadmodel, jambaforcausallm, kimilinearforcausallm, lfm2forcausallm, lfm2moeforcausallm, llama4forcausallm, llamaforcausallm, longcatflashforcausallm, mamba2forcausallm, mambaforcausallm, mimoforcausallm, mimov2flashforcausallm, minicpm3forcausallm, minicpmforcausallm, minimaxforcausallm, minimaxm1forcausallm, minimaxm2forcausallm, minimaxtext01forcausallm, ministral3forcausallm, mistralforcausallm, mistrallarge3forcausallm, mixtralforcausallm, mptforcausallm, nemotronforcausallm, nemotronhforcausallm, nemotronhpuzzleforcausallm, olmo2forcausallm, olmo3forcausallm, olmoeforcausallm, olmoforcausallm, olmohybridforcausallm, optforcausallm, orionforcausallm, ouroforcausallm, panguembeddedforcausallm, pangupromoev2forcausallm, panguultramoeforcausallm, param2moeforcausallm, persimmonforcausallm, phi3forcausallm, phiforcausallm, phimoeforcausallm, plamo2forcausallm, plamo3forcausallm, qwen2forcausallm, qwen2moeforcausallm, qwen3forcausallm, qwen3moeforcausallm, qwen3nextforcausallm, qwenlmheadmodel, rnj1forcausallm, rwforcausallm, sarvammlaforcausallm, sarvammoeforcausallm, seedossforcausallm, solarforcausallm, stablelmepochforcausallm, stablelmforcausallm, starcoder2forcausallm, step1forcausallm, step3p5forcausallm, step3textforcausallm, telechat2forcausallm, telechat3forcausallm, telechatforcausallm, teleflmforcausallm, xverseforcausallm, zamba2forcausallm

Embedding models (11)

bertmodel, bertspladesparseembeddingmodel, bgem3embeddingmodel, erniemodel, gemma2model, gemma3textmodel, gtemodel, gtenewmodel, jinaembeddingsv5model, llamabidirectionalmodel, llamamodel

Late interaction models (11)

colbertjinarobertamodel, colbertlfm2model, colbertmodernbertmodel, colmodernvbertforretrieval, colpaliforretrieval, colqwen3, colqwen3_5, hf_colbert, jinaforranking, opscolqwen3model, qwen3vlnemotronembedmodel

Reward models (3)

internlm2forrewardmodel, qwen2forprocessrewardmodel, qwen2forrewardmodel

Token classification models (4)

bertfortokenclassification, erniefortokenclassification, modernbertfortokenclassification, qwen3asrforcedalignerfortokenclassification

Sequence classification models (11)

bertforsequenceclassification, ernieforsequenceclassification, gpt2forsequenceclassification, gtenewforsequenceclassification, jambaforsequenceclassification, jinavlforranking, llamabidirectionalforsequenceclassification, llamanemotronvlforsequenceclassification, modernbertforsequenceclassification, robertaforsequenceclassification, xlmrobertaforsequenceclassification

Multimodal models (106)

ariaforconditionalgeneration, audioflamingo3forconditionalgeneration, ayavisionforconditionalgeneration, bagelforconditionalgeneration, beeforconditionalgeneration, blip2forconditionalgeneration, chameleonforconditionalgeneration, cheers, cheersforconditionalgeneration, cohere2visionforconditionalgeneration, cohereasrforconditionalgeneration, deepseekocr2forcausallm, deepseekocrforcausallm, deepseekvlv2forcausallm, dotsocrforcausallm, eagle2_5_vlforconditionalgeneration, ernie4_5_vlmoeforconditionalgeneration, exaone4_5_forconditionalgeneration, fireredasr2forconditionalgeneration, fireredlidforconditionalgeneration, funasrforconditionalgeneration, funaudiochatforconditionalgeneration, fuyuforcausallm, gemma3forconditionalgeneration, gemma3nforconditionalgeneration, gemma4forconditionalgeneration, glm4vforcausallm, glm4vforconditionalgeneration, glm4vmoeforconditionalgeneration, glmasrforconditionalgeneration, glmocrforconditionalgeneration, granite4visionforconditionalgeneration, granitespeechforconditionalgeneration, h2ovlchatmodel, hunyuanvlforconditionalgeneration, idefics3forconditionalgeneration, interns1forconditionalgeneration, interns1proforconditionalgeneration, internvlchatmodel, internvlforconditionalgeneration, isaacforconditionalgeneration, kananavforconditionalgeneration, keyeforconditionalgeneration, keyevl1_5forconditionalgeneration, kimiaudioforconditionalgeneration, kimik25forconditionalgeneration, kimivlforconditionalgeneration, lfm2vlforconditionalgeneration, lightonocrforconditionalgeneration, llama4forconditionalgeneration, llama_nemotron_nano_vl, llavaforconditionalgeneration, llavanextforconditionalgeneration, llavanextvideoforconditionalgeneration, llavaonevisionforconditionalgeneration, mantisforconditionalgeneration, midashenglmmodel, minicpmo, minicpmv, minimaxvl01forconditionalgeneration, mistral3forconditionalgeneration, molmo2forconditionalgeneration, molmoforcausallm, moonshotkimiaforcausallm, musicflamingoforconditionalgeneration, nemotronh_nano_omni_reasoning_v3, nemotronh_nano_vl_v2, nemotronh_super_omni_reasoning_v3, nemotronparseforconditionalgeneration, nvlm_d, opencuaforconditionalgeneration, openpanguvlforconditionalgeneration, ovis, ovis2_5, ovis2_6_moeforcausallm, ovis2_6forcausallm, paddleocrvlforconditionalgeneration, paligemmaforconditionalgeneration, phi3vforcausallm, phi4forcausallmv, phi4mmforcausallm, pixtralforconditionalgeneration, qwen2_5_vlforconditionalgeneration, qwen2_5omniforconditionalgeneration, qwen2_5omnimodel, qwen2audioforconditionalgeneration, qwen2vlforconditionalgeneration, qwen3_5forconditionalgeneration, qwen3_5moeforconditionalgeneration, qwen3asrforconditionalgeneration, qwen3asrrealtimegeneration, qwen3omnimoeforconditionalgeneration, qwen3vlforconditionalgeneration, qwen3vlmoeforconditionalgeneration, qwenvlforconditionalgeneration, rforconditionalgeneration, skyworkr1vchatmodel, smolvlmforconditionalgeneration, step3vlforconditionalgeneration, stepvlforconditionalgeneration, tarsier2forconditionalgeneration, tarsierforconditionalgeneration, ultravoxmodel, voxtralforconditionalgeneration, voxtralrealtimegeneration, whisperforconditionalgeneration

Speculative Decoding (33)

deepseekmtpmodel, deepseekv4mtpmodel, dflashdraftmodel, eagle3deepseekv2forcausallm, eagle3deepseekv3forcausallm, eagle3llamaforcausallm, eagle3minimaxm2forcausallm, eagle3qwen2_5vlforcausallm, eagle3qwen3vlforcausallm, eagledeepseekmtpmodel, eaglellama4forcausallm, eaglellamaforcausallm, eagleminicpmforcausallm, eaglemistrallarge3forcausallm, erniemtpmodel, exaone4_5_mtp, exaonemoemtp, extracthiddenstatesmodel, glm4moelitemtpmodel, glm4moemtpmodel, glmocrmtpmodel, hyv3mtpmodel, llamaforcausallmeagle3, longcatflashmtpmodel, medusamodel, mimomtpmodel, mlpspeculatorpretrainedmodel, nemotronhmtpmodel, openpangumtpmodel, qwen3_5moemtp, qwen3_5mtp, qwen3nextmtp, step3p5mtp

Transformers supported models (2)

emu3forconditionalgeneration, smollm3forcausallm

Transformers Backend models (10)

transformersembeddingmodel, transformersforcausallm, transformersforsequenceclassification, transformersmoeembeddingmodel, transformersmoeforcausallm, transformersmoeforsequenceclassification, transformersmultimodalembeddingmodel, transformersmultimodalforcausallm, transformersmultimodalforsequenceclassification, transformersmultimodalmoeforcausallm