Inference models

Serverless Endpoints

OctoAI currently supports the self-service models & checkpoints organized on this page, and we’ll continue to expand our models and services. Ready to run your first inference? Navigate to our Quickstart guide to get started.

Text Gen Models

OrganizationUse CasesModel NameAPI Model StringContext Length
MetaChat, CodingLlama-3.1-Instruct (8B)meta-llama-3.1-8b-instruct131,072
MetaChat, CodingLlama-3.1-Instruct (70B)meta-llama-3.1-70b-instruct131,072
MetaChat, CodingLlama-3.1-Instruct (405B)meta-llama-3.1-405b-instruct131,072
MetaChatLlama3-Instruct (70B)meta-llama-3-70b-instruct8,192
MistralChat, CodingMistral Instruct v0.3 (7B)mistral-7b-instruct32,768
MistralChat, CodingMistral Nemo Instruct (12B)mistral-nemo-instruct65,536
MistralChat, CodingMixtral Instruct (8x7B)mixtral-8x7b-instruct32,768
Nous ResearchContent ModerationNous Hermes 2 Mixtral DPO (8x7B)nous-hermes-2-mixtral-8x7b-dpo32,768
MicrosoftChat, CodingWizardLM-2 (8x22B)wizardlm-2-8x22b65,536
MicrosoftImage UnderstandingPhi-3.5-Vision (4B)phi-3.5-vision-instructn/a
MetaContent ModerationLlama Guard 2llamaguard-2-7b4,096
Alibaba DAMOEmbeddingGTE Largethenlper/gte-largen/a

Check out our REST API, Python SDK, or TypeScript SDK docs when you’re ready to use text gen models programmatically.

Media Gen Models

ServiceModelAPI Model String
Image GenStable Diffusion XL v1.0sdxl
Image GenControlNet SDXLcontrolnet-sdxl
Image AnimationStable Video Diffusion v1.1svd
Background RemovalIS-Netbackground-removal
UpscalingREAL-ESRGAN x4 Plusreal-esrgan-x4-plus
UpscalingREAL-ESRGAN x4 v3real-esrgan-x4-v3
UpscalingREAL-ESRGAN x4 v3 WDNreal-esrgan-x4-v3-wdn
UpscalingREAL-ESRGAN Anime Video v3real-esrgan-animevideo-v3
UpscalingREAL-ESRGAN x4 Plus Animereal-esrgan-x4-plus-anime
UpscalingREAL-ESRGAN x2 Plusreal-esrgan-x2-plus
AdetailerFace YOLOv8nface_yolov8n
AdetailerHand YOLOv8nhand_yolov8n
AdetailerFace Full MediaPipeface_full_mediapipe
AdetailerFace Short MediaPipeface_short_mediapipe
AdetailerFace Mesh MediaPipeface_mesh_mediapipe
AdetailerEyes Mesh MediaPipeeyes_mesh_mediapipe

Check out our Image Gen API and Video Gen API docs when you’re ready to use media gen models programmatically. You can also easily upload and run custom checkpoints and assets using OctoAI’s Asset Library.