Keeping up with the latest in text gen & what's new with OctoAI
Yesterday, we introduced three new text-generation models to the OctoAI platform:
Gemma-7B-Instruct , a new open source model from Google
Smaug-72B-v0.1, a fine tune of the Qwen1.0 family of models that shows impressive leaderboard performance; and
Nous-Hermes-2-Mixtral-8x7B-DPO, a fine tune of the powerful Mixtral-8x7b model, offered as the “flagship” checkpoint of Nous Research (the current best producer of open source fine tunes for popular models).
Users can now run each of these models and evaluate their quality and capabilities directly in OctoAI.
OctoAI customers tell us consistently that keeping up with the latest innovations in the LLM market feels like a full-time job. The data bears this out — upwards of 8,000 new text-generation models were added to HuggingFace in the last few weeks alone.
Because it wouldn’t be practical (or possible) to evaluate each one, part of our work at OctoAI is curating and optimizing the most promising new models for our customers to evaluate for their applications. We use customer/community feedback (Nous Hermes 2 Mixtral), benchmarking data (Smaug 72B), and excitement over new entrants in the OSS model market (Gemma) to inform our product decisions.
Let's get experimental
You may have noticed that there’s a new section in OctoAI Text Gen Solution. Both Gemma 7B and Smaug 72B currently carry the “experimental” label. The new designation allows the OctoAI team to more quickly add interesting new LLMs to the platform for customer experimentation before all the data is in on use-case applicability, quality, and overall performance. Keep an eye on this section to try out some of the latest text-gen models, and don’t forget about our Core Models, which are tried and tested for production use. If you find one of our experimental models works well for your use case, let us know and we can provide a stable, latency-optimized endpoint for you to use!
What's next?
In addition to new curated models coming over the next few weeks, OctoAI is building new capabilities for serving fine-tuned models, running LLMs locally, and expanding technology partnerships with other providers in the independent LLM stack. Stay in touch with us on Discord, X, or LinkedIn.