Sign up
Log in
Sign up
Log in
Live Webinar: June 25th
Join our Builder's Roundtable to learn all about fine-tuning LLMs
Register now
Home
Blog

Latitude Games lowers costs by 5x and unlocks new game experiences with Mixtral on OctoAI

Blog Author - Deepak Mohan
Blog Author - Nick Walton

Feb 16, 2024

6 minutes
OctoAI and Latitude games logos with a video game scene from Dungeon AI

Latitude Games is an award winning game developer and the creator of AI Dungeon, a role playing game that delivers engaging large language model (LLM) powered interactive experiences to players. AI Dungeon brings the momentum and excitement of language model progress to gamers, and key to this is its rapid adoption of impactful new models - from the earliest GPT models, to AI21’s Jurassic-1 and Jurassic-2 models, to GPT-3.5 Turbo and GPT 4. With the launch of Mixtral from Mistral AI, the team received overwhelming interest from gamers to have Mixtral added to AI Dungeon.

Working with OctoAI, Latitude was able to evaluate the new Mixtral model within days of its release, and add it to AI Dungeon just 10 days after Mixtral’s initial announcement. Today, Mixtral on OctoAI is the default model for the hundreds of thousands of monthly active users on the platform and serves billions of tokens for AI Dungeon every day. The switch to Mixtral on OctoAI has also reduced Latitude’s inference costs by over 5x, enabling them to accelerate planned product enhancements and add new AI-powered experiences for players.

Significantly higher intelligence-to-cost ratios don’t just let us do existing things cheaper and better, they actually unlock entirely new applications and products that wouldn’t have been possible otherwise.

Nick Walton, CEO & Co-Founder @ Latitude Games

Read on to learn more about Latitude Games and their adoption of Mixtral on OctoAI. You can get started with Mixtral on OctoAI on the OctoAI Text Gen Solution today.

AI Dungeon brings the excitement of LLM progress to AI gaming

AI Dungeon delivers an interactive role playing game (RPG) experience to players, building on the power of leading-edge LLMs. Taking inspiration from games like Dungeons and Dragons, AI Dungeon is built to maximize the flexibility and creative freedom possible in the game environment.

While many AI-powered games attempt to abstract away the specifics of the generative AI foundation models and technology under the hood, AI Dungeon is transparent about these. Players get to choose the models that power their gaming experience. AI Dungeon has also been proactive in adding new models to the platform and to communicating these additions and enhancements to players. Since the games initial introduction in 2019, AI Dungeon has iteratively added and evolved its list of story generator models across several generations of open and closed-source GPT models from OpenAI and AI21. This transparency and speed have effectively acted as a bridge connecting end players to the excitement and progress in the world of language model evolution.

Prior to late 2023, AI Dungeon was primarily built on the early GPT models followed by proprietary models from OpenAI and AI21. Open source models had been evolving, but none had started approaching the quality and performance of GPT. This changed with Mixtral, which quickly established itself as a tipping point in the evolution of open source models. With the quality and flexibility demonstrated by Mixtral and the broader buzz among early adopters, there was an overwhelming level of interest in Mixtral from AI Dungeon players.

Early surveys by the Latitude Games team validated the strong demand for Mixtral, and the team was eager to quickly get the model into AI Dungeon and to players. Equally critical was to make sure it was a production grade service, ready for traffic and scaling, with predictable latency and availability.

Mixtral on OctoAI: Production grade Mixtral endpoint for AI Dungeon

OctoAI was one of the earliest platforms to make Mixtral available for application builders. The speedy inclusion of Mixtral 8x7B as a scalable, reliable and performant model on the platform was enabled by OctoAI’s deep ML systems expertise and technology capability through the GenAI stack. This was complemented by quick access to powerful NVIDIA H100 and NVIDIA A100 GPUs, through OctoAI’s deep cloud provider partnerships. Mixtral on OctoAI builds on the reliability, speed, and efficiencies of the OctoAI Text Gen Solution, delivering a cost-effective and easy-to-use API endpoint for LLM inferences.

Working with the OctoAI team, Latitude Games was able to validate that the reliability and latency possible on OctoAI was sufficient for their needs. The competitive pricing was an additional bonus, with initial estimates indicating that it could have an order of magnitude improvement to inference costs. With this assurance, the team introduced Mixtral to AI Dungeon as an early access offer for players. The adoption and response validated the demand, and user feedback continued to overwhelmingly highlight preference for Mixtral. Surveys and forum discussions indicated that over 75% of users preferred the Mixtral model as the promotional premium model available for free actions. With this validation and the demonstrated performance of the endpoint through the early access, the team moved Mixtral to general availability and out of early access. With this, Mixtral is also now the default model that new players start with and the promotional premium model available on the platform.

Higher intelligence to cost ratio unlocks new products and experiences

Significantly higher intelligence to cost ratios don’t just make existing things cheaper and better, they unlock entirely new applications and products that wouldn’t have been possible otherwise. Latitude has been at the cutting edge of AI and games since the first GPT-2 model was released, and in the process, constantly pushing the boundaries of how to build the AI games people dream of playing. The almost order of magnitude improvement in the intelligence to cost ratio unlocks a new type of game experience that the team currently have in development. The new capabilities use significantly more advanced AI systems to deliver more powerful experiences to players, and would not be possible without this improvement in costs.

As better open source models get deployed at lower prices, there will be more innovative products built and launched that wouldn’t have been possible with previous generations of AI technology. This ongoing shift in the intelligence to cost ratio will be key to unlocking future waves of products and capabilities powered by AI.

Timely inclusion of Mixtral, new product capabilities, and a 5x reduction in inference costs

Latitude Games was able to launch Mixtral into AI Dungeon within days of the initial announcement, delivering a new more flexible experience and tapping into the booming player demand for Mixtral.

“If it helps, I’m a new annual sub entirely because you recently added mixtral.”
“I’ve only been using Mixtral for about an hour now, but so far: SO MUCH BETTER than Dragon 2.1!”

-Selected Reddit user comments on Mixtral addition to AI Dungeon

In addition to the new experience and speed of inclusion, Mixtral on OctoAI’s lower costs have resulted in a 5x reduction in inference costs compared to the ChatGPT (GPT 4) model. This has also unlocked the addition of new and more demanding AI-powered experience on the platform. Today, Mixtral on OctoAI servers over a billion tokens every day to hundreds of thousands of players on AI Dungeon.

We wanted to quickly respond to the overwhelming demand for Mixtral from our players. At the same time, it was critical to deliver Mixtral with the latency and availability needed to ensure an engaging end-player experience. Working with the OctoAI team, we were able to quickly evaluate the new model, validate its performance through our proof of concept phase, and move the model to production. Mixtral on OctoAI serves a majority of the inferences and end player experiences on AI Dungeon today.

Nick Walton, CEO & Co-Founder @ Latitude Games

The LLM evolution is far from over, and Latitude Games and OctoAI are expanding our collaboration to evaluate new models and fine tunes. These efforts are crucial for the continued iteration of models and options on AI Dungeon, and for the platform to remain on the leading-edge of language model capabilities. We’ll share more about these models as they are ready!

Try the OctoAI Text Gen Solution today

You can start evaluating LLMs and building on OctoAI Text Gen Solution with a free trial today. If you’d like to engage with the broader OctoAI community and teams, please join our OctoAI Discord.

For customers currently using a closed source LLM like GPT-3.5 Turbo, GPT-4 or GPT-4 Turbo from OpenAI, OctoAI has a new promotion to help accelerate adoption of open source LLMs. You can read more about this in the OctoAI Model Remix Program introduction.

OUR BLOG

Related Posts

All Posts
2024-02-07
Reduce LLM costs for text summarization by over 50% with Mixtral on OctoAI

We show how switching to Mixtral on OctoAI can deliver a 1.5x to 3x lower LLM usage bill for your solution compared to GPT 3.5 on OpenAI and over an order of magnitude lower costs compared to GPT 4.

Blog Author - Alex Burlacu
Blog Author - Deepak Mohan
Alex Burlacu & Deepak Mohan
2024-01-23
OthersideAI achieves 12x reduction in LLM costs over OpenAI, with Mixtral on OctoAI

OctoAI serves production inferences supporting HyperWrite’s millions of customers, at 12x lower cost compared to OpenAI, comparable quality, and up to 30% lower latency.

Blog Author - Ben Hamm
Blog Author - Matt Shumer
Ben Hamm & Matt Shumer
2023-11-27
OctoAI Model Remix Program Shrinks Your Dependency on OpenAI

In the generative AI community, there’s a growing movement towards open source models in an effort to reduce dependence on OpenAI – and the recent corporate governance blow up is only a small part of why. Many OpenAI customers are finding it’s not the best-fit for every use-case.

Blog Author - Deepak Mohan
Deepak Mohan