Open source LLMs are having a big moment. The number of open source text-gen models on HuggingFace has accelerated to a pace of nearly 10,000 new models a month. Models like Llama2 and Mixtral are rapidly being adopted and incorporated into production applications in gaming, writing, document Q&A, and many more.
As LLM-based applications scale to record numbers of users, promoting user safety becomes a bigger lift. As an AI inference provider, OctoAI sits in a unique place in the stack. Our platform serves billions of tokens from our hosted LLM endpoints (Llama2, Mixtral, Mistral) as well as custom models and fine-tunes that customers bring to the platform. This volume of traffic makes it especially challenging to ensure that inference traffic adheres to OctoAI’s Acceptable Use Policy and Terms of Service, and remains safe for our customers and their users.
That’s where LlamaGuard comes in.
OctoAI and LlamaGuard
LlamaGuard is an open source safeguard model from Meta geared towards Human-AI conversation use cases. Not only are OctoAI customers using LlamaGuard in production to detect unsafe prompts, queries, and chat traffic, OctoAI saw an opportunity to apply it to our own Trust and Safety efforts.
OctoAI’s Head of Trust and Safety, Mackenzie Tudor, in collaboration with product and engineering, is implementing a traffic sampling protocol as an additional layer of protection, detecting unsafe prompts that may have snuck past moderation and filters. This protocol allows us to automatically monitor a huge volume of logs for content violations while preserving user privacy. From there, we can flag offending accounts for our customers and provide actionable feedback on how their users may be attempting to skirt safeguards.
This is an invaluable tool in the ongoing effort to curb unsafe use of LLMs and further evolve practices for better protections. Mackenzie's expertise as head of Trust and Safety at Clubhouse has been invaluable in developing and implementing this system. We’ve added this to the robust measures already in place on in our Image Gen solution, which OctoAI customers are leveraging to keep harmful content out of their applications.
Try LlamaGuard on OctoAI today
Finally, we’re excited to announce that we've recently made LlamaGuard available to all users, along with a web UI that allows them to customize their own moderation parameters. With LlamaGuard, we're committed to providing a safe and responsible platform for our customers to keep build amazing things with LLMs.