Integrations

LangChain Integration

Langchain developers can leverage OctoAI LLM and embedding endpoints to easily access efficient compute across a wide selection of LLMs.

Introduction

LangChain provides a framework to easily build LLM-powered apps. Developers using LangChain can now utilize OctoAI LLMs and Embedding endpoints to access efficient, fast, and reliable compute.

Using OctoAI’s LLMs and LangChain

To use OctoAI LLMs with LangChain, first install the following dependencies in your environment:

$ pip install langchain langchain-community openai

Secondly, obtain an OctoAI API Token. Then paste your API token and run the code example below:

1 import os
2 
3 os.environ["OCTOAI_API_TOKEN"] = "OCTOAI_API_TOKEN"
4 os.environ["ENDPOINT_URL"] = "https://text.octoai.run/v1/chat/completions"
5 
6 from langchain.chains import LLMChain
7 from langchain_community.llms.octoai_endpoint import OctoAIEndpoint
8 from langchain_core.prompts import PromptTemplate
9 
10 template = """Below is an instruction that describes a task. Write a response that appropriately completes the request.\n Instruction:\n{question}\n Response: """
11 prompt = PromptTemplate.from_template(template)
12 
13 llm = OctoAIEndpoint(
14     model="meta-llama-3-8b-instruct",
15     max_tokens=128,
16     presence_penalty=0,
17     temperature=0.1,
18     top_p=0.9,
19 )
20 
21 question = "Who was leonardo davinci?"
22 
23 llm_chain = LLMChain(prompt=prompt, llm=llm)
24 
25 print(llm_chain.run(question))

It should produce the following output:

  Sure thing! Here's my response:
Leonardo da Vinci was a true Renaissance man - an Italian polymath who excelled in various fields,
including painting, sculpture, engineering, mathematics, anatomy, and geology. He is widely considered
one of the greatest painters of all time, and his inventive and innovative works continue to inspire and
influence artists and thinkers to this day. Some of his most famous works include the Mona Lisa,
The Last Supper, and Vitruvian Man.

Using OctoAI’s Embeddings and LangChain

Using OctoAI’s Embeddings endpoint is easy with LangChain.

We require the following dependencies:

$ pip install langchain langchain-community openai transformers

And now you can run the code example below:

1 from langchain_community.embeddings.octoai_embeddings import OctoAIEmbeddings
2 
3 embeddings = OctoAIEmbeddings()
4 text = "This is a test query."
5 query_result = embeddings.embed_query(text)
6 print(query_result)

Learn with our demo apps

Get started today by following along with one of our demo apps:

1	import os
2
3	os.environ["OCTOAI_API_TOKEN"] = "OCTOAI_API_TOKEN"
4	os.environ["ENDPOINT_URL"] = "https://text.octoai.run/v1/chat/completions"
5
6	from langchain.chains import LLMChain
7	from langchain_community.llms.octoai_endpoint import OctoAIEndpoint
8	from langchain_core.prompts import PromptTemplate
9
10	template = """Below is an instruction that describes a task. Write a response that appropriately completes the request.\n Instruction:\n{question}\n Response: """
11	prompt = PromptTemplate.from_template(template)
12
13	llm = OctoAIEndpoint(
14	model="meta-llama-3-8b-instruct",
15	max_tokens=128,
16	presence_penalty=0,
17	temperature=0.1,
18	top_p=0.9,
19	)
20
21	question = "Who was leonardo davinci?"
22
23	llm_chain = LLMChain(prompt=prompt, llm=llm)
24
25	print(llm_chain.run(question))

1	from langchain_community.embeddings.octoai_embeddings import OctoAIEmbeddings
2
3	embeddings = OctoAIEmbeddings()
4	text = "This is a test query."
5	query_result = embeddings.embed_query(text)
6	print(query_result)