Overview

Keeping our users’ data private and secure is our priority. OctoAI requires token authentication for all API requests, along with TLS to enforce encryption in transit for all connections between the customer and OctoAI. We also use encryption at rest for any data written to disk.

SecureLink is an additional private connectivity security measure, ensuring that network traffic between an OctoAI endpoint and the customer environment is not exposed to the public internet. SecureLink is available for Enterprise customers.

Setup Steps

1. Configure your AWS Account ID in OctoAI
2. Create and configure a VPC Interface Endpoint
3. Configure OctoAI’s SDKs & CLI to use the SecureLink subdomain
4. If you intend to use Asset Library to upload your assets, configure a separate PrivateLink connection for Amazon S3 to ensure the uploads are also completed via a private connection to S3

Setup Instructions

Configure your AWS Account ID in OctoAI

First, run a simple curl command to configure your AWS account ID within OctoAI using the https://api.octoai.cloud/v1/account/securelink endpoint:

curl -X POST "https://api.octoai.cloud/v1/account/securelink" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $OCTOAI_TOKEN" \
    --data '{"aws_account_id": "account-ID-value"}'

This allows OctoAI to generate a VPC Service Name, which you’ll use in the next step. If you don’t receive a successful response, contact us to ensure your OctoAI account is enabled as Enterprise tier.

Create and configure a VPC Interface Endpoint

Now you’ll create the VPC Interface Endpoint in your AWS account using the VPC Service Name. Navigate to the VPC Dashboard, and click Create Endpoint from your AWS console: https://console.aws.amazon.com/vpc/home?#Endpoints

Configure the Service Name value to com.amazonaws.vpce.us-east-1.vpce-svc-0e914445c09bbe700, then click Verify to ensure the service name is found and verified. Contact us for help if the service name is not found.

Next, choose the VPC and subnets that should be peered with the VPC service endpoint. Make sure that Enable DNS name is checked.

Then, choose the security group(s) who can send traffic to the VPC endpoint. The security group must accept inbound traffic on TCP port 443 - you can verify this within the Inbound Rules page. You can now click Create endpoint to create the VPC endpoint. The endpoint maybe take up to 10 minutes to move from Pending to Available. Once it shows Available, it’s ready for use.

Configure OctoAI’s SDKs & CLI to use SecureLink URL

Each OctoAI endpoint uses a SecureLink ingress URL, which will only work with a fully configured VPC Endpoint.

OctoAI CLI

Configure an environment variable by running:

export OCTO_API_ENDPOINT=https://api.securelink.octo.ai

TypeScript SDK

Configure the SecureLink URLs by passing in the SecureLink environment during client instantiation.

1
import { OctoAIClient, OctoAIEnvironment } from "@octoai/sdk";
2
3
const octoai = new OctoAIClient({
4
  apiKey: process.env.OCTOAI_TOKEN,
5
  environment: OctoAIEnvironment.SecureLink,
6
});

Python SDK

• For text generation, fine-tuning, or asset library, configure the environment parameter to use OctoAIEnvironment.SECURE_LINK in the client instantiation:

1
import os
2
3
from octoai.client import OctoAI
4
from octoai.environment import OctoAIEnvironment
5
token=os.environ.get("OCTOAI_TOKEN")
6
7
client = OctoAI(api_key=token, environment=OctoAIEnvironment.SECURE_LINK)

This table summarizes the SecureLink equivalent to each public API URL:

Configure private connection for Amazon S3 to upload assets through a private connection

If you are a user of Asset Library, you’ll need to configure a private connection for Amazon S3 to ensure uploads are also secured behind a private connection. Depending on your setup and your needs, you can either create a gateway endpoint, where a route table entry is added to your VPC, or create an interface endpoint, which is similar to configuring an interface endpoint for OctoAI. This guide covers setting up a gateway endpoint for S3.

To create a gateway endpoint, choose AWS services under Service category, and select com.amazonaws.us-east-1.s3. Ensure the type is Gateway.

Choose the route table where the routing entry is added, then click Create Endpoint. For more information, see the S3 gateway endpoint documentation on AWS.