Skip to main content
When building LLM applications, you sometimes need direct control over prompt compilation without routing through the AI Gateway. The SDK provides an alternative integration method that allows you to pull and compile prompts directly in your application.

​
SDK vs AI Gateway

We provide SDKs for both TypeScript and Python that offer two ways to use Helicone prompts:
  1. AI Gateway Integration - Use prompts through the Helicone AI Gateway (recommended)
  2. Direct SDK Integration - Pull prompts directly via SDK (this page)
Prompts through the AI Gateway come with several benefits:
  • Cleaner code: Automatically performs compilation and substitution in the router.
  • Input traces: Traces inputs on each request for better observability in Helicone requests.
  • Faster TTFT: The AI Gateway adds significantly less latency compared to the SDK.
The SDK is a great option for users that need direct interaction with compiled prompt bodies without using the AI Gateway.

​
Installation

  • TypeScript
  • Python
npm install @helicone/helpers

​
Types and Classes

  • TypeScript
  • Python
The SDK provides types for both integration methods when using the OpenAI SDK:
TypeDescriptionUse Case
HeliconeChatCreateParamsStandard chat completions with promptsNon-streaming requests
HeliconeChatCreateParamsStreamingStreaming chat completions with promptsStreaming requests
Both types extend the OpenAI SDK’s chat completion parameters and add:
  • prompt_id - Your saved prompt identifier
  • environment - Optional environment to target (e.g., “production”, “staging”)
  • version_id - Optional specific version (defaults to production version)
  • inputs - Variable values
Important: These types make messages optional because Helicone prompts are expected to contain the required message structure. If your prompt template is empty or doesn’t include messages, you’ll need to provide them at runtime.For direct SDK integration:
import { HeliconePromptManager } from '@helicone/helpers';  const promptManager = new HeliconePromptManager({  apiKey: "your-helicone-api-key" });

​
Methods

Both SDKs provide the HeliconePromptManager with these main methods:
MethodDescriptionReturns
pullPromptVersion()Determine which prompt version to usePrompt version object
pullPromptBody()Fetch raw prompt from storageRaw prompt body
pullPromptBodyByVersionId()Fetch prompt by specific version IDRaw prompt body
mergePromptBody()Merge prompt with inputs and validationCompilation result
getPromptBody()Complete compile process with inputsCompiled body + validation errors
extractPromptPartials()Extract prompt partial references from prompt bodyArray of prompt partial objects
getPromptPartialSubstitutionValue()Get the content to substitute for a prompt partialSubstitution string

​
Usage Examples

  • TypeScript
  • Python
import OpenAI from 'openai'; import { HeliconePromptManager } from '@helicone/helpers';  const openai = new OpenAI({  baseURL: "https://ai-gateway.helicone.ai",  apiKey: process.env.HELICONE_API_KEY, });  const promptManager = new HeliconePromptManager({  apiKey: "your-helicone-api-key" });  async function generateWithPrompt() {  // Get compiled prompt with variable substitution  const { body, errors } = await promptManager.getPromptBody({  prompt_id: "abc123",  model: "gpt-4o-mini",  inputs: {  customer_name: "Alice Johnson",  product: "AI Gateway"  }  });   // Check for validation errors  if (errors.length > 0) {  console.warn("Validation errors:", errors);  }   // Use compiled prompt with OpenAI SDK  const response = await openai.chat.completions.create(body);  console.log(response.choices[0].message.content); }
Both approaches are fully compatible with all OpenAI SDK features including function calling, response formats, and advanced parameters. The HeliconePromptManager, while not providing input traces, will provide validation error handling.

​
Handling Prompt Partials

Prompt partials allow you to reference messages from other prompts using the syntax {{hcp:prompt_id:index:environment}}. This enables code reuse across your prompt library.

​
AI Gateway vs SDK

When using the AI Gateway, prompt partials are automatically resolved - you don’t need to do anything special: 
import OpenAI from 'openai';  const openai = new OpenAI({  baseURL: "https://ai-gateway.helicone.ai",  apiKey: process.env.HELICONE_API_KEY, });  // Partials like {{hcp:abc123:0}} are automatically resolved! const response = await openai.chat.completions.create({  model: "gpt-4o-mini",  prompt_id: "xyz789", // This prompt may contain partials  inputs: {  user_name: "Alice"  } });
When using the SDK directly, you must manually resolve prompt partials by fetching and substituting the referenced prompts:
  • TypeScript
  • Python
Manual Prompt Partial Resolution
import OpenAI from 'openai'; import { HeliconePromptManager } from '@helicone/helpers';  const openai = new OpenAI({  apiKey: process.env.OPENAI_API_KEY, });  const promptManager = new HeliconePromptManager({  apiKey: "your-helicone-api-key" });  async function generateWithPromptPartials() {  // Step 1: Fetch the main prompt body  const mainPromptBody = await promptManager.pullPromptBody({  prompt_id: "xyz789"  });   // Step 2: Extract all prompt partial references  const promptPartials = promptManager.extractPromptPartials(mainPromptBody);   // Step 3: Fetch and resolve each prompt partial  const promptPartialInputs: Record<string, string> = {};   for (const partial of promptPartials) {  // Fetch the referenced prompt's body  const partialBody = await promptManager.pullPromptBody({  prompt_id: partial.prompt_id,  environment: partial.environment || "production"  });   // Extract the specific message content  const substitutionValue = promptManager.getPromptPartialSubstitutionValue(  partial,  partialBody  );   // Map the template tag to its resolved content  promptPartialInputs[partial.raw] = substitutionValue;  }   // Step 4: Merge the prompt with inputs and resolved partials  const { body, errors } = await promptManager.mergePromptBody(  {  prompt_id: "xyz789",  model: "gpt-4o-mini",  inputs: {  user_name: "Alice"  }  },  mainPromptBody,  promptPartialInputs // Pass resolved partials  );   if (errors.length > 0) {  console.warn("Validation errors:", errors);  }   // Step 5: Use the compiled prompt  const response = await openai.chat.completions.create(body);  console.log(response.choices[0].message.content); }

​
Understanding Prompt Partial Syntax

Prompt partials use the format {{hcp:prompt_id:index:environment}}:
  • prompt_id - The 6-character alphanumeric identifier of the prompt to reference
  • index - The message index (0-based) to extract from that prompt
  • environment - Optional environment identifier (defaults to production)
Examples: 
{{hcp:abc123:0}} // Message 0 from prompt abc123 (production) {{hcp:abc123:1:staging}} // Message 1 from prompt abc123 (staging) {{hcp:xyz789:2:development}} // Message 2 from prompt xyz789 (development)
If your prompts don’t contain any prompt partials (no {{hcp:...}} tags), you don’t need to worry about this section. The SDK will work normally without any special handling.
When using the SDK directly, each prompt partial requires a separate API call to fetch the referenced prompt. For prompts with many partials, consider using the AI Gateway instead for better performance and automatic caching.

Overview

Get started with Prompt Management

Prompt Assembly

Understand how prompts are compiled

AI Gateway

Use prompts through the AI Gateway (recommended)