QuickStart

Grant Access at the Tenancy Level

To allow a user group to manage all Generative AI resources across the tenancy:

allow group <your-group-name> to manage generative-ai-family 
in tenancy

Grant Access at the Compartment Level

To scope access to a specific compartment:

allow group <your-group-name> to manage generative-ai-family 
in compartment <your-compartment-name>

After these permissions are in place, you’re ready to create the first project.

Find the API Key OCID

To scope permissions to a specific API key, you need its OCID.

In the Console:

1. Navigate to the API Keys list page.
2. Select the API key you created.
3. Copy the OCID (it typically starts with ocid1.generativeaiapikey...)

Grant Permission to the API Key

Create an IAM policy to allow the API key to invoke the Responses API:

allow group <your-group-name> 
to manage generative-ai-response in tenancy where ALL 
{request.principal.type='generativeaiapikey', 
request.principal.id='<your-api-key-OCID>'}

This policy allows requests authenticated with the specified API key to access the Responses API, while keeping access scoped and controlled.

Base URL 🔗

Use the following base URL to access the OCI Responses API:

https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1

Replace <region> with the appropriate region identifier (for example, us-chicago-1).

SDK Support 🔗

The OCI Responses API is compatible with the OpenAI SDK, which is recommended for interacting with the service. It is supported across many languages, including Python, Java, TypeScript, Go, and .NET.

You can also use it with popular agent frameworks such as LangChain, LlamaIndex, and OpenAI Agents SDK.

When to Use IAM Authentication

Consider using IAM authentication when:

• Running applications in OCI services (for example, Functions or OKE)
• Avoiding long-lived credentials such as API keys
• Enforcing fine-grained access control through IAM policies

Install the OCI IAM Auth Library

Install the oci-genai-auth library, which provides helper utilities for integrating OCI IAM authentication with the OpenAI SDK:

pip install oci-genai-auth

This library includes the following authentication helpers:

• OciSessionAuth (for local development)
• OciUserPrincipalAuth
• OciInstancePrincipalAuth
• OciResourcePrincipalAuth (for OCI-managed environments)

Configure the OpenAI Client

When using IAM authentication, initialize the OpenAI client with a custom HTTP client and authentication handler. The api_key value is "not used" in this case.

Example: Local Development (OciSessionAuth)

Use this approach when running code locally (for example, on a laptop using an OCI CLI profile):

from openai import OpenAI
from oci_openai import OciSessionAuth
import httpx

client = OpenAI(
    base_url="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/openai/v1",  # update region if needed
    api_key="not-used",
    project="ocid1.generativeaiproject.oc1.us-chicago-1.xxxxxxxx",  # project OCID created earlier
    http_client=httpx.Client(auth=OciSessionAuth(profile_name="DEFAULT"))  # update profile if needed
)

response = client.responses.create(
    model="xai.grok-4-1-fast-reasoning",
    input="Write a one-sentence explanation of what a database is."
)

print(response.output_text)

Example: OCI Managed Environments (OciResourcePrincipalAuth)

Use this approach when running in OCI services such as OCI Functions or OCI Container Engine for Kubernetes (OKE):

If Using OCI IAM Authentication

If you added the policies in Prerequisite: Set Up IAM Permissions, skip this step.

To allow a user group to call the Responses API add the following policy:

allow group <your-group-name> 
to manage generative-ai-response in tenancy

If Using Generative AI API Key Authentication

When using API key authentication, another policy is required to authorize requests made with the API key. If you added the policies in 3. Add Permission to the API Key, skip this step.

To grant access to a specific API key:

allow group <your-group-name> 
to manage generative-ai-response in tenancy 
where ALL {request.principal.type='generativeaiapikey', 
request.principal.id='<your-api-key-OCID>'}

For broader access (for example, during testing), you can use a more general policy:

allow any-user 
to manage generative-ai-family in tenancy 
where ALL {request.principal.type='generativeaiapikey', 
request.principal.id='<your-api-key-OCID>'}

These policies ensure that requests, whether authenticated through IAM or API keys are authorized to access OCI Generative AI resources.

Third-Party Hosted Models

OCI Generative AI provides access to models from third-party providers. Specify the model using its fully qualified name:

response = client.responses.create(
    model="xai.grok-4-1-fast-reasoning",
    input="Write a one-sentence explanation of what a database is."
)

response = client.responses.create(
    model="google.gemini-2.5-pro",
    input="Write a one-sentence explanation of what a database is."
)

On-Demand Models

On-demand models are hosted and managed by OCI and are available without requiring dedicated infrastructure:

response = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Write a one-sentence explanation of what a database is."
)

Dedicated AI Clusters (Dedicated Mode)

For production workloads requiring isolation or predictable performance, you can deploy models on a dedicated AI cluster. In this case, use the cluster endpoint OCID as the model identifier:

response = client.responses.create(
    model="<dedicated-ai-cluster-endpoint-ocid>",
    input="Write a one-sentence explanation of what a database is."
)

Stream All Events

response_stream = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Explain the difference between structured and unstructured data.",
    stream=True
)

for event in response_stream:
    print(event)

Stream Only Text Output (Delta Tokens)

response_stream = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Explain the difference between structured and unstructured data.",
    stream=True
)

for event in response_stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

Streaming is especially useful for interactive applications where you want to display responses to users as they're generated.

Integrate with Observability Tools 🔗

For deeper insight such as latency, cost, and execution traces—you can integrate the OCI Responses API with observability platforms.

Many providers support OpenAI-compatible APIs. One such option is Langfuse, an open-source LLM engineering platform that helps developers debug, monitor, and improve LLM applications. It provides end-to-end observability for tracing agent actions, supports prompt versioning, and helps with evaluation of model outputs. Langfuse integrates with popular frameworks such as OpenAI, LangChain, and LlamaIndex.

The following example shows how to use Langfuse to trace and monitor Responses API calls.

Step 1: Install the Langfuse SDK

pip install langfuse

Step 2: Configure Environment Variables

Set the required Langfuse and OCI environment variables:

LANGFUSE_SECRET_KEY="sk-lf-xxxxxxxxx"
LANGFUSE_PUBLIC_KEY="pk-lf-xxxxxxxxx"
LANGFUSE_BASE_URL="https://us.cloud.langfuse.com"

# OCI Generative AI credentials
OCI_GENAI_API_KEY="sk-xxxxxxxxx"
OCI_GENAI_PROJECT_ID="ocid1.generativeaiproject.oc1.xxx"

Step 3: Instrument the OpenAI Client

Import the OpenAI client from the Langfuse SDK. Existing code remains unchanged, but requests are automatically traced.

import os
from langfuse.openai import OpenAI  # Import from Langfuse

client = OpenAI(
    base_url="https://inference.generativeai.us-ashburn-1.oci.oraclecloud.com/openai/v1",
    api_key=os.getenv("OCI_GENAI_API_KEY"),
    project=os.getenv("OCI_GENAI_PROJECT_ID"),
)

# Requests are automatically instrumented by Langfuse
response = client.responses.create(
    model="openai.gpt-oss-120b",
    tools=[
        {
            "type": "mcp",
            "server_label": "dmcp",
            "server_url": "https://mcp.deepwiki.com/mcp",
            "require_approval": "never",
        },
    ],
    input="Explain why tracing and observability are important in distributed systems."
)

print(response.output_text)

This integration provides end-to-end visibility into API calls without requiring significant changes to your application code.

Image Input as Base64-Encoded Data URL

import base64

def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode("utf-8")

base64_image = encode_image("/path/to/image.png")

response = client.responses.create(
    model="openai.gpt-oss-120b",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Describe the main objects in this image."},
                {
                    "type": "input_image",
                    "image_url": f"data:image/jpeg;base64,{base64_image}",
                    "detail": "high",
                },
            ],
        }
    ],
)

print(response.output_text)

Image Input as Internet-Accessible URL

response = client.responses.create(
    model="openai.gpt-oss-120b",
    store=False,
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Describe the scene shown in this image."},
                {
                    "type": "input_image",
                    "image_url": "https://example.photos/id/123",
                },
            ],
        }
    ],
)

print(response.output_text)

Replace the "image_url" with a valid image URL.

File Input as File ID

file = client.files.create(
    file=open("<path-to-file>", "rb"),
    purpose="user_data"
)

response = client.responses.create(
    model="google.gemini-2.5-pro",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_file",
                    "file_id": file.id,
                },
                {
                    "type": "input_text",
                    "text": "Summarize this document.",
                },
            ]
        }
    ]
)

print(response.output_text)

File Input as Internet-Accessible URL

response = client.responses.create(
    model="google.gemini-2.5-flash",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Summarize this file."},
                {
                    "type": "input_file",
                    "file_url": "https://www.example.com/letters/example-letter.pdf",
                },
            ],
        }
    ],
)

print(response.output_text)

Reasoning Effort

import json

response = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Solve 18 * (4 + 2).",
    reasoning={"effort": "medium"},
    store=False,
)

print(response.output_text)

Reasoning Summary Output

If you're building a chatbot, enabling reasoning summaries can help users better understand how the model arrived at a result. During streaming, users can also see reasoning tokens while the model is thinking.

import json

response = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Solve 18 * (4 + 2).",
    reasoning={"summary": "auto"},
    store=False,
)

print(response.output_text)

What This Enables

Function tools are useful when the application needs to stay in control of execution while still allowing the model to decide when outside information is needed.

Typical use cases include:

• Looking up calendar events
• Retrieving application data
• Calling internal or external APIs
• Running business logic or calculations

This approach gives you flexibility while keeping the execution path inside the application.

Execution Flow 🔗

A typical function tool interaction works like this:

1. The client sends a request that includes one or more tool definitions.
2. The model decides whether one of those tools is needed.
3. If a tool is needed, the model returns the tool name and arguments.
4. The application runs the tool and prepares the result.
5. The application sends that result back in a follow-up request.
6. The model uses that result to complete the response.

State Handling Options 🔗

There are two common ways to manage state across these requests:

• Service-managed state: Recommended for most use cases. The follow-up request includes previous_response_id, and the service tracks the earlier exchange.
• Client-managed state: The application keeps the entire interaction history and sends the accumulated context with each request.

Define a Function Tool

The following example defines a tool that retrieves calendar events for a specified date.

tools = [
    {
        "type": "function",
        "name": "get_calendar_events",
        "description": "Return calendar events scheduled for a specific date.",
        "parameters": {
            "type": "object",
            "properties": {
                "date": {
                    "type": "string",
                    "description": "Date to query, for example 2026-04-02"
                }
            },
            "required": ["date"],
        },
    },
]

Include this tools array in the client.responses.create() request.

Example: Service-Managed State

In this pattern, the first request allows the model to decide whether the tool is needed. The second request sends the tool result back and references the earlier response.

import json

tools = [
    {
        "type": "function",
        "name": "get_calendar_events",
        "description": "Return calendar events scheduled for a specific date.",
        "parameters": {
            "type": "object",
            "properties": {
                "date": {
                    "type": "string",
                    "description": "Date to query, for example 2026-04-02"
                }
            },
            "required": ["date"],
        },
    },
]

def get_calendar_events(date):
    # Replace this with actual calendar logic or an API call
    return [
        {"time": "09:00", "title": "Team standup"},
        {"time": "13:00", "title": "Design review"},
        {"time": "16:00", "title": "Project check-in"},
    ]

# Initial request
response = client.responses.create(
    model="openai.gpt-oss-120b",
    tools=tools,
    input="Show the calendar events for 2026-04-02.",
)

# Execute the requested function
tool_outputs = []
for item in response.output:
    if item.type == "function_call" and item.name == "get_calendar_events":
        args = json.loads(item.arguments)
        events = get_calendar_events(**args)
        tool_outputs.append({
            "type": "function_call_output",
            "call_id": item.call_id,
            "output": json.dumps({"events": events}),
        })

# Follow-up request
final = client.responses.create(
    model="openai.gpt-oss-120b",
    instructions="Summarize the schedule clearly for the user.",
    tools=tools,
    input=tool_outputs,
    previous_response_id=response.id,
)

print(final.output_text)

Example: Service-Managed State

In this pattern, the application keeps the full exchange and resubmits it with the follow-up request.

import json

tools = [
    {
        "type": "function",
        "name": "get_calendar_events",
        "description": "Return calendar events scheduled for a specific date.",
        "parameters": {
            "type": "object",
            "properties": {
                "date": {
                    "type": "string",
                    "description": "Date to query, for example 2026-04-02"
                }
            },
            "required": ["date"],
        },
    },
]

def get_calendar_events(date):
    # Replace this with actual calendar logic or an API call
    return [
        {"time": "09:00", "title": "Team standup"},
        {"time": "13:00", "title": "Design review"},
        {"time": "16:00", "title": "Project check-in"},
    ]

conversation = [
    {"role": "user", "content": "Show the calendar events for 2026-04-02."}
]

response = client.responses.create(
    model="openai.gpt-oss-120b",
    tools=tools,
    input=conversation,
)

conversation += response.output

for item in response.output:
    if item.type == "function_call" and item.name == "get_calendar_events":
        args = json.loads(item.arguments)
        events = get_calendar_events(**args)
        conversation.append({
            "type": "function_call_output",
            "call_id": item.call_id,
            "output": json.dumps({"events": events}),
        })

final = client.responses.create(
    model="openai.gpt-oss-120b",
    instructions="Summarize the schedule clearly for the user.",
    tools=tools,
    input=conversation,
)

print(final.output_text)

Function tools are a strong option when the application must remain responsible for execution, access control, and integration logic, while still allowing the model to request the information it needs.

When to Use the MCP Tool

Use the MCP tool when the model needs access to external capabilities that are hosted on an MCP server.

This approach is helpful when you want:

• The service to communicate with the MCP server directly
• Fewer client-side orchestration steps
• Lower latency than a client-executed tool pattern
• Access to capabilities exposed through a remote MCP server

The MCP tool is part of the OCI Generative AI toolset and can be used alongside other supported tools.

Transport Support

Remote MCP calling uses Streamable HTTP.

Define an MCP Tool

Add the MCP server definition in the tools field of the request.

response_stream = client.responses.create(
    model="xai.grok-code-fast-1",
    tools=[
        {
            "type": "mcp",
            "server_label": "calendar",
            "server_description": "An MCP server that retrieves calendar events for a specified date.",
            "server_url": "https://example.com/mcp",
            "require_approval": "never",
        },
    ],
    input="What events are scheduled for 2026-04-02?",
    stream=True,
)

for event in response_stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

This example streams the response and prints text as it is generated.

Restrict the Tools Exposed by the MCP Server

If the remote MCP server exposes more tools than the application needs, you can narrow the available set by using allowed_tools.

response_stream = client.responses.create(
    model="openai.gpt-oss-120b",
    tools=[
        {
            "type": "mcp",
            "server_label": "calendar",
            "server_description": "An MCP server that retrieves calendar events for a specified date.",
            "server_url": "https://example.com/mcp",
            "require_approval": "never",
            "allowed_tools": ["get_events"],
        },
    ],
    input="Show the calendar events for 2026-02-02.",
    stream=True,
    store=False,
)

Provide Authentication to the MCP Server

response_stream = client.responses.create(
    model="xai.grok-4-1-fast-reasoning",
    tools=[
        {
            "type": "mcp",
            "server_label": "calendar",
            "server_url": "https://calendar.example.com/mcp",
            "authorization": "$CALENDAR_OAUTH_ACCESS_TOKEN"
        },
    ],
    input="List my meetings for 2026-02-02.",
    stream=True,
    store=False,
)

Pass only the token value. Don't include the Bearer prefix. OCI sends the token over TLS as part of the request and does not decode, inspect, store, or log it.

MCP Server Hosting

OCI Generative AI also provides MCP server hosting to help deploy and scale MCP servers.

The MCP tool is a good option when you want OCI Generative AI to work directly with a remote MCP server during request execution, without requiring the client application to handle each tool step itself.

Prepare a Vector Store

Before using File Search, create a vector store and add the files you want the model to reference. OCI Generative AI supports several file-handling patterns through APIs that follow the OpenAI Files API style.

Example

To make File Search available in a request, add a tool entry with type: "file_search" and provide the vector store ID.

response = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Summarize the main ideas covered in the documents in this vector store.",
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": ["<vector_store_id>"]
        }
    ]
)

print(response)

What You Can Use It For

If you're using this for the first time, it helps to think of Code Interpreter as a temporary Python workspace for the model.

You can use it for tasks such as:

• solving math problems
• analyzing uploaded files
• cleaning or transforming data
• creating charts or tables
• generating output files such as logs, or processed datasets

Execution Environment

The Python environment includes more than 420 preinstalled libraries, so many common tasks work without extra setup.

The code runs inside a container. This container is the working environment where Python runs and where files are stored during the session.

Container Memory Limits

Code Interpreter containers use a shared memory pool of 64 GB per tenancy.

Supported container sizes are:

• 1 GB
• 4 GB
• 16 GB
• 64 GB

This shared limit can be divided across multiple containers. For example, it could support:

• sixty-four 1 GB containers
• sixteen 4 GB containers
• four 16 GB containers
• one 64 GB container

If you need more capacity, you can submit a service request.

Container Expiration

A container expires after 20 minutes of inactivity.

This is important to know when building multi-step flows:

• an expired container can't be reused
• you must create a new container
• files must be uploaded again if needed
• in-memory state, such as Python variables, is lost

Because of this, it's best to treat containers as temporary working environments.

Example

response = client.responses.create(
    model="xai.grok-4-1-fast-reasoning",
    tools=[
        {
            "type": "code_interpreter",
            "container": {"type": "auto"}
        }
    ],
    instructions="Use the python tool to solve the problem and explain the result.",
    input="Find the value of (18 / 3) + 7 * 2."
)

print(response.output_text)

Containers for Code Interpreter

Code Interpreter needs a container. The container is the isolated environment where the model runs Python code.

A container can hold:

• uploaded files
• files created by the model
• temporary working data during execution

When you use Code Interpreter, you can assign one of two container modes:

• auto: OCI Generative AI creates the container for you and automatically assigns a container size.
• conatiner OCID: You create the container yourself and define the container size and provide the OCID.

For both options, the containers are created and managed in OCI Generative AI. The code that runs in those containers also runs in the OCI Generative AI tenancy.

Auto Mode

In auto mode, the service creates the container for you. This is the easiest option and a good starting point for most users.

Use auto mode when:

• you want OCI Generative AI to manage the container
• you do not need direct control over the environment
• you want a simpler setup

response = client.responses.create(
    model="xai.grok-code-fast-1",
    tools=[{
        "type": "code_interpreter",
        "container": {
            "type": "auto"
        }
    }],
    input="Use the python tool to calculate the average of 12, 18, 24, and 30."
)

Explicit Mode

In explicit mode, you create the container first and set the size. Then you pass the container ID in the request.

Use explicit mode when you want more control over the container specifics such as the memory size.

container = client.containers.create(name="test-container", memory_limit="4g")

response = client.responses.create(
    model="xai.grok-code-fast-1",
    tools=[{
        "type": "code_interpreter",
        "container": container.id
    }],
    tool_choice="required",
    input="Use the python tool to calculate the average of 12, 18, 24, and 30."
)

print(response.output_text)

Files in Code Interpreter

Code Interpreter can work with files during the life of the container. The model can read files you provide and can also create new files.

This is useful for workflows such as:

• reading a CSV or PDF
• generating a chart
• saving processed output
• creating logs or reports

File Persistence 🔗

Files created or changed by the python tool stay available in the same container as long as the container has not expired.

This means the model can build on earlier work in the same session. For example, it can:

1. read a file
2. analyze it
3. save a chart
4. use that chart later in the same container

When the container expires, that state is no longer available.

Uploading and Managing Files 🔗

You can manage container files through the container file APIs.

Common operations include:

• Create Container File: Add a file to the container
• List Container Files: View files in the container
• Delete Container File: Remove a file
• Retrieve Container File Content: Download a file from the container

This lets you use the container as a temporary workspace for model-driven code execution.

Output Files and Citations 🔗

When the model creates files, those files are stored in the container and can be referenced in the response.

These references include:

• container_id
• file_id
• filename

You can use these values to retrieve the generated file content.

The OCI Responses API supports OpenAI-compatible endpoints for features such as Responses, Files, Containers, and Container Files. Because of that, the related OpenAI documentation can be used as a reference for request structure, response formats, and general workflows. However, when using these APIs with OCI, send requests to the OCI Generative AI inference endpoints, use OCI authentication, and note that the resources and execution remain in OCI Generative AI, not in an OpenAI tenancy.

Prerequisites

Before using the NL2SQL tool, first create a database and configure the required database connections.

For more information, see:

• Clone an Autonomous AI Database Instance
• Using the Oracle Cloud Infrastructure Console

Permissions for Semantic Stores

Access to Secrets

Grant the group access to read the secrets used by Database Tools:

allow group <your-group-name> 
to read secret-family in compartment <your-compartment-name> 
where all {request.principal.type='generativeaisemanticstore'}

Access to Database Tools Connections

Grant the group access to the required Database Tools resources:

allow group <your-group-name> 
to use database-tools-family in compartment <compartment-name> 
where all {request.principal.type='generativeaisemanticstore'}

allow group <your-group-name> 
to read database-family in compartment <compartment-name> 
where all {request.principal.type='generativeaisemanticstore'}

allow group <your-group-name> 
to read autonomous-database-family in compartment <compartment-name> 
where all {request.principal.type='generativeaisemanticstore'}

Access to Generative AI Resources

If the broader policy below is already in place, it includes access to semantic store resources:

allow group <your-group-name> 
to manage generative-ai-family in tenancy

If you want narrower access, use the following policy instead to allow the group to create and manage semantic stores:

allow group <your-group-name> 
to manage generative-ai-semantic-store 
in compartment <your-compartment-name>

If the group only needs to use an existing semantic store and call NL2SQL, use these more specific policies:

allow group <your-group-name> 
to use generative-ai-semantic-store 
in compartment <your-compartment-name>

allow group <your-group-name> 
to manage generative-ai-nl2sql 
in compartment <your-compartment-name>

Responses Chaining

With responses chaining, each new response points to the previous one. This is a simple option when you want to carry context forward without explicitly creating a conversation resource.

# first turn
response1 = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Give me three ideas for a team offsite.",
)
print("Response 1:", response1.output_text)

# second turn
response2 = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Make the second idea more budget friendly.",
    previous_response_id=response1.id,
)
print("Response 2:", response2.output_text)

Conversations API

# create a conversation
conversation = client.conversations.create(
    metadata={"topic": "demo"}
)
print("Conversation ID:", conversation.id)

# first turn
response1 = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Give me three ideas for a team offsite.",
    conversation=conversation.id,
)
print("Response 1:", response1.output_text)

# second turn
response2 = client.responses.create(
    model="openai.gpt-oss-120b",
    input="Make the second idea more budget friendly.",
    conversation=conversation.id,
)
print("Response 2:", response2.output_text)