---
title: "Call on-premises models on ARX through the API"
description: "Get the model API endpoint, API key, and model ID, then call an on-premises model with cURL or the Python SDK in either OpenAI or Claude API format"
canonical_url: "https://docs.ardge.com/en/knowledge/arx-os/api-model-calls"
---
# Call on-premises models on ARX through the API

> Get the model API endpoint, API key, and model ID, then call an on-premises model with cURL or the Python SDK in either OpenAI or Claude API format

## Overview

ARX runs on a server with GPU resources, so you can deploy models quickly and give your generative AI applications local inference.

The ARX model gateway speaks two formats: the OpenAI API and the Claude API. Your applications keep using the mainstream API format they already use, which also makes switching between cloud and on-premises models straightforward.

This article covers where to find the three values a call needs, with examples in cURL and Python.

## What you'll learn

- Get the model API endpoint, API key, and model ID
- Tell the OpenAI and Claude API formats apart
- Call an on-premises model with cURL and the Python SDKs

**Estimated time: about 10 minutes**

## Applies to

- ARX OS 1.0 and later

## Before you begin

- You have deployed a model in AI Model Center, and its status on the **Model Instances** page reads **Running**

If you haven't deployed one yet, see [AI Model Center](https://docs.ardge.com/en/guide/use-ai/ai-model-center) for the download and deploy walkthrough.

## What you'll need

A call needs these three values:

<table>
<thead>
  <tr>
    <th>
      Value
    </th>
    
    <th>
      What it is
    </th>
    
    <th>
      Where to find it
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <strong>
        Model API Endpoint
      </strong>
    </td>
    
    <td>
      The URL of the ARX model gateway
    </td>
    
    <td>
      The <strong>
        Home
      </strong>
      
       page in AI Model Center, or <strong>
        API Key › API Endpoint
      </strong>
    </td>
  </tr>
  
  <tr>
    <td>
      <strong>
        API Key
      </strong>
    </td>
    
    <td>
      The credential that authenticates the call
    </td>
    
    <td>
      The <strong>
        API Key
      </strong>
      
       page in AI Model Center — you create it yourself
    </td>
  </tr>
  
  <tr>
    <td>
      <strong>
        Model ID
      </strong>
    </td>
    
    <td>
      The identifier you pass in the request's <code>
        model
      </code>
      
       field
    </td>
    
    <td>
      The <strong>
        Home
      </strong>
      
       page, <strong>
        API Key › API Endpoint
      </strong>
      
      , or <strong>
        Model Instances
      </strong>
    </td>
  </tr>
</tbody>
</table>

## Step 1: Get the three values

### Model API endpoint

Look on the **Home** page in AI Model Center, or on **API Key › API Endpoint**.

<note>

**<API_ENDPOINT> already includes /ai/v1:**

The ARX interface shows an endpoint that ends with `/ai/v1`, for example `http://192.168.1.100/ai/v1`. Each example appends its own path to it.

</note>

### API key

1. Go to the **API Key** page in AI Model Center
2. Click **Add API Key**
3. Follow the fields to finish the setup — you can set an expiry date for the key and choose which models it can reach
4. Copy the generated key

<tip>

**API keys double as access control:**

The set of models you pick at creation time is that key's permission boundary. Create a separate key per application so that revoking one doesn't affect the others.

</tip>

### Model ID

The **Home** page, the **API Key › API Endpoint** page, and the **Model Instances** page all show the model's **Instance Name**. That string is the model ID you pass in the request's `model` field.

## How the two API formats differ

The same endpoint and key work with both formats. Here's what changes:

<table>
<thead>
  <tr>
    <th>
      
    </th>
    
    <th>
      OpenAI API format
    </th>
    
    <th>
      Claude API format
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Request path
    </td>
    
    <td>
      <code>
        <API_ENDPOINT>/chat/completions
      </code>
    </td>
    
    <td>
      <code>
        <API_ENDPOINT>/messages
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Auth header
    </td>
    
    <td>
      <code>
        Authorization: Bearer <YOUR_API_KEY>
      </code>
    </td>
    
    <td>
      <code>
        x-api-key: <YOUR_API_KEY>
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        max_tokens
      </code>
    </td>
    
    <td>
      Optional
    </td>
    
    <td>
      Required
    </td>
  </tr>
  
  <tr>
    <td>
      Where the system prompt goes
    </td>
    
    <td>
      An item with <code>
        role
      </code>
      
       <code>
        system
      </code>
      
       in the <code>
        messages
      </code>
      
       array
    </td>
    
    <td>
      A separate <code>
        system
      </code>
      
       field
    </td>
  </tr>
  
  <tr>
    <td>
      Where the reply text sits
    </td>
    
    <td>
      <code>
        choices[0].message.content
      </code>
    </td>
    
    <td>
      The <code>
        text
      </code>
      
       block in the <code>
        content
      </code>
      
       array
    </td>
  </tr>
  
  <tr>
    <td>
      SDK <code>
        base_url
      </code>
    </td>
    
    <td>
      The endpoint <strong>
        as shown
      </strong>
    </td>
    
    <td>
      The endpoint <strong>
        minus
      </strong>
      
       the final <code>
        /v1
      </code>
    </td>
  </tr>
</tbody>
</table>

Pick the format your application's existing SDK already uses: if it calls OpenAI, stay on the OpenAI format; if it calls Claude, stay on the Claude format. Either way, your application logic doesn't change.

## Step 2: Make your first request with cURL

Before you start, replace the three placeholders with the values from Step 1: `<API_ENDPOINT>`, `<YOUR_API_KEY>`, and `<MODEL_ID>`.

### List the available models

Run this first to confirm the key and the endpoint are both correct:

```bash
curl <API_ENDPOINT>/models \
  -H "Authorization: Bearer <YOUR_API_KEY>"
```

The deployed model should appear in the returned list.

### OpenAI API format

```bash
curl <API_ENDPOINT>/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -d '{
    "model": "<MODEL_ID>",
    "messages": [
      {"role": "user", "content": "Explain on-premises model deployment in one sentence."}
    ]
  }'
```

A successful call returns something like the following, with the model's output in `choices[0].message.content`:

```json
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "<MODEL_ID>",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "On-premises model deployment means running an AI model on your own hardware, so your data never leaves for the cloud."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 21, "completion_tokens": 35, "total_tokens": 56 }
}
```

### Claude API format

The auth header and the request path both differ from the OpenAI format, and `max_tokens` is required:

```bash
curl <API_ENDPOINT>/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: <YOUR_API_KEY>" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "<MODEL_ID>",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Explain on-premises model deployment in one sentence."}
    ]
  }'
```

## Step 3: Call the model from the Python SDK

### OpenAI API format

Install the package:

```bash
pip install openai
```

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["ARX_ENDPOINT"],  # ← Change 1: point at your ARX device
    api_key=os.environ["ARX_API_KEY"],    # ← Change 2: your API key
)

response = client.chat.completions.create(
    model=os.environ["ARX_MODEL_ID"],     # ← Change 3: the model ID
    messages=[
        {"role": "system", "content": "You are a friendly technical assistant. Keep answers short and precise."},
        {"role": "user", "content": "Explain on-premises model deployment in one sentence."},
    ],
)

print(response.choices[0].message.content)
```

An application that already calls OpenAI needs no logic changes — only the three marked values.

<warning>

**Watch the end of base_url:**

The OpenAI SDK's `base_url` is the endpoint as shown, ending with `/ai/v1`. The SDK appends `/chat/completions` itself.

</warning>

### Claude API format

Install the package:

```bash
pip install anthropic
```

```python
import os
import anthropic

client = anthropic.Anthropic(
    base_url=os.environ["ARX_ENDPOINT"].removesuffix("/v1"),  # ← Change 1: point at your ARX device
    api_key=os.environ["ARX_API_KEY"],                        # ← Change 2: your API key
)

message = client.messages.create(
    model=os.environ["ARX_MODEL_ID"],                         # ← Change 3: the model ID
    max_tokens=1024,                                          # Required in the Claude format
    system="You are a friendly technical assistant. Keep answers short and precise.",
    messages=[
        {"role": "user", "content": "Explain on-premises model deployment in one sentence."},
    ],
)

for block in message.content:
    if block.type == "text":
        print(block.text)
```

<warning>

**The Claude SDK's base_url drops the final /v1:**

The Anthropic SDK appends `/v1/messages` itself — the opposite of the OpenAI SDK.

</warning>

## Official API documentation

For the full field reference, see the vendor documentation:

- OpenAI API: [https://developers.openai.com/api/docs](https://developers.openai.com/api/docs)
- Claude API: [https://platform.claude.com/docs/en/api/overview](https://platform.claude.com/docs/en/api/overview)


## Sitemap

See the full [sitemap](https://docs.ardge.com/sitemap.md) for all pages.