---
title: "Deploy your fine-tuned model on an ARX device"
description: "Check your fine-tuned model against the formats and files ARX requires, upload it to a private Hugging Face repository, then download and deploy it from AI Model Center"
canonical_url: "https://docs.ardge.com/en/knowledge/arx-os/deploy-fine-tuned-model"
---
# Deploy your fine-tuned model on an ARX device

> Check your fine-tuned model against the formats and files ARX requires, upload it to a private Hugging Face repository, then download and deploy it from AI Model Center

## Overview

ARX runs on a server with GPU resources. Beyond the open-source models already listed in AI Model Center, you can also deploy models you have fine-tuned yourself.

Upload the fine-tuned model to your private Hugging Face repository and save a Hugging Face access token in ARX. AI Model Center can then find and deploy that model.

This article covers the model formats and files ARX requires, plus the full upload and deployment path.

## What you'll learn

- Check that your fine-tuned model's format and files meet ARX's requirements
- Upload the model to a private Hugging Face repository
- Download and deploy the model from AI Model Center

**Estimated time: about 20 minutes**, not counting upload and download time

## Applies to

- ARX OS 1.0 and later

## Before you begin

- You have a Hugging Face account and have saved an access token in ARX. See [Register a Hugging Face account and get an access token](https://docs.ardge.com/en/knowledge/arx-os/huggingface-token)
- Your account has access to AI Model Center

## Supported formats and required files

ARX supports two weight formats: **safetensors** and **GGUF**. Before uploading, check that your repository contains these files:

<table>
<thead>
  <tr>
    <th>
      File
    </th>
    
    <th>
      Purpose
    </th>
    
    <th>
      When it's required
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        config.json
      </code>
    </td>
    
    <td>
      Model architecture settings
    </td>
    
    <td>
      Always
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        tokenizer.json
      </code>
    </td>
    
    <td>
      The tokenizer itself
    </td>
    
    <td>
      Always
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        tokenizer_config.json
      </code>
    </td>
    
    <td>
      Tokenizer settings
    </td>
    
    <td>
      Always
    </td>
  </tr>
  
  <tr>
    <td>
      Weight files (<code>
        *.safetensors
      </code>
      
       or <code>
        *.gguf
      </code>
      
      )
    </td>
    
    <td>
      Model weights
    </td>
    
    <td>
      Always
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        model.safetensors.index.json
      </code>
    </td>
    
    <td>
      Shard index for the weights
    </td>
    
    <td>
      When the weights are split across several files
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        preprocessor_config.json
      </code>
    </td>
    
    <td>
      Input preprocessing settings
    </td>
    
    <td>
      For vision models
    </td>
  </tr>
</tbody>
</table>

<tip>

**Fine-tuning a popular open-source model? Most files carry over:**

You can usually take these files from the original model's repository. Only `config.json` needs updating to match how you fine-tuned the model.

</tip>

ARX supports vLLM, NVIDIA vLLM, Llama.cpp, and Faster Whisper as deployment engines. You choose the engine yourself — the interface walks you through it when you download the model in Step 3.

## Step 1: Check that the model files are complete

1. Review your fine-tuned model directory against the file list above
2. Add any missing files
3. Update `config.json` to match how you fine-tuned the model

## Step 2: Upload the model to a private Hugging Face repository

1. Sign in to Hugging Face and go to [huggingface.co/new](https://huggingface.co/new) to create a model repository
2. Enter a repository name and set the visibility to **Private**
3. Open the repository's **Files and versions** tab
4. Click **Add file › Upload files** and select your model files
5. Write a commit message, then click **Commit changes**

## Step 3: Download and deploy from AI Model Center

1. Open AI Model Center and go to the **Model Hub** page
2. Switch to the **Hugging Face** tab
3. Search for the model you uploaded — its card appears in the results
4. Click **Download & Deploy** on the card
5. Follow the prompts to pick a deployment engine and its settings

The model downloads from Hugging Face; how long that takes depends on the model size and your network speed. Watch live progress on the **Downloaded Models** page. Deployment starts as soon as the download finishes — switch to the **Model Instances** page to follow it. The model is deployed once its status reads **Running**.

![The Hugging Face tab in Model Hub, with Download & Deploy on each model card](https://docs.ardge.com/screenshots/knowledge/en/arx-os/deploy-fine-tuned-model/step-3-1.webp)

<warning>

**Models in a private repository need an access token saved in ARX first:**

Without a saved token, the **Hugging Face** tab won't find models in your private repository.

</warning>

![The Hugging Face access token setting in AI Model Center Settings](https://docs.ardge.com/screenshots/knowledge/en/arx-os/deploy-fine-tuned-model/step-3-2.webp)

<tip>

**Check GPU capacity before deploying:**

If the target GPU card doesn't have enough remaining capacity for the model's estimated usage, the system blocks the deployment. See [What does GPU VRAM Allocated mean?](https://docs.ardge.com/en/knowledge/arx-os/gpu-vram-allocated)

</tip>

## Related articles

- [Register a Hugging Face account and get an access token](https://docs.ardge.com/en/knowledge/arx-os/huggingface-token)
- [What does GPU VRAM Allocated mean?](https://docs.ardge.com/en/knowledge/arx-os/gpu-vram-allocated)
- [Call on-premises models on ARX through the API](https://docs.ardge.com/en/knowledge/arx-os/api-model-calls)


## Sitemap

See the full [sitemap](https://docs.ardge.com/sitemap.md) for all pages.