---
title: "Model Inference Requirements"
slug: "model-inference-requirements"
updated: 2025-04-24T02:46:51Z
published: 2025-04-24T02:46:51Z
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ai-op.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Inference Requirements

# Koios Model Inference Standards

This section outlines how to properly format and deliver input data to Koios for AI model inference. Koios is designed to work with **.tflite** models, using timestamped time-series data from your industrial or commercial systems.

By following these guidelines, you ensure your model performs correctly and consistently within the Koios platform.

## Supported Model Format

Koios requires all models to be in `.tflite` format for deployment. Below are the most common paths to convert models built in other frameworks

| **Requirement** | **Description** |
| --- | --- |
| **Model Type** | .tflite – TensorFlow Lite format |
| **Model Purpose** | Inference only (no training occurs inside Koios) |
| **Optimization Tips** | Quantize or prune models before converting to .tflite for performance |
| **AVOID UNSUPPORTED OPS** | Koios uses TFLite runtime. Avoid custom or exotic TensorFlow operations not supported in TFLite. [TFLite Supported Ops List](https://www.tensorflow.org/lite/guide/ops_compatibility) |

### Model Conversions to .tflite

#### Tensorflow (Keras —> .tflite)

**✅ Supported:**

- Keras (`.h5`)
- SavedModel format

**🔧 Tool:**

- `tf.lite.TFLiteConverter` (TensorFlow to TFLite)

**Conversion Example**

```python
import tensorflow as tf

# From SavedModel
converter = tf.lite.TFLiteConverter.from_saved_model("path/to/saved_model")
tflite_model = converter.convert()

# Save to file
with open("model.tflite", "wb") as f:
    f.write(tflite_model)
```

#### PyTorch —> TFLite

⌛ **Steps:**

1. Export PyTorch model to **ONNX** format.
2. Convert ONNX to **TensorFlow SavedModel**.
3. Convert SavedModel to **TFLite**.

**🛠️ Tools:**

- `torch.onnx.export` (PyTorch to ONNX)
- `onnx-tf` or `onnx2tf` (ONNX to TensorFlow)
- `tf.lite.TFLiteConverter` (TensorFlow to TFLite)

**Example Workflow**

```python
# Step 1: PyTorch to ONNX
import torch
import torchvision.models as models

model = models.resnet18(pretrained=True)
model.eval()
dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(model, dummy_input, "model.onnx")

# Step 2: ONNX to TensorFlow
# Terminal command using onnx-tf:
# pip install onnx-tf
onnx-tf convert -i model.onnx -o tf_model

# Step 3: TensorFlow to TFLite (as in previous example)
```

## Input Data Format

Koios expects a **2D array** for model input, shaped as follows:

```python
Input Shape: [1, T, N]
# Where:
#  - 1 = Sample
#  - T = Number of time steps (depth of history, fixed)
#  - N = Number of tags (sensor inputs/features)
```

| **Feature** | **Description** |
| --- | --- |
| **Time-Ordered Rows** | Rows represent timestamped data, ordered **oldest to newest** |
| **Tags as Columns** | Each column is a process variable (tag) bound to the model |
| **Fixed History Depth** | All tags must have the **same number of time steps** (T) |
| **Tag Order** | Input tag order must match the model's design (same as during training) |

#### Example Input Layout

| **Time (Oldest → Newest)** | **Tag 1** | **Tag 2** | **Tag 3** | **...** |
| --- | --- | --- | --- | --- |
| **T-4** | 1.2 | 3.5 | 7.8 | ... |
| **T-3** | 1.1 | 3.6 | 7.7 | ... |
| **T-2** | 1.3 | 3.4 | 7.9 | ... |
| **T-1** | 1.2 | 3.5 | 7.8 | ... |
| **T** | 1.1 | 3.6 | 7.7 | ... |

## Coming Soon: Variable-Length Inputs

Future versions of Koios will allow each tag to have a **different historical depth**. This feature is inspired by flexible input handling approaches like gym.spaces.Dict or stable_baselines3's spaces.Tuple.

This will enable models to use:

- Long-term history for slow-changing variables
- Short-term slices for fast-changing variables

**Stay tuned for updates in future releases.**

## Best Practices

- Ensure your model was **trained with [1, T, N] data**, in the same format as Koios expects.
- Preprocess and align tag data (handle nulls, normalize) before using it for inference.
- Validate .tflite models offline using test arrays that mimic expected Koios input format.
- Use Koios’ built-in historian to avoid relying on external systems for inference.

## Troubleshooting

| **Issue** | **Recommendation** |
| --- | --- |
| **Input shape mismatch** | Double-check matrix dimensions and tag order |
| **Model fails to run** | Ensure the .tflite is trained, finalized, and compatible |
| **Missing or stale data** | Confirm historian sync and data freshness |
| **Output not returned** | Verify output bindings and network interface settings |