Model Bundles

Model bundles are containers for your external model artifacts, making it easy to deploy models trained outside the OICM platform. See the GitHub Examples for sample configurations using common ML libraries.

---

1. Model Bundle Module

In the Model Bundle module screen, you can view existing model bundles or create a new one.

1.1. Creating a Model Bundle

The Create Model Bundle button appears in the center of the screen if no model bundles exist, otherwise it is at the top right of the main content window.

Clicking on Create Model Bundle reveals a modal window that allows you to:

• name your model bundle,
• describe your model bundle in detail, and
• add tags to your model bundle.

Once complete, click on the Create model bundle button in the modal window to see the overview of the model bundle.

1.2. Viewing a Model Bundle

Click on View Model Bundle on an existing model bundle to see its overview.

2. Overview

Creating a new model bundle or selecting a model bundle takes you to the overview page where you can:

• View or modify existing files in the Model Bundle.
• Upload inference code or artifacts (pickle files, Python scripts, requirements.txt) into the Model Bundle.

By default, a model bundle contains 4 starter files which you can build upon.

2.1. Settings

In the Settings tab, you can update or delete your model bundle as needed.

2.2. Viewing a file

Click on a file in the file tree to display its contents in the text editor window.

2.3. Editing a file

Edit your code on the fly by clicking into the text editor window and typing into it.

Save your file edits by clicking on the Save changes button at the bottom of the main window. This button only appears when changes are detected.

2.3. Uploading a file

In the overview, click on the upload file (leftmost) icon above the file tree to display the file upload modal window.

There are two ways to upload a file:

1. Click on the file upload area to open a file selector and select a file from your file explorer.
2. Drag a file from your file explorer into the file upload area

Confirm your file in the card below, then determine its final path in the Model Bundle file tree via the File relative path field.

You can also upload a directory of files by clicking the upload directory (second from left) icon.

---

3. Model Bundle Files

A model bundle contains everything needed to:

• Configure the environment: via requirements.txt.
• Run the model: via model.py and a weights file (e.g. .pkl, .json, .joblib, etc.).

For library-specific demos, check the GitHub model-bundle-examples repository.

3.1. Requirements

• Place a requirements.txt file in the root of the model bundle.
• Include all library dependencies (e.g., numpy, scikit-learn).

3.2. Model Code

Add your model application code in model.py. The file must contain a Model class with two methods: load (to load the model) and predict (to perform inference).

class Model:

    def __init__(self, **kwargs) -> None:
        ...

    def load(self) -> None:
        # Load the model
        ...

    def predict(self, model_input: Any) -> Any:
        # Apply inference logic
        ...
        return model_output

NOTE: We only currently support inference and loading via a single model.py file. If model.py loads adjacent modules or functions, it will not work.

3.3. Model Weights

Store model weights (e.g., .pkl, .h5 files) in the root of the file tree.

3.4 Custom endpoints

You can define custom endpoints by uploading an endpoints.yaml file. The following example shows how to expose the Model.predict_stream() method to the /custom-predict endpoint via the POST verb.

endpoints:
  - endpoint: /custom-predict
    http_method: POST
    function_name: predict_stream
    stream: true

This assumes that you have defined a method named predict_stream() in your model.py.

If endpoints.yaml is absent or empty, it is ignored during runtime.

3.4.1 Restrictions

• endpoint:
  • You must use alphanumeric characters, -, and _ only.
  • You cannot define two different function names for the same endpoint.
  • You cannot override the following endpoints:
    • /health-check
    • /predict
• http_method:
  • You can only define a GET or POST HTTP method.
• function_name:
  • You cannot expose dunder methods.
  • You cannot expose private methods (i.e. methods that begin with _).
  • You cannot expose the load method, this is restricted for internal use.
  • A GET endpoint must be a method that have no arguments.
  • A POST endpoints must be a method that have a single argument.
• stream:
  • true or false. false by default.

---

4. Example: Scikit-learn Bundle

Below is a minimal layout for deploying a scikit-learn model as a model bundle:

requirements.txt
model.py
model.joblib

4.1. requirements.txt

Lists required libraries:

joblib==1.5.1
numpy==2.0.2
scikit-learn==1.6.1
scipy==1.13.1

4.2. model.py

Loads model.pkl and performs inference:

import logging
import os
from typing import Any, Optional

import joblib
import numpy as np
from sklearn.ensemble import RandomForestRegressor


class Model:
    def __init__(self, **kwargs) -> None:
        self._data_dir = kwargs["data_dir"]
        config = kwargs["config"]
        model_metadata = config["model_metadata"]
        self._model_binary_dir = model_metadata["model_binary_dir"]
        self._model: Optional[RandomForestRegressor] = None

    def load(self) -> None:
        model_binary_dir_path = os.path.join(
            str(self._data_dir), str(self._model_binary_dir)
        )
        joblib_filepath = os.path.join(model_binary_dir_path, "model.joblib")
        self._model = joblib.load(joblib_filepath)

    def predict(self, model_input: Any) -> Any:
        x = np.asarray(model_input)  # Convert to NumPy array
        result = self._model.predict(x)
        predictions = result.tolist()  # Convert to Python list
        return {"predictions": predictions}

4.3. model.joblib

A binary file containing the parameters of a trained RandomForestRegressor.

---

Next Steps

• Registered Models & Versions – Learn how to register and version your models.
• Model Version Configuration – Customize resources and environment settings for your bundles.
• Deployment Process – Explore how to deploy and monitor your bundles on the OICM platform.