Skip to content

Model Bundles

Model bundles are containers for your external model artifacts, making it easy to deploy models trained outside the OICM platform. See the GitHub Examples for sample configurations using common ML libraries.

Model Bundles

1. Overview

Selecting a model bundle takes you to an overview page where you can:

  • Upload inference code or artifacts (pickle files, training scripts, requirements.txt).
  • View or modify existing files.

Model Bundles Overview

1.1 Settings

In the Settings tab, you can update or delete your model bundle as needed.

Model Bundles Settings

2. Model Bundle Files

A model bundle contains everything needed to:

  • Configure the environment (e.g., Python version, system packages).
  • Run the model (inference code, weights, etc.).

For library-specific demos, check the GitHub model-bundle-examples repository.

2.1 Environment Configuration

Configure the model server settings under the Model Version configuration in OICM.

Model Bundles Settings

Key fields include:

  • model server – Currently supports RayServe for model bundles.
  • python_version – Choose among py39, py310, or py311.
  • system packages – Specify OS-level packages for the deployment environment.

2.2 Requirements

  • Place a requirements.txt file in the root of the model bundle.
  • Include all library dependencies (e.g., numpy, scikit-learn).

2.3 Model Code

Add your model application code in model/model.py. The file must contain a Model class with two methods: load (to load the model) and predict (to perform inference).

class Model:

    def __init__(self, **kwargs) -> None:
        ...

    def load(self) -> None:
        # Load the model
        ...

    def predict(self, model_input: Any) -> Any:
        # Apply inference logic
        ...
        return model_output

Optionally, store supporting artifacts (e.g., .pkl, .h5 files) in a data folder.

3. Example: Scikit-learn Bundle

Below is a minimal layout for deploying a scikit-learn model as a model bundle:

config.yaml
requirements.txt
model/model.py
data/model.pkl

3.1 config.yaml

Defines the environment for Python 3.9:

model_framework: custom
python_version: py39

3.2 requirements.txt

Lists required libraries:

mlflow==2.9.2
numpy==1.26.3
scikit-learn==1.1.1

3.3 model.py

Loads model.pkl and performs inference:

from typing import Any
import os
import numpy as np
import pickle
import logging

class Model:
    def __init__(self, **kwargs) -> None:
        self._data_dir = kwargs["data_dir"]
        config = kwargs["config"]
        model_metadata = config["model_metadata"]
        self._model_binary_dir = model_metadata["model_binary_dir"]
        self._model = None

    def load(self) -> None:
        model_binary_dir_path = os.path.join(
            str(self._data_dir), str(self._model_binary_dir)
        )
        pkl_filepath = os.path.join(model_binary_dir_path, "data", "model.pkl")
        logging.info(f"Loading model file {pkl_filepath}")
        with open(pkl_filepath, "rb") as hand:
            self._model = pickle.load(hand)

    def predict(self, model_input: Any) -> Any:
        inputs = np.asarray(model_input)  # Convert the input to numpy array
        result = self._model.predict(inputs)
        predictions = result.tolist()  # Convert the model output to a Python list
        return {"predictions": predictions}

Next Steps