Skip to content

Custom Docker Image Creation

This section shows how to create a custom Docker image that can handle various model endpoints on a linux/amd64 architecture. The service typically runs on port 8080, exposing endpoints for:

  • Classical ML models (e.g., linear regression, random forest).
  • LLMs for text generation (e.g., GPT-based).
  • Custom endpoints for specialized functions.

1. Service Overview

The Custom Image Creation Service listens on port 8080 and includes multiple endpoints to cover different model scenarios.

2. Health check

The OICM inference module requires the Docker image to have a /health-check endpoint. A successful deployment and usage of the model is strictly conditional on this endpoint returning a 200 OK status.

3. Example: Calling a Custom Endpoint in Python

import requests

base_url = "https://inference.develop.openinnovation.ai/models/<deployment_id>/proxy"
api_key = "api_key"
inference_endpoint = f"{base_url}/custom-endpoint"

payload = {}

headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
response = requests.post(inference_endpoint, json=payload, headers=headers)

4. Example Implementation

Below is a minimal FastAPI app showcasing a classical Linear Regression model on port 8080.

4.1 FastAPI Application (app.py)

from fastapi import FastAPI
from pydantic import BaseModel
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_diabetes
import numpy as np

app = FastAPI()

# Load dataset & train model
db = load_diabetes()
X_train, X_test, y_train, y_test = train_test_split(db.data, db.target, test_size=0.2, random_state=42)
model = LinearRegression().fit(X_train, y_train)

class InputData(BaseModel):
    input: list

@app.post("/v1/models/model:predict")
async def predict(data: InputData):
    input_data = np.array(data.input).reshape(-1, X_train.shape[1])
    prediction = model.predict(input_data)
    return {"prediction": prediction.tolist()}

@app.get("/health-check")
async def health_check():
    return {"status": "healthy", "message": "Model is ready and healthy"}

4.2 Dockerfile

Note: All docker images can't run as root user, so we need to run the app as a non-root user (runner) with PID 10000

# Python 3.9 slim base image
FROM python:3.9-slim

# create a non-root user
RUN useradd --uid 10000 runner
RUN mkdir /app && chown runner /app

USER runner

# Working directory
WORKDIR /app

# Copy files
COPY . /app

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Expose port 8080
EXPOSE 8080

# Start server using Uvicorn
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

4.3 Requirements

Include all necessary libraries in requirements.txt:

fastapi
uvicorn
scikit-learn
numpy

Next Steps