Skip to content

Tracking Overview

OICM Tracking is used to record machine learning experiment metadata in a workspace. TrackingClient is a lightweight wrapper around MLflow that standardizes how you create experiments, start runs, and log parameters, metrics, and tags programmatically.

Tracked data appears in OICM Platform -> Tracking, where teams can review experiments, inspect runs, chart metrics, and compare model training attempts.

TrackingClient supports the core tracking workflow for experiments, runs, parameters, metrics, tags, run lookup, experiment lookup, and experiment deletion.

Core Concepts

Workspace

A workspace is the OICM boundary where experiments and runs are stored. When using TrackingClient in online mode, provide the target workspace_id when setting or creating an experiment.

Experiment

An experiment is a container for related runs. Use one experiment for one analysis objective, benchmark, dataset, or training problem.

Examples:

  • fraud-detection-baseline
  • diabetes-regression-baseline
  • llama-evaluation-temperature-sweep

Experiments can be created from the OICM UI or from TrackingClient.

Run

A run is a single execution inside an experiment. Each run can contain:

  • Parameters, such as model_name, learning_rate, or dataset_version
  • Metrics, such as accuracy, loss, auc, or rmse
  • Tags, such as baseline, candidate, production-review, or dataset-current
  • System metrics captured by MLflow, when enabled by the client runtime

Use multiple runs inside the same experiment when you want to compare alternatives. For example, train two models on the same dataset and log each model training attempt as a separate run.

Parameters

Parameters are stable key/value configuration values for a run. They are best for metadata that describes how the run was configured.

Examples:

  • model_name = random_forest
  • n_estimators = 100
  • dataset = diabetes

Metrics

Metrics are numeric values logged during or after a run. You can log the same metric multiple times with different steps to create a time series.

Examples:

  • loss at every training step
  • accuracy after each epoch
  • auc once after evaluation

Metrics logged against a key can be viewed on the Metrics tab under the run.

Tags

Tags are labels associated with experiments or runs. They can be added from the OICM UI or through TrackingClient, and help users filter, categorize, and compare related work.

Examples:

  • Experiment tags: customer-demo, regression, benchmark
  • Run tags: baseline, candidate, sklearn, pytorch

Basic Workflow

Before writing tracking code, collect:

  • API host: must end with /api/tracking
  • API key: generated from the OICM user menu
  • Workspace ID: the target workspace where the experiment should appear
from oip_tracking_client.v2.tracking import TrackingClient

tc = TrackingClient(
    api_host="https://YOUR-OICM-HOST/api/tracking",
    api_key="YOUR_API_KEY",
)

tc.set_experiment(
    experiment_name="my-first-experiment",
    workspace_id="YOUR_WORKSPACE_ID",
)

with tc.start_run(run_name="smoke_test", tags=["baseline"]):
    tc.log_param("model", "xgboost")
    tc.log_metric("auc", 0.91)

After the script runs, open the target workspace in OICM and go to Tracking. The experiment should appear in the experiments list. Open it to view the smoke_test run, its parameters, metrics, tags, and metric charts.

Use OICM Tracking for:

  • Comparing multiple model runs inside one experiment
  • Capturing key parameters and metrics from training jobs
  • Labeling experiments and runs with tags
  • Reviewing metric trends over training steps
  • Comparing two or more runs side by side in the UI

Next Steps