Skip to content

osmosis-ai/osmosis

v0.1.1MIT

Evaluate an AI agent on your own examples, compare models on benchmarks, and train and deploy a checkpoint in your Osmosis workspace.

create-rollouts

Creates or adapts an Osmosis rollout from an agent or task, including its grader or Harbor verifier and authorized local validation. Use for new implementation or integration work.

debug-rollouts

Diagnoses or fixes Osmosis rollout startup, trajectory, grading, and timeout failures. Use for broken rollouts or unexpected rewards; model-quality comparisons belong to run-evals.

deploy-models

Selects and serves Osmosis LoRA checkpoints. Use for deployment, undeployment, deployment capacity, or inference endpoint usage.

get-started

Guides Osmosis onboarding, the bundled sample evaluation, and next steps from the user's goal and existing project. Use for first-time setup, a guided workflow, or deciding what to do next; specific run or debugging requests use their task skill.

plan-eval

Plans an Osmosis evaluation when the task, data, scoring, or comparison is undecided. Use for experiment design and budget; defined model comparisons belong to run-evals.

plan-training

Designs an Osmosis training experiment around a target behavior, reward, data split, and baseline. Run configuration and existing results belong to run-training.

prepare-datasets

Prepares, validates, splits, uploads, or downloads datasets for Osmosis. Use for concrete data files or platform datasets; experiment design belongs to the planning skills.

run-benchmarks

Prepares, runs, or analyzes Osmosis catalog benchmarks. Use for catalog tasks, harness comparisons, trial budgets, results, or stopping a benchmark run.

run-evals

Prepares, runs, or analyzes Osmosis model-quality evaluations, including small local comparisons and trained checkpoints. Use also for eval results, retries, publishing, or stopping runs.

run-training

Configures, submits, inspects, analyzes, or stops Osmosis training runs. Use for run settings, progress, reward trends, held-out regressions, or checkpoint comparisons.

test-rollouts

Verifies Osmosis rollout startup, trajectories, and grading with a bounded local smoke or regression test. Model-quality comparisons belong to run-evals, even on a small sample.