osmosis-ai/osmosis
Evaluate an AI agent on your own examples, compare models on benchmarks, and train and deploy a checkpoint in your Osmosis workspace.
Creates or adapts an Osmosis rollout from an agent or task, including its grader or Harbor verifier and authorized local validation. Use for new implementation or integration work.
Diagnoses or fixes Osmosis rollout startup, trajectory, grading, and timeout failures. Use for broken rollouts or unexpected rewards; model-quality comparisons belong to run-evals.
Selects and serves Osmosis LoRA checkpoints. Use for deployment, undeployment, deployment capacity, or inference endpoint usage.
Guides Osmosis onboarding, the bundled sample evaluation, and next steps from the user's goal and existing project. Use for first-time setup, a guided workflow, or deciding what to do next; specific run or debugging requests use their task skill.
Plans an Osmosis evaluation when the task, data, scoring, or comparison is undecided. Use for experiment design and budget; defined model comparisons belong to run-evals.
Designs an Osmosis training experiment around a target behavior, reward, data split, and baseline. Run configuration and existing results belong to run-training.
Prepares, validates, splits, uploads, or downloads datasets for Osmosis. Use for concrete data files or platform datasets; experiment design belongs to the planning skills.
Prepares, runs, or analyzes Osmosis catalog benchmarks. Use for catalog tasks, harness comparisons, trial budgets, results, or stopping a benchmark run.
Prepares, runs, or analyzes Osmosis model-quality evaluations, including small local comparisons and trained checkpoints. Use also for eval results, retries, publishing, or stopping runs.
Configures, submits, inspects, analyzes, or stops Osmosis training runs. Use for run settings, progress, reward trends, held-out regressions, or checkpoint comparisons.
Verifies Osmosis rollout startup, trajectories, and grading with a bounded local smoke or regression test. Model-quality comparisons belong to run-evals, even on a small sample.