Decision Workbench

Turn messy choices and proposals into transparent, defensible decisions.

imageLicence: MIT imagePython 3, standard library only imageAgent Plugins 1.0

Decision Workbench is an agent plugin for ChatGPT, Codex CLI, Claude Code and GitHub Copilot CLI — one package, the same skill and engine behind each. It turns a difficult choice into a ranked answer and a record of why, without pretending the ranking is the decision.

The work is split so each part does what only it can:

• the agent frames the decision, proposes the alternatives you had not considered, asks adaptive questions and argues with its own emerging conclusion;
• the engine does every calculation — weights, rankings, consistency, sensitivity, group aggregation — so the numbers are reproducible rather than generated;
• you supply the values, the context and the authority, and every judgement in the model records which of you it came from;
• the files hold the record: a decision model, an interactive report and a group ballot you can share.

Everything the plugin itself does happens on your machine: the calculation scripts, the reports and the ballots never make a network request, and there is nothing to sign up for or configure. Your conversation still goes to whichever assistant you are using — the plugin simply adds no second destination, and the files it produces are yours alone.

What it can do

• frame an ambiguous decision and identify the actual decision owner, scope and horizon;
• generate credible alternatives, including status quo, conditional, pilot, hybrid and defer options;
• separate hard constraints from compensating preferences;
• elicit priorities through a small number of natural-language comparisons;
• rate alternatives against explicit 0–100 anchors;
• critique a submitted proposal while separating decision merit from argument quality;
• calculate incomplete or complete AHP-style weights deterministically;
• test winner sensitivity and expose fragile conclusions;
• generate an interactive HTML report that recalculates as you explore scenarios;
• generate a private group ballot that works offline in any browser;
• hand you a clickable ballot instead of a long text interview where the assistant cannot draw its own UI;
• aggregate returned ballot JSON using AIJ or AIP;
• track which judgements the decision owner actually confirmed, and which are still the agent's;
• answer "what would have to be true for my preferred option to win?" when the result conflicts with instinct;
• produce a reusable decision record with assumptions, evidence, dissent and revisit triggers.

How it works

flowchart LR
  U[User or facilitator] <--> H[ChatGPT / Codex / Claude Code / Copilot host]
  H --> S[Decision Workbench skill]
  S --> R[Method references and templates]
  S --> E[Pure Python decision engine]
  E --> J[Decision and analysis JSON]
  J --> P[Interactive HTML report]
  J --> B[Offline HTML ballot]
  B --> V[Participant JSON exports]
  V --> G[AIJ / AIP aggregation]
  G --> P

The skill directs the conversation and hands every calculation to the engine, so the numbers in the final record are reproducible from decision.json alone — not sampled from a language model.

The method

First, decide whether to decide

Not every choice deserves a matrix. The skill triages: a cheap, reversible, familiar choice gets a concise recommendation and nothing more. The full machinery is reserved for decisions that are costly or hard to reverse, where objectives conflict, stakeholders disagree, or the rationale must survive an audit.

Frame before you rank

The workbench establishes the actual question, the decision owner, scope, time horizon and constraints before any scoring. Two rules matter here:

• Constraints are not weights. A regulatory prohibition or minimum threshold screens alternatives out; it is never traded off against convenience.
• Alternatives come before criteria. Real decision sets usually miss options — status quo, a pilot, a conditional adoption, a hybrid, deferral. The skill proposes them rather than ranking only what was supplied.

AHP in five minutes

The comparison core is based on the Analytic Hierarchy Process (Thomas Saaty, 1970s). AHP structures a decision as a hierarchy — a goal, the criteria that matter, the alternatives — and derives priorities from pairwise comparisons instead of asking anyone to invent weights directly. People are more reliable at "which of these two matters more, and by how much?" than at "give security a percentage".

Judgements are captured in words and mapped to ratios:

From these ratios the engine derives one weight per criterion:

• Complete comparisons use the classical principal-eigenvector method, with the consistency ratio (CR) as a diagnostic. Human judgements are rarely perfectly transitive; CR compares your matrix against random judgement, and the conventional rule of thumb is CR ≲ 0.10. The engine reports inconsistency and shows which judgements drive it — it never silently "corrects" an answer.
• Incomplete comparisons are first-class. As long as the comparison graph is connected, weights are estimated by weighted logarithmic least squares, with a log residual as the fit diagnostic. The skill asks only the comparisons that connect the graph, then only the missing ones that could actually change the result.

Alternatives are scored differently from criteria. The default is AHP's ratings (absolute measurement) mode: each alternative is rated 0–100 against explicit, criterion-specific anchors, which scales to many alternatives and keeps evidence visible. Pairwise comparison of alternatives is used only for small sets where relative judgement is genuinely easier. The final ranking is a weighted additive synthesis with a per-criterion contribution breakdown.

The last step is sensitivity: for each criterion, how far would its weight have to move before the winner changes? A 2-point lead that flips on a 3-percentage-point weight shift is reported as fragile, not as a verdict.

Keeping the human in charge

A multi-criteria model is only as good as whose values are in it. Left unchecked, an agent that fills gaps fluently will produce a confident ranking built largely on its own guesses — and the fluency is what stops anyone noticing.

Three mechanisms address this directly.

Provenance on every judgement. Each comparison, rating and weight is marked user (stated or confirmed by the person deciding), source (extracted from material they supplied) or agent-proposed (the agent's provisional estimate). Provenance never touches the arithmetic; it makes authorship visible.

A ratification audit. analyse reports what share of the criterion weight, and of each alternative's score, rests on ratified judgements — plus a queue of what is still outstanding, sorted by how much it influences the result:

{ "weight_ratified_share": 1.0,
  "score_ratified_share": {"existing-suite": 1.0, "lightweight-practice": 0.0},
  "fully_ratified": false,
  "queue": [{"kind": "alternative_performance", "influence": 1.0,
             "message": "'lightweight-practice' ranks 3 on estimates you have not confirmed..."}] }

This becomes the elicitation agenda. The agent asks about high-influence unratified judgements by showing its current value and asking you to confirm or correct it — never by asking you to invent a number from scratch — and does not present a final recommendation while material judgements remain unratified.

A challenge command. When the ranking disagrees with your instinct, the unease is usually information: the model is missing something you have not articulated. Rather than defending the arithmetic, the workbench inverts it:

python scripts/decision_engine.py challenge --input decision.json --prefer existing-suite

It reports exactly what would have to be true for your preferred option to lead — which weight shifts and which score revisions reach a tie, cheapest first — so the conversation becomes "is everyday user value really 49% of this decision, or did we over-weight it?" rather than "the model says you are wrong." If your option was screened out by a hard constraint, it says so, because that is a conversation about the constraint, not the weights.

The workbench also records your initial lean before any calculation runs, so anchoring is visible rather than invisible, and it records an override — you choosing against the calculated leader, with your reason — as a successful outcome rather than a failure. The weights are never retuned afterwards to make the model agree; an honest divergence has audit value that a retrofitted consensus destroys.

How it asks

Clicking one of nine buttons is a better way to answer "which matters more?" than writing a sentence about it, so the skill uses the best input surface the assistant actually has, checked per session rather than assumed:

The ballot route matters most in ChatGPT, where a skill cannot render its own interface at all: OpenAI reserves interactive UI for MCP servers, and this plugin ships none. Rather than run a twelve-question interview in chat, it generates the ballot, you click through it in your browser, and it merges your export back with apply-ballot — no answers retyped, every judgement recorded as yours.

python scripts/render_ballot.py --input decision.json --solo --output decision-ballot.html
python scripts/decision_engine.py apply-ballot --input decision.json --ballot returned.json --output decision.json

Sliders you never move are not exported, so a rating you did not give is never recorded as one you did.

Group decisions

For several participants the workbench generates an offline ballot — a single HTML file with no remote scripts, fonts or requests. Each participant completes it independently in a browser and exports a JSON file; the facilitator aggregates the returned files with one of two standard AHP group methods:

• AIJ (aggregation of individual judgements): geometric mean of the pairwise judgements — the group acts as one decision body;
• AIP (aggregation of individual priorities): each participant's priorities are computed first, then combined — separate constituencies stay visible.

The engine reports between-person disagreement separately from within-person inconsistency, so a dispute about values is never disguised as a data-quality problem.

What the numbers are not

The scores are preference indices — a transparent restatement of the judgements you supplied — not probabilities, utilities or objective truth. The skill leads with a plain-language position, its conditions and its fragility; the highest score never mechanically makes the decision.

Install

This repository doubles as a one-plugin marketplace named rohde-tools, which both Claude Code and Codex CLI can install from directly.

Claude Code

Inside a Claude Code session:

/plugin marketplace add ThomasRohde/decision-workbench
/plugin install decision-workbench@rohde-tools

Or try a local clone for one session without installing anything:

claude --plugin-dir ./decision-workbench

This adds the model-invoked skill plus three explicit entry points:

See docs/CLAUDE_CODE.md.

Codex CLI

codex plugin marketplace add ThomasRohde/decision-workbench
codex plugin add decision-workbench@rohde-tools

A local clone works as a marketplace source too: codex plugin marketplace add ./decision-workbench.

ChatGPT

Upload the package as a Skills only plugin ZIP. Build a validated archive from a clone:

python skills/decision-workbench/scripts/package_self_test.py --build-zip decision-workbench-0.2.0-chatgpt.zip

Then follow docs/CHATGPT_UPLOAD.md. After installation, start a new chat and invoke Decision Workbench explicitly.

GitHub Copilot CLI

copilot plugin install ThomasRohde/decision-workbench

Or install a local clone with copilot plugin install ./decision-workbench. See docs/GITHUB_COPILOT.md.

Use it

Prompts that suit it:

• "Help me decide whether to move, renovate, or stay. Do not ask a long questionnaire; propose a frame first."
• "Critique this recommendation. Reconstruct the alternatives it omitted and tell me what would change the conclusion."
• "Prioritise these six initiatives, but first determine whether a ranking or a portfolio decision is the real problem."
• "Create an offline group ballot for this decision and explain how I should collect and aggregate the results."

A session leaves inspectable files in your working directory:

The two HTML files are single files with everything embedded, so they open in any browser, work with no connection, and can be emailed or archived as they are.

Run the engine directly

The engine is an ordinary CLI, usable without any agent. From skills/decision-workbench/ (use python3 on macOS and Linux):

python scripts/decision_engine.py validate --input assets/example-decision.json
python scripts/decision_engine.py analyse --input assets/example-decision.json --output analysis.json
python scripts/decision_engine.py challenge --input assets/example-decision.json --prefer existing-suite
python scripts/render_report.py --input assets/example-decision.json --analysis analysis.json --output report.html
python scripts/render_ballot.py --input assets/example-decision.json --output ballot.html

Development

decision-workbench/
├── plugin.json                         # Portable Agent Plugins 1.0 manifest
├── .codex-plugin/plugin.json           # ChatGPT / Codex adapter manifest
├── .claude-plugin/
│   ├── plugin.json                     # Claude Code manifest
│   └── marketplace.json                # Marketplace catalogue (Claude Code, Codex CLI)
├── commands/                           # Claude Code slash-command entry points
├── skills/decision-workbench/
│   ├── SKILL.md                        # Main workflow
│   ├── agents/openai.yaml              # OpenAI presentation and invocation metadata
│   ├── scripts/                        # Pure Python 3, standard library only
│   ├── references/                     # Progressive method guidance
│   ├── assets/                         # Templates and example model
│   └── tests/                          # Unit and package tests
├── assets/                             # Plugin icons
└── docs/                               # Installation, testing and publication guidance

Run the tests and the package self-test from the repository root:

python -m unittest discover -s skills/decision-workbench/tests -v
python skills/decision-workbench/scripts/package_self_test.py

package_self_test.py checks that the three manifests agree on identity, that no script in the package can reach the network, and that generated HTML embeds every asset rather than fetching one. With Claude Code installed, claude plugin validate . checks the marketplace and plugin manifests as well.

Design rationale, trust boundaries and the multi-host packaging story are in docs/DESIGN.md.

Limits

• The scores are preference indices, not probabilities or objective truth.
• Provenance is self-reported by the agent; it makes authorship auditable, not tamper-proof.
• The offline ballot is not authenticated or tamper-evident and is not suitable for binding elections.
• Reports and ballots open in your browser, not inside the chat window. In ChatGPT that is a hard limit, not a preference: interactive UI there requires an MCP server, which this plugin does not ship.
• High-stakes decisions still require appropriate professional and accountable human judgement.

Licence

MIT. See LICENSE.