polarizetech/scientific-research-scaffold
Create and adopt reproducible scientific study, simulator, and research-tool repositories.
Changelog
v0.12.1 (2026-11-17)
-
Generated CI checks out
v0.12.1; theagentssubmodule is unchanged (scientific-research-agents v0.10.0). -
scaffold checkskips the scaffold's own CI checkout. Generated CI checks the scaffold out into.scaffold/inside the repo, and a tool repo's check walked into it and failed on the templates'preregistrations/folders (first seen on polarizetech/entropy-fetch's first CI run). The tree walks now skip.scaffold.
v0.12.0 (2026-10-29)
-
Generated CI checks out
v0.12.0; theagentssubmodule is scientific-research-agents v0.10.0. -
Exploratory work needs no licence yet. The person's requirement (2026-10-29): "it's just exploratory. It shouldn't need a hypothesis or a license."
scaffold checknow warns, instead of failing, when an exploratory repo has noLICENSE: a study whoseHYPOTHESIS.tomlhasmode = "exploratory", or a tool or sim whose rootSCOPE.tomlhasstage = "exploratory". Claimed work still needs one. (An exploratory study already needed no hypothesis; that check shipped with the exploratory default above and is in this release too.) -
Apps say where they are viewed beyond localhost. The person's requirement (2026-10-29): scaffolded apps use one tailnet host by default, with an environment variable to change it. A profile may set
[serving] tailnet_url;scaffold new appwrites it into the app'sserve.pyas the default ofTAILNET_URL(read from the environment first), and the app prints$TAILNET_URL/<slug>/<version>/beside its localhost URL, so the URL ends in the version ($TAILNET_URL/<slug>/v1/); a bare/redirects there and the/<slug>and/<version>prefixes are stripped.new appprints the same line. A profile with notailnet_urlprints nothing. The polarizetech profile nameshttps://joshuas-macbook-pro.tailc463ae.ts.net. Existing apps are not touched:serve.pybelongs to the repo once written. -
Studies and tools start exploratory, from a goal and a rough approach. The person's requirement (2026-10-28): studies and tools are set up as exploratory by default and need no strict claim, only "a general goal" and a "rough idea of how we might get there". That first conversation is "a back and forth to decide if the goal is too big for the scope of one study or one tool, and whether it should be broken up".
scaffold new study <name> --goal "..." --approach "..."writesHYPOTHESIS.tomlwithmode = "exploratory", the goal and approach, and one exploratory element (no counts-against needed), and labels the README.scaffold new tool <name> --goal "..." --approach "..."(or--job) writes an exploratorySCOPE.tomlwith the same fields. Both record the sizing decision asscope = "one"or"split: a, b"withscope_decided;scaffold checkwarns until it is there, and an exploratory study must stay at SKETCH. The claim-first start remains on request: a study's--hypothesis, or--claim. The study and tool agent rules, the new-study skill, the README and PROTOCOL.md describe the sizing conversation. -
Every question starts with the retrieval tool, and the tool picks the specialist. The person's requirement (2026-10-28): every question asked in these projects, by Claude, Codex or any other assistant, runs the retrieval tool, and specialists are brought in only where the tool's pipeline routes them. When a profile names a retrieval tool, the rule in
AGENTS.mdnow covers every question, including questions about the repo's own code (retrieve, then read the checkout), instead of only science questions. A second rule has the assistant callpipeline__profile_routebefore bringing in a research role or specialist, and use only what it returns. Claude Code also gets aUserPromptSubmithook,scaffold rag-hook --server NAME, installed beside the session hook. On every prompt it adds the rule to the context, and when$RESEARCH_RAG_URLand$RESEARCH_RAG_TOKEN_FILEpoint at the tool's HTTP gateway on that machine, it adds the route for the prompt (specialist, brief path and guidance). Nothing machine-specific is written into a repo. The hook never blocks a prompt and skips slash commands; without the gateway it adds the rule alone.scaffold updateadds or refreshes the hook, and removes it if the profile stops naming a tool. Other assistants follow the rule inAGENTS.md, since only Claude Code runs the hook. Theagentssubmodule moves to scientific-research-agents v0.10.0, whose block template says the same ("Retrieval first, for every question") in place of "Retrieval before search". -
Bench sims in a tool. A sim in a tool's
sims/<slug>/whoseSCOPE.tomlsayskind = "bench"(the prereg kit'sTOOL_VERSIONING.md§1b) runs like part of the test suite against reference simulators that live in another repo, pinned by tag. The person's decisions (2026-10-27): keep the name sims, since "a sim that's in a tool is a little bit different than a sim that's in a study"; nosims/folder by default, "only if we start adding sims"; in a tool, not meeting the sim's preregistered tolerances "is essentially a failure"; and bench sims are not run automatically but "when tagging a new version".scaffold new sim <name> -q "..." --inside <tool> --reference "NAME=OWNER/REPO@TAG[#SUBDIR],licence=L[,runs=process]"resolves the tag's commit and renders the sim:SCOPE.tomlwith[[references]], its ownpyproject.tomlpinning each imported reference aspkg @ git+URL@TAG#subdirectory=..., an undecidedtolerances.toml,run.py(refuses to score before the owner's tolerances are decided and committed, or against a reference at another commit; fetchesruns = "process"references, such as GPL code, into an ignored cache and never imports them; writesresults/<tool version>.json) and a pytest wrapper. It is registered under[[comparisons]]withkind = "bench".scaffold checkfails a reference pinned to a branch, without a full commit, missing from the sim'spyproject.tomlor pinned there at another tag, repo or subdirectory; a bench sim withouttolerances.toml, with results before tolerances were decided, or whose latest result was scored against other tolerances; andpreregistrations/in a bench sim. It warns on GPL code that is imported. The toolMakefilegainsmake sims(not run by CI), andmake checknow runspytest testsso the sims' tests stay out of the tool's own run. The minimal TOML reader (Python < 3.11) now reads floats written with an exponent. -
Comparison units in a tool. A tool repo may hold
sims/<slug>/anddatasets/<slug>/"as long as they are about comparing the tool's functionality against other tools (aka preregistering predictions based on inputs and seeing if the run is correct - essentially treating these like an extension on the test suite)" (the person's wording, 2026-10-24; the prereg kit'sTOOL_VERSIONING.md§1a).scaffold new sim <name> --inside <tool> --compares "<other> <version>"andscaffold new dataset <name> --study <tool> --compares ...create one: itsSCOPE.tomlsays what itcompares, it is registered under[[comparisons]]inTOOL.toml, and the tool gets a rootEXPERIMENTS.md.scaffold checkon a tool now allows thatEXPERIMENTS.mdandpreregistrations/directly inside a comparison unit, and fails a comparison unit with nocompares, one missing from[[comparisons]], orpreregistrations/anywhere else. The agent rules and templates for tools say the same.
v0.11.1 (2026-10-21)
- Pins scientific-research-agents v0.8.0, including the public installer for private researcher extensions. Future scaffolded repositories receive the current public researcher brief; private specialist briefs remain in a separate checkout and are installed only into selected repositories.
- Generated CI checks out
v0.11.1.
v0.11.0 (2026-10-18)
- Agent-managed impact propagation. A study records whether hypothesis impacts are
agent-managedorperson-managed. Under the default agent-managed policy, routine mapping and stale-impact queues do not ask the person to repeat decisions:scaffold link-hypothesisrecords an existing unit's role and accepted element links, andscaffold sync-hypothesismarks affected unitsneeds-reviewwithout changing their claims. - Local context for every unit. Exploratory as well as testing units carry element IDs, accepted revisions,
fingerprints and review state in
SCOPE.toml. Existing unit claims remain intact: association with a broader study element does not silently replace a narrower accepted unit claim. - Cross-study routing through the corpus. The corpus project overview is the reverse impact index. An agent changing a corpus claim, finding or evidence link follows its study/element references, synchronizes each available study, and records any unavailable checkout as pending rather than asking the person to coordinate it.
- Pins scientific-research-agents v0.6.0, whose researcher, analyst and computational-engineer briefs assign ownership of that propagation to the agent making the evidence, finding or implementation change.
- Generated CI checks out
v0.11.0.
v0.10.0 (2026-10-17)
- Study-level hypothesis maps. New studies require
--hypothesisand carry the person's wording verbatim inHYPOTHESIS.toml; the question is never silently substituted. Specialist agents propose falsifiable steps and integrations, but each remains proposed until the person accepts its exact revision. Evidence status is separate from established/supported/derived/gap/exploratory basis, and every non-established accepted element has a real goal and falsifier rather than a placeholder. - Hypothesis-to-unit dependency graph. Each app version, sim, dataset analysis and calculator maps to one
or more accepted elements.
scaffold new ... --hypothesis-element IDprevalidates before writing. Testing scopes carry accepted element provenance. Changes require a disposition bound to the exact stale revisions, fingerprints and corpus revision. Legacy mappings remainunmappedand become failures from PROBE onward. - Corpus project overview and local reconciliation.
RESEARCH.mdlinks the canonical corpus overview and evidence IDs are matched exactly. The study rules use specialist profiles and the retrieval tool first. Offline checks compare the recorded corpus revision with a sibling corpus checkout; reconciling a new revision makes linked units stale for explicit review. - Generated CI checks out
v0.10.0.
v0.9.2 (2026-10-14)
- Apps are laid out as
apps/<slug>/vN/: one folder per app, one inside it per version, each app numbered from v1.scaffold new app <slug>gives a new slug itsv1and an existing slug its next version. Before, a study had one number sequence and folders namedapps/vN-<slug>/, which put unrelated apps side by side in number order and split one app's versions apart. - The earlier layout still passes
check. An[[apps]]entry with nopathis looked for atapps/<slug>/<version>/and then atapps/<version>-<slug>/. No repo has to move anything. - The app template's
serve.pylooks one folder further up for the design system, to match the new depth. - Needs the preregistration kit release that reads the same layout (adaptive-preregistration, unreleased).
v0.9.1 (2026-10-13)
- Research questions go to the retrieval tool first. A profile can name the organisation's literature and
evidence search (
[retrieval]:server, the MCP server's name in a session, andrepo). When it does, the scaffold's section ofAGENTS.mdin every kind of repo tells every assistant to search with it before answering a question about science, the literature or evidence, to say so when the tool isn't available, and not to use it when the person says so. A profile that names none gets no such rule. - Pins scientific-research-agents v0.3.0 (adds a local-model roles template; the coordination protocol points to it).
- Generated CI checks out
v0.9.1.
v0.9.0 (2026-10-07)
- Exploratory units.
scaffold new app|sim|tool|dataset|calculator ... --exploratorycreates the unit without a claim: it writes the unit'sSCOPE.tomlat the exploratory stage, with a build note taken from--question(or--job) and the date, and labels the unit's README. Sessions don't ask for a claim or an evidence basis in such a unit. A study or workbench can't be exploratory. checkfails an exploratory unit that doesn't say so in its README, or that has preregistered experiments: it gets its claim (andstage = "testing") first.- The claim step is one question for every other new unit, from preregistration kit 0.9.0: the claim and
what would count against it. The shared rules in
AGENTS.mdsay so for each kind. - Generated CI checks out
v0.9.0.
v0.8.1 (2026-10-06)
- Generated CI can install private dependencies. A repo that pins the organisation's private repos by git
URL failed at
uv syncin GitHub Actions (could not read Username for 'https://github.com'), because the default token reads only its own repo, andmake checknever ran. The workflow of every kind now reads an optionalPRIVATE_DEPS_TOKENsecret and, when it is set, uses it for the profile'sorgon github.com. Without the secret, an install that fails in a repo with such pins is a warning and the scaffold check still runs; any other failed install still fails the job. See PROTOCOL.md § 4. - Generated CI checks out
v0.8.1.
v0.8.0 (2026-10-02)
- The workbench kind.
WORKBENCH.toml: one repo holding generaltools/,sims/andapps/and one folder per study.scaffold new workbenchcreates one;new study --inside <workbench>creates a study that shares the repo's kit, agents, CI and licence;new tool|sim --inside <study or workbench>adds units to either. A workbench has no one stage or corpus project, may mount the corpus as a submodule, and declares itsprivatefolders, which requires a private repo.checkon a workbench checks every study it lists. - A study may hold its own tools (
[[tools]]with apath); a tool inside a study is a unit, not a tool repo. - A study inside a workbench carries no
visibilityof its own: the workbench decides it. - Adopted units. A
[[datasets]]or[[calculators]]entry withadopted = truepredates the unit formats:checkrequires only that it exists, and warns that it owes its manifest when next worked on. - Generated CI checks out
v0.8.0.
v0.7.0 (2026-10-01)
- Pins scientific-research-agents v0.2.0, with provider-neutral role bodies and Claude Code subagent frontmatter retained as an adapter.
- Makes
AGENTS.mdthe shared instruction surface for Claude, ChatGPT, Codex and other repository-aware assistants. Per-kind scientific rules now live in its scaffold-managed block; newCLAUDE.mdfiles import it without duplication, while existing Claude customizations remain untouched. - Packages
new-studyas a portable ChatGPT/Codex skills-only plugin and documents Claude and Codex skill installation separately. - Generated CI checks out
v0.7.0.
v0.6.0 (2026-10-01)
- Needs adaptive-preregistration v0.7.0 or later. Generated CI checks out
v0.6.0. - New tools start at
0.1.0, in preparation (an undated changelog heading), sorelease-checkandcheckagree. - Tools use kit v0.7.0.
new toolinstallstool-versioning(releases: annotated tags, outputs changed, consumers,release-check) in place of the profile's preregistration modules;prereg, a kit default, now scopes the tool claim first (tool-scopewas merged into it). A tool's override experiments are preregistered in the research corpus.checkwarns while a tool with the scope protocol has noSCOPE.toml.
v0.5.1 (2026-09-30)
- Adopted repos stay adopted.
.scaffold.lockrecords"adopted": truefor a repo that took up the scaffold rather than being made by it, andupdatenever adds a missing file to such a repo unasked. Before, a lock written by an adoptingupdatemade the nextupdatetreat the repo as scaffold-made and add its CI andMakefile.
v0.5.0 (2026-09-30)
- Needs adaptive-preregistration v0.5.0 or later, whose
tool-scopeapplies to tool repositories. - Generated CI checks out
v0.5.0. - Tools are scoped.
new toolinstalls the profile's[prereg] tool_modules(tool-scope) in place of its preregistration modules, so a tool's claim, features and every scientific feature's evidence and decision are settled with the person first (SCOPE.tomlat the root; kittool-scopenow applies to tool repositories).checkwarns while a scoped tool has noSCOPE.toml.
v0.4.0 (2026-09-30)
- Generated CI checks out
v0.4.0;updatemoves existing repos' CI to it, andUPGRADING.mdsays what else to do. - Repos notice new releases.
UPGRADING.mdsays, per release, whatscaffold updatedoes and what a session does by hand.scaffold versioncompares a repo's recorded release with the current one (and with GitHub's latest) and prints the steps in between; a session hook in.claude/settings.jsonruns it when a Claude Code session starts, beside the kit's hooks, and theAGENTS.mdsection tells Codex to.updateadds the hook, and prints the upgrade notes with its plan. - Work types replace
experiments/. A study is built from apps, sims, datasets, calculators and tools (PROTOCOL.md § 5), each with its own folder, entry requirement and way of writing back to research.scaffold new datasetcreatesdatasets/<slug>/(a dataset-fetch reference pinned to a version, selection criteria,analysis/);scaffold new calculatorcreatescalculators/<slug>/(CALCULATOR.md,reference.csv). Preregistrations live in the unit they test,<unit>/preregistrations/<EID>/, listed inEXPERIMENTS.md; sims usepreregistrations/too.checkenforces a selected dataset's pinned reference and licence, and calculator reference values from PROBE on;experiments/,data/manifest.jsonand unregistered unit folders are warnings. PROTOCOL.md sections from Apps on are renumbered. - The agents moved to their own repo, scientific-research-agents,
included as a submodule at
agents/pinned tov0.1.1: the briefs, the coordination protocol, discipline briefs, prices and the usage tool.scaffold usagepasses through to itsbin/agents usage. Thenew-studyskill moved toskills/. Clone with--recurse-submodules. - Agent coordination.
agents/COORDINATION.md: one lead delegates, agents pass 150-word handoffs instead of transcripts, parallel work only for independent tasks (at most three, each in its own worktree), agent teams only for tight back-and-forth, and the science gate never skipped. - Mathematician agent. Equations, constants, units and valid ranges; records each piece of maths with its source (BioNumbers style) and specifies calculators with reference values, handing code to the engineer.
- Model routing. Each agent's frontmatter now sets its starting
modelandeffort(a newscouton Haiku; the others on Sonnet), overridable in a profile's[agents.routing]; the lead escalates a task on a clear signal. - Codex. A marked scaffold section in
AGENTS.mdgives every assistant the roles, the coordination rules and Codex routing from the profile's[agents.codex].updatemaintains it and leaves the rest of the file alone. scaffold usage. Tokens and API-equivalent cost per repo and model from Claude Code's and Codex's local logs, a monthly projection, and Codex's plan rate-limit windows. Prices are inagents/prices.toml. Each response is counted once: Claude Code writes one response as several transcript lines.
v0.3.0 (2026-09-30)
- The tool kind. A third kind of repo, for one job done for other repos with no research in it:
TOOL.tomlwith ajobin place of a question, aversion, and[[consumers]]naming what each dependent repouses.scaffold new tool <name> --job "..."creates one.checkfails a tool that holds research (a question,[corpus],RESEARCH.md,EXPERIMENTS.md,experiments/), whose version differs acrossTOOL.toml,pyproject.toml,CITATION.cffandCHANGELOG.md, or whose consumers don't say what they use. Modelled on theTOOL.tomlan existing tool repo already used; PROTOCOL.md § 7. scaffold statusandscaffold update.statusgives one line per repo: kind, stage, scaffold release, CI ref, check result and pending update.updatebrings a repo's scaffold-owned files (the CI workflow,Makefile,shared/workbench.py) up to this release. It replaces only files nobody edited, known from the new.scaffold.lockthatnewwrites, or by matching what an earlier release wrote. In an edited workflow it moves only the scaffoldref:; other edited files are kept unless--adopted. A dry run unless--apply; refuses to overwrite uncommitted work; never commits. PROTOCOL.md § 12.- Specialist agents. Every new repo gets Claude Code subagents in
.claude/agents/:computational-engineer,designer,frontend-developer,researcher,analystandscience-writer(a sim has no UI agents), plus.claude/disciplines.md. A profile's new[agents]table chooses the discipline briefs (agents/disciplines/) and the code conventions written into the agents. The agents are scaffold-owned, soupdatekeeps them current;--adoptnow takes a folder, to add them to an existing repo. GeneratedCLAUDE.mdsays which agent takes which task, and in what order. - Studies and sims list the tools they use under
[[tools]], andcheckfails a tool pinned to a branch. - PROTOCOL.md sections from Preregistration on are renumbered by one (Preregistration is now § 8).
- The built-in TOML parser (Python < 3.11) reads arrays that span lines, and decodes non-ASCII strings
correctly; before,
"µV"came back garbled. - Generated CI checks out
v0.3.0, andupdatemoves existing repos' CI to it.
v0.2.0 (2026-09-29)
- No default profile.
scaffold newtakes--profile NAMEor$SCAFFOLD_PROFILE, and fails with the list of profiles if neither is set. Before, it silently usedpolarizetech. - Visibility is never defaulted. New manifests say
visibility = "undecided"unless--visibility public|privateis given, andscaffold checkfails onundecided. Before, the template wrote"private"with today's date, which passedcheckwithout anyone deciding. - Generated study and sim CI checks out
v0.2.0.
v0.1.0 (2026-09-27)
First release.
- The study protocol, the adoption guide, study/sim/app templates, and
scaffold(new,promote-sim,check,profiles), with theexampleandpolarizetechprofiles. - Generated study and sim CI checks out this release (
v0.1.0) rather thanmain.