duo-measurement-validity
Design and report a number that survives an outsider's scrutiny — prove the outcome on an instrument you did not build, break a headline score into components only when they add information, publish the component where you lose, audit bias at intersections instead of one variable at a time, and disclose exactly where your own hand touched the evidence. Also covers the inference traps that make an instrument lie — classifier categories that encode their training population, resemblance mistaken for lineage, non-independent samples, contested definitions. Use when someone asks will this number survive outside scrutiny, is our metric actually valid, should we ship a subscore, why doesn't our completion rate convince buyers, how do we prove our product caused this, is our model biased, can we trust this benchmark, or how do we measure something nobody can measure directly.
- Version
- 2.0.0
- License
- MIT
Pinned to revision 78072c9528fb, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/duo-measurement-validity/SKILL.md
- skills/duo-measurement-validity/references/audit-bias-at-intersections.md
- skills/duo-measurement-validity/references/benchmark-on-an-instrument-you-did-not-build.md
- skills/duo-measurement-validity/references/classifier-categories-encode-their-training-population.md
- skills/duo-measurement-validity/references/clean-cohort-buys-attribution-costs-generality.md
- skills/duo-measurement-validity/references/measure-willingness-alongside-capability.md
- skills/duo-measurement-validity/references/publish-the-component-where-you-lose.md
- skills/duo-measurement-validity/references/resemblance-is-not-lineage.md
- skills/duo-measurement-validity/references/sample-cases-that-could-not-have-influenced-each-other.md
- skills/duo-measurement-validity/references/ship-a-subscore-only-if-it-adds-information.md
- skills/duo-measurement-validity/references/triangulate-weak-independent-sources.md
Every link opens the file at its source, pinned to the revision this page describes.