control-limits-and-error-costs

Set the threshold for a monitored measurement or quality check, balancing false alarms against missed problems. Always use this skill when someone asks where to put an alert limit, control limit, pass or fail cut-off or significance level for a process or metric; how costly a false alarm is compared with a miss; the chance a batch, pack, shipment or day breaches a specification or legal limit, even as a quick calculation from an average and spread; what target average keeps output inside a limit; or how quickly a shift in the process would be detected. Also use it for control charts, acceptance sampling, type I and type II errors, power to detect a shift, and choosing alpha from business costs. Not for setting a classifier's probability cut-off, not for deciding alert volumes from a review team's capacity, and not for sizing an A/B test.