skillshop-ostyles/agents-ai-ml

Find model-based decision points and check if they are logged with sufficient context. Trigger: /ai-log

Embedding quality scanner: audit chunking strategy, model selection, and embedding configuration. Read-only. Trigger: /embed-quality

Find fine-tuned model references and check base model deprecation status. Trigger: /finetune-deps

Find LLM API calls that lack observability - no logging, error handling, timeout, or cost tracking. Trigger: /llm-obs

LLM cost controller: audits all LLM API calls in a codebase, detects cost anti-patterns (expensive models, unlimited tokens, no caching, batchable calls), and estimates monthly spend with optimization savings. Read-only. Trigger: /llm-cost

Find sources of non-determinism in ML training pipelines. Trigger: /ml-determinism

Model output guardrail auditor: find unvalidated LLM outputs that cause crashes, data corruption, or bad decisions. Read-only. Trigger: /guardrails

Track prompt changes across git history and flag drift that affects output quality or safety. Trigger: /prompt-drift

Prompt injection vulnerability scanner: statically detects LLM API call sites, traces untrusted data flowing into system prompts and user messages, and classifies injection countermeasures (none/weak/adequate). Read-only. Trigger: /prompt-inspect

Prompt quality auditor: audit every prompt for clarity, safety, and injection resistance. Read-only. Trigger: /prompt-quality

Audit RAG pipeline configuration for consistency issues that produce bad answers. Trigger: /rag-consistency

Analyze static code for token usage patterns, waste, and budget risks. Trigger: /token-budget

Check tool/function definitions for hallucination-prone schemas. Trigger: /tool-fidelity

Training data leakage detector: find cross-contamination between train/test splits in ML pipelines. Read-only. Trigger: /train-leak