aidp-check-data
Pre-migration data-availability scan. Reads every notebook in a migration manifest, extracts every spark.read.table / spark.read.parquet / saveAsTable reference, and probes whether each target schema/table/path exists on the AIDP cluster BEFORE you spend Pass-2 cluster time. Use after aidp-build-dag and before aidp-migrate-job, especially the first time you migrate against a target environment.
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/aidp-check-data/SKILL.md
- skills/aidp-check-data/LICENSE
- skills/aidp-check-data/engine/scripts/check_data_availability.py
- skills/aidp-check-data/engine/scripts/check_data_availability_for_workflow.py
Every link opens the file at its source, pinned to the revision this page describes.