gcp-managed-spark-upgrades
Upgrades GCP Spark/Dataproc jobs to newer versions by analyzing, remediating, and testing job on target image version. Use when: - Upgrading Spark/Dataproc versions (major, minor, or sub-minor). Example prompt: "Upgrade Dataproc job [job_id] in project [project_id], region [region] to target image version [target_version]. Temporary staging: [gcs_staging_path], Target GCS bucket: [gcs_target_path]" Don't use when: - Writing new Spark code. - Migrating non-Spark workloads (e.g., Hive/Flink to Spark). Limitation: This skill does not support data/result validation currently.
- Version
- v1
- License
- Apache-2.0
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/gcp-managed-spark-upgrades/SKILL.md
- skills/gcp-managed-spark-upgrades/LICENSE
- skills/gcp-managed-spark-upgrades/references/cluster_input_validations.md
- skills/gcp-managed-spark-upgrades/references/cluster_verifications.md
- skills/gcp-managed-spark-upgrades/references/pyspark_analysis.md
- skills/gcp-managed-spark-upgrades/references/spark_migration_reference.md
- skills/gcp-managed-spark-upgrades/scripts/cluster_resolve_version.py
Every link opens the file at its source, pinned to the revision this page describes.