gcp-managed-spark-upgrades
Upgrades Spark workloads on Google Cloud Managed Service for Apache Spark (formerly known as Dataproc) clusters and Serverless to a newer image or runtime version. Supports static compatibility analysis, code upgrade, build checks against the target version, and optional verification runs. Use when: - Upgrading Managed Spark versions (major, minor, or sub-minor) or assessing upgrade readiness. Example prompt: "Upgrade Managed Spark job [job_id] in project [project_id], region [region] to target version [target_version]. Code location: [local path or GCS path]. Target GCS path: [gcs_target_path]" Don't use when: - Writing new Spark code. - Migrating non-Spark workloads (e.g., Hive/Flink to Spark). - Troubleshooting a failed Spark job that is not part of an upgrade. Limitation: This skill does not support data/result validation currently.
- Version
- v3
- License
- Apache-2.0
Pinned to revision 716f429da267, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/gcp-managed-spark-upgrades/SKILL.md
- skills/gcp-managed-spark-upgrades/references/cluster_input_validations.md
- skills/gcp-managed-spark-upgrades/references/cluster_verifications.md
- skills/gcp-managed-spark-upgrades/references/java_analysis.md
- skills/gcp-managed-spark-upgrades/references/jvm_build_reference.md
- skills/gcp-managed-spark-upgrades/references/pyspark_analysis.md
- skills/gcp-managed-spark-upgrades/references/scala_analysis.md
- skills/gcp-managed-spark-upgrades/references/serverless_input_validations.md
- skills/gcp-managed-spark-upgrades/references/serverless_verifications.md
- skills/gcp-managed-spark-upgrades/references/spark_migration_reference.md
- skills/gcp-managed-spark-upgrades/scripts/fetch_migration_notes.py
- skills/gcp-managed-spark-upgrades/scripts/resolve_version.py
Every link opens the file at its source, pinned to the revision this page describes.