gcp-spark-troubleshooting
Provides expert guidance for troubleshooting Google Cloud Spark and Dataproc workloads (Dataproc Serverless batches, interactive sessions, and standard Dataproc clusters), and inspecting, streaming, searching, tailing, or summarizing Spark driver outputs and event logs in Cloud Storage. Use when debugging, diagnosing, or performing Root Cause Analysis (RCA) on failed or slow Spark jobs, PySpark scripts (.py), JVM archives (.jar), compressed event logs (.zst), or IAM, VPC, and classpath errors such as OutOfMemoryError, 403 PERMISSION_DENIED, NoSuchMethodError, and ClassNotFoundException. Do NOT use for authoring new Spark jobs (use gcp-spark), planning or performing runtime version upgrades (use gcp-managed-spark-upgrades), or non-Spark engines such as Dataflow or BigQuery Data Transfer Service.
- Version
- v8
- License
- Apache-2.0
Pinned to revision 716f429da267, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/gcp-spark-troubleshooting/SKILL.md
- skills/gcp-spark-troubleshooting/references/diagnostic_rules_catalog.md
- skills/gcp-spark-troubleshooting/references/gcs_event_log_guide.md
- skills/gcp-spark-troubleshooting/references/ingestion_triage_matrix.md
- skills/gcp-spark-troubleshooting/references/rca_report_template.md
- skills/gcp-spark-troubleshooting/references/spark_env_inspection.md
- skills/gcp-spark-troubleshooting/references/spark_log_patterns.md
- skills/gcp-spark-troubleshooting/scripts/spark_applications.py
- skills/gcp-spark-troubleshooting/scripts/spark_code_inspector.py
- skills/gcp-spark-troubleshooting/scripts/spark_env_inspector.py
- skills/gcp-spark-troubleshooting/scripts/spark_gcs_log_reader.py
- skills/gcp-spark-troubleshooting/scripts/spark_stage_diagnostics.py
- skills/gcp-spark-troubleshooting/tests/ingestion_triage_matrix_test.py
- skills/gcp-spark-troubleshooting/tests/spark_applications_test.py
- skills/gcp-spark-troubleshooting/tests/spark_code_inspector_test.py
- skills/gcp-spark-troubleshooting/tests/spark_env_inspector_test.py
- skills/gcp-spark-troubleshooting/tests/spark_gcs_log_reader_test.py
- skills/gcp-spark-troubleshooting/tests/spark_stage_diagnostics_test.py
Every link opens the file at its source, pinned to the revision this page describes.