gcp-dataflow
Guides writing, packaging, executing, and troubleshooting Apache Beam pipelines on Dataflow. Use when creating new pipelines, configuring Flex Templates, or analyzing performance of Dataflow jobs. Capabilities include Java/Python/Go setup, Cloud Build integration, and deep diagnostic analysis of job health and autoscaling. Use when: - Creating an Apache Beam Dataflow pipeline. - Creating a Google Dataflow Flex Template. - Using an existing Google Dataflow Template. - Debugging Dataflow pipeline - Troubleshooting Dataflow pipeline - Analyzing Performance of Dataflow pipeline. Key capabilities: Java/Python/Go project setup, Flex Templates (with Cloud Build), and diagnostics for streaming job health, bottlenecks, and autoscaling. Do NOT use for: - General GCP resource management unrelated to Dataflow. - Issues with other GCP services (e.g., GCE, GCS, BigQuery) unless directly impacting Dataflow pipeline execution. - Pipeline technologies other than Apache Beam on Dataflow.
- Version
- v4
- License
- Apache-2.0
Pinned to revision db28b191f5db, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/gcp-dataflow/SKILL.md
- skills/gcp-dataflow/references/bottlenecks_and_parallelism_context.md
- skills/gcp-dataflow/references/dataflow_diagnostics_reference.md
- skills/gcp-dataflow/references/dataflow_metrics_bigquery.md
- skills/gcp-dataflow/references/dataflow_metrics_core_job.md
- skills/gcp-dataflow/references/dataflow_metrics_pubsub.md
- skills/gcp-dataflow/references/dataflow_metrics_streaming_engine.md
- skills/gcp-dataflow/references/python_flex_template_reference.md
- skills/gcp-dataflow/references/streaming_horizontal_autoscaling_analysis.md
- skills/gcp-dataflow/references/streaming_job_health.md
Every link opens the file at its source, pinned to the revision this page describes.