Skip to content

khasa1996/aws-data-analytics

v1.0.0Apache-2.0

Data lake, analytics, and ETL workflows with S3 Tables, AWS Glue, and Athena. Covers managed Iceberg tables on S3 Tables, ingestion from JDBC databases, Amazon Redshift, Snowflake, BigQuery, and DynamoDB, AWS Glue Data Catalog inventory and asset discovery, federated Athena queries, and vector storage and semantic search on Amazon S3 Vectors.

upgrading-mwaa-environments

Upgrades an MWAA environment to a newer Airflow version — within 2.x, within 3.x, or across the 2.x-to-3.x boundary. Computes the version-jump path, inserting the 2.11.x stepping-stone and Python-transition step when needed. Chooses an approach by whether run history and the same environment (URL/ARN) must be kept: a new-environment upgrade (blue-green), a rehearsed in-place upgrade validated on a test copy, or a direct in-place upgrade. Runs Ruff scanning and deprecation-warning log scans for 3.x moves, plus Docker validation, batched deployment, and switchover. Saves a resumable upgrade plan for multi-session work. Triggers on: upgrade MWAA, upgrade Airflow, migrate to Airflow 3, MWAA Airflow 3, Airflow 2 to 3, preserve Airflow history, keep same MWAA environment, blue-green cutover, validation environment before cutover. Not for authoring new DAGs (authoring-mwaa-workflow), debugging unrelated DAG failures (debugging-mwaa-workflow), or MWAA Serverless YAML workflows (provisioned Python DAGs only).

Version
1
Read SKILL.md at the source

Pinned to revision a2c98f2037dc, so it is the text this page describes rather than whatever the author pushed since.

Files

Every link opens the file at its source, pinned to the revision this page describes.