uw-ssec/zarr-data-format
Comprehensive agents and skills for working with the Zarr array storage format — creation, compression, cloud backends, xarray integration, and data migration
Configures Zarr stores on cloud object storage, covering S3, GCS, and Azure Blob backends via fsspec (s3fs, gcsfs, adlfs), the high-performance Rust-based obstore library, and Icechunk for versioned cloud storage. Covers authentication patterns, caching strategies, and performance tuning for remote I/O. Use when the user asks about reading or writing Zarr data on S3, GCS, or Azure, or mentions fsspec, obstore, or Icechunk for remote Zarr access.
Configures and optimizes compression for Zarr arrays. Covers all numcodecs compressors (Blosc, Zstd, LZ4, Gzip, LZMA, BZ2), pre-compression filters (Delta, Quantize, FixedScaleOffset, PackBits), codec pipelines, Blosc thread safety, and the trade-offs between compression speed and ratio. Use when the user asks about configuring compression for Zarr arrays, choosing numcodecs compressors, optimizing compression settings for chunked array storage, or debugging codec-related corruption or performance issues.
Migrates data between formats with a focus on converting HDF5 and NetCDF datasets to Zarr. Covers xarray-based conversion, direct zarr.copy operations, VirtualiZarr for reference-based virtual Zarr stores, kerchunk for legacy workflows, validation strategies, and batch migration pipelines. Use when the user asks about converting HDF5 or NetCDF files to Zarr, migrating scientific datasets, or working with VirtualiZarr or kerchunk for virtual stores.
Work with the Zarr array storage format for chunked, compressed, N-dimensional arrays. Covers array creation, hierarchical groups, metadata/attributes, advanced indexing modes, data types, thread/process safety, sharding, and Zarr v2 vs v3 differences. Use when the user asks about Zarr arrays, .zarr files, zarr-python, zarr stores, chunked array storage, or cloud-optimized scientific array storage.
Integrates Zarr with xarray and Dask for labeled, multi-dimensional scientific data workflows. Covers reading and writing Zarr stores with xarray, append and region-write operations, multi-file virtual datasets, Dask chunk alignment with Zarr chunks, encoding configuration, consolidated metadata, and performance optimization. Use when the user asks about reading or writing Zarr stores with xarray, aligning Dask chunks with Zarr arrays, or optimizing large-scale multi-dimensional data pipelines.