model-serving

Deploy and serve ML models in production. Covers REST API serving (FastAPI, Flask), gRPC, BentoML, NVIDIA Triton, TorchServe, TF Serving, Seldon Core, KServe on Kubernetes, batch inference (Spark, Ray, Dask), model optimization for serving (ONNX Runtime, TensorRT, quantization, pruning), A/B testing, canary and blue-green deployments, shadow deployments, request batching, auto-scaling, containerization with Docker, health checks, input validation, Prometheus metrics, caching, multi-model serving, and model ensembles. Use when deploying models, building inference APIs, or optimizing serving performance.

Version

1.0

License

Apache-2.0