ml-pipeline

Featured

Designs and implements production-grade ML pipeline infrastructure: configures experiment tracking with MLflow or Weights & Biases, creates Kubeflow or Airflow DAGs for training orchestration, builds feature store schemas with Feast, deploys model registries, and automates retraining and validation workflows. Use when building ML pipelines, orchestrating training workflows, automating model lifecycle, implementing feature stores, managing experiment tracking systems, setting up DVC for data versioning, tuning hyperparameters, or configuring MLOps tooling like Kubeflow, Airflow, MLflow, or Prefect.

AI & Automation 11,089 stars 1052 forks Updated 1 weeks ago MIT

Install

View on GitHub

Quality Score: 94/100

Stars 20%
100
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# ML Pipeline Expert Senior ML pipeline engineer specializing in production-grade machine learning infrastructure, orchestration systems, and automated training workflows. ## Core Workflow 1. **Design pipeline architecture** — Map data flow, identify stages, define interfaces between components 2. **Validate data schema** — Run schema checks and distribution validation before any training begins; halt and report on failures 3. **Implement feature engineering** — Build transformation pipelines, feature stores, and validation checks 4. **Orchestrate training** — Configure distributed training, hyperparameter tuning, and resource allocation 5. **Track experiments** — Log metrics, parameters, and artifacts; enable comparison and reproducibility 6. **Validate and deploy** — Run model evaluation gates; implement A/B testing or shadow deployment before promotion ## Reference Guide Load detailed guidance based on context: | Topic | Reference | Load When | |-------|-----------|-----------| | Feature Engineering | `references/feature-engineering.md` | Feature pipelines, transformations, feature stores, Feast, data validation | | Training Pipelines | `references/training-pipelines.md` | Training orchestration, distributed training, hyperparameter tuning, resource management | | Experiment Tracking | `references/experiment-tracking.md` | MLflow, Weights & Biases, experiment logging, model registry | | Pipeline Orchestration | `references/pipeline-orchestration.md` | Kubeflow Pipeli...

Details

Author
Jeffallan
Repository
Jeffallan/claude-skills
Created
10 months ago
Last Updated
1 weeks ago
Language
Python
License
MIT

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

ml-pipeline

Use when building ML pipelines, orchestrating training workflows, automating model lifecycle, implementing feature stores, or managing experiment tracking systems.

3 Updated yesterday
zacklecon
AI & Automation Solid

ml-pipeline-creation

Design, implement, and validate reproducible machine-learning pipelines spanning data preparation, training, evaluation, registry, and deployment gates. Use when the user requests an ML pipeline, needs to turn model scripts into an orchestrated workflow, or provides pipeline components that must be connected safely.

161 Updated 1 weeks ago
seb1n
AI & Automation Solid

ml-engineer

Build production ML systems: model training pipelines, serving infrastructure, performance optimization, and automated retraining. Use when: (1) designing or building ML pipelines (data validation → training → deployment), (2) optimizing model training (hyperparameter search, distributed training, checkpointing), (3) deploying models to production (REST/gRPC endpoints, batch/stream processing, canary releases), (4) setting up ML monitoring (prediction drift, feature drift, performance decay), (5) implementing feature engineering or feature stores, (6) automating retraining triggers, (7) debugging model performance or serving latency issues. Triggers on: ML pipeline, model training, model serving, feature engineering, hyperparameter tuning, model deployment, inference optimization, model monitoring, MLOps, retraining.

39 Updated today
OpenCoven