databricks-core-workflow-b

Featured

Execute Databricks secondary workflow: MLflow model training and deployment. Use when building ML pipelines, training models, or deploying to production. Trigger with phrases like "databricks ML", "mlflow training", "databricks model", "feature store", "model registry".

AI & Automation 2,266 stars 315 forks Updated today MIT

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Databricks Core Workflow B: MLflow Training & Serving ## Overview Full ML lifecycle on Databricks: Feature Engineering Client for discoverable features, MLflow experiment tracking with auto-logging, Unity Catalog model registry with aliases (`champion`/`challenger`), and Mosaic AI Model Serving endpoints for real-time inference via REST API. ## Prerequisites - Completed `databricks-install-auth` and `databricks-core-workflow-a` - `databricks-sdk`, `mlflow`, `scikit-learn` installed - Unity Catalog enabled (required for model registry) ## Instructions ### Step 1: Feature Engineering with Feature Store Create a feature table in Unity Catalog so features are discoverable and reusable. ```python from databricks.feature_engineering import FeatureEngineeringClient from pyspark.sql import SparkSession import pyspark.sql.functions as F spark = SparkSession.builder.getOrCreate() fe = FeatureEngineeringClient() # Build features from gold layer tables user_features = ( spark.table("prod_catalog.gold.user_events") .groupBy("user_id") .agg( F.count("event_id").alias("total_events"), F.avg("session_duration_sec").alias("avg_session_sec"), F.max("event_timestamp").alias("last_active"), F.countDistinct("event_type").alias("unique_event_types"), F.datediff(F.current_date(), F.max("event_timestamp")).alias("days_since_last_active"), ) ) # Register as a feature table (creates or updates) fe.create_table( name="prod_catalog.ml...

Details

Author
jeremylongshore
Repository
jeremylongshore/claude-code-plugins-plus-skills
Created
7 months ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

databricks-core-workflow-a

Execute Databricks primary workflow: Delta Lake ETL pipelines. Use when building data ingestion pipelines, implementing medallion architecture, or creating Delta Lake transformations. Trigger with phrases like "databricks ETL", "delta lake pipeline", "medallion architecture", "databricks data pipeline", "bronze silver gold".

2,266 Updated today
jeremylongshore
AI & Automation Featured

mlflow

Track ML experiments, manage model registry with versioning, deploy models to production, and reproduce experiments with MLflow - framework-agnostic ML lifecycle platform

27,681 Updated today
davila7
AI & Automation Solid

mlflow

Track ML experiments, manage model registry with versioning, deploy models to production, and reproduce experiments with MLflow - framework-agnostic ML lifecycle platform

9,117 Updated 1 months ago
Orchestra-Research
AI & Automation Solid

ml-pipeline

Designs and implements production-grade ML pipeline infrastructure: configures experiment tracking with MLflow or Weights & Biases, creates Kubeflow or Airflow DAGs for training orchestration, builds feature store schemas with Feast, deploys model registries, and automates retraining and validation workflows. Use when building ML pipelines, orchestrating training workflows, automating model lifecycle, implementing feature stores, managing experiment tracking systems, setting up DVC for data versioning, tuning hyperparameters, or configuring MLOps tooling like Kubeflow, Airflow, MLflow, or Prefect.

9,509 Updated 1 weeks ago
Jeffallan
AI & Automation Solid

mle-workflow

Production machine-learning engineering workflow for data contracts, reproducible training, model evaluation, deployment, monitoring, and rollback. Use when building, reviewing, or hardening ML systems beyond one-off notebooks.

199,470 Updated today
affaan-m