eks-genailisted

Use whenever someone is building, training, fine-tuning, or serving a generative AI / LLM workload on Amazon EKS — phrased as "GPU vs Trainium/Inferentia", "vLLM on EKS", "Ray Serve / KubeRay", "distributed training on EKS", "FSx for Lustre for ML", "Karpenter for GPU", "EFA / NCCL multi-node", "DCGM / Neuron Monitor", "LiteLLM / AI gateway", "RAG on EKS", "agentic AI on EKS", or "self-host Llama / Mistral / Qwen". Walks the opinionated 6-layer stack (compute → cluster/scheduler → frameworks → storage → observability → AI gateway), the GPU-vs-Neuron decision, the JARK + vLLM + LiteLLM canonical reference, KV-cache tiering, cost levers (Neuron, Spot, Capacity Blocks), and a non-negotiable security baseline. Trigger even if "GenAI" is never said — any GPU/Neuron, inference-serving, or distributed-training decision on EKS qualifies. Skip for SageMaker-only or Bedrock-only (no self-hosting) asks, and for generic cluster design/build with no AI/ML workload (use eks-design / eks-build).
aws-samples/sample-apex-skills · ★ 37 · AI & Automation · score 73

Install: claude install-skill aws-samples/sample-apex-skills

# GenAI on Amazon EKS End-to-end opinionated guidance for running generative AI / LLM workloads on Amazon EKS, structured as a **6-layer stack** from compute hardware up through the AI gateway. This skill is opinionated: it recommends one AWS-canonical reference stack — the **JARK stack** (JupyterHub + Argo + Ray + Karpenter) extended with **vLLM** serving and a **LiteLLM** gateway — and surfaces the alternatives plus the customer-context flags that justify deviating. Two sources are the canonical foundation and every recommendation must align with one or both: the [EKS AI/ML Best Practices guide](https://docs.aws.amazon.com/eks/latest/best-practices/aiml.html) and the [`awslabs/ai-on-eks`](https://github.com/awslabs/ai-on-eks) reference implementation. For "how do I run a single EKS cluster well" (compute, networking, upgrades) use `eks-best-practices`; for designing/building the cluster itself use `eks-design` / `eks-build`. This skill is the GenAI *workload* layer on top. ## When to Use This Skill **Activate when the user wants to:** - Choose GenAI hardware — NVIDIA GPU (p5/g6/g6e) vs AWS Neuron (Trainium/Inferentia) - Serve an LLM (vLLM, Ray Serve, Triton, KServe) or run distributed training/fine-tuning on EKS - Design GPU/Neuron scheduling (Karpenter NodePools, device plugins, EFA, Capacity Blocks, Spot) - Pick ML storage (FSx for Lustre, Mountpoint S3 CSI, EFS, S3 Vectors) or wire GPU/Neuron observability - Stand up an AI gateway (LiteLLM/Envoy), RAG pipeline, or ag