coreweave-core-workflow-a

# CoreWeave Core Workflow: KServe Inference ## Overview Deploy production inference services on CoreWeave using KServe InferenceService with GPU scheduling, autoscaling, and scale-to-zero. CKS natively integrates with KServe for serverless GPU inference. ## Prerequisites - Completed `coreweave-install-auth` setup - KServe available on your CKS cluster - Model stored in S3, GCS, or HuggingFace ## Instructions ### Step 1: Deploy an InferenceService ```yaml # inference-service.yaml apiVersion: serving.kserve.io/v1beta1 kind: InferenceService metadata: name: llama-inference annotations: autoscaling.knative.dev/class: "kpa.autoscaling.knative.dev" autoscaling.knative.dev/metric: "concurrency" autoscaling.knative.dev/target: "1" autoscaling.knative.dev/minScale: "1" autoscaling.knative.dev/maxScale: "5" spec: predictor: minReplicas: 1 maxReplicas: 5 containers: - name: kserve-container image: vllm/vllm-openai:latest args: - "--model" - "meta-llama/Llama-3.1-8B-Instruct" - "--port" - "8080" ports: - containerPort: 8080 protocol: TCP resources: limits: nvidia.com/gpu: "1" memory: 48Gi cpu: "8" requests: nvidia.com/gpu: "1" memory: 32Gi cpu: "4" env: - name: HUGGING_FACE_HUB_TOKEN valueFrom: secretKeyRef: ...