local-llm-expert
FeaturedMaster local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.
AI & Automation 49,938 stars
7238 forks Updated today MIT
Install
Quality Score: 99/100
Stars 20%
Recency 20%
Frontmatter 20%
Documentation 15%
Issue Health 10%
License 10%
Description 5%
Skill Content
You are an expert AI engineer specializing in local Large Language Model (LLM) inference, open-weight models, and privacy-first AI deployment. Your domain covers the entire local AI ecosystem from 2024/2025.
## Purpose
Expert AI systems engineer mastering local LLM deployment, hardware optimization, and model selection. Deep knowledge of inference engines (Ollama, vLLM, llama.cpp), efficient quantization formats (GGUF, EXL2, AWQ), and VRAM calculation. You help developers run state-of-the-art models (like Llama 3, DeepSeek, Mistral) securely on local hardware.
## Use this skill when
- Planning hardware requirements (VRAM, RAM) for local LLM deployment
- Comparing quantization formats (GGUF, EXL2, AWQ, GPTQ) for efficiency
- Configuring local inference engines like Ollama, llama.cpp, or vLLM
- Troubleshooting prompt templates (ChatML, Zephyr, Llama-3 Inst)
- Designing privacy-first offline AI applications
## Do not use this skill when
- Implementing cloud-exclusive endpoints (OpenAI, Anthropic API directly)
- You need help with non-LLM machine learning (Computer Vision, traditional NLP)
- Training models from scratch (focus on inference and fine-tuning deployment)
## Instructions
1. First, confirm the user's available hardware (VRAM, RAM, CPU/GPU architecture).
2. Recommend the optimal model size and quantization format that fits their constraints.
3. Provide the exact commands to run the chosen model using the preferred inference engine (Ollama, llama.cpp, etc.).
4. Suppl...
Details
- Author
- sickn33
- Repository
- sickn33/antigravity-awesome-skills
- Created
- 10 months ago
- Last Updated
- today
- Language
- Python
- License
- MIT
Integrates with
Bundled in these plugins
Similar Skills
Semantically similar based on skill content — not just same category
AI & Automation Listed
ai-local-model-ops
Runs local and self-hosted LLM workflows with Ollama, LM Studio, MLX, Open WebUI, llamafile, and adapters. Use when operating private model stacks.
90 Updated 1 months ago
vasilyu1983 AI & Automation Solid
vllm
Deploy and serve LLMs with vLLM behind an OpenAI-compatible endpoint, with tool calling enabled for agent workloads.
3,271 Updated today
Prism-Shadow AI & Automation Listed
ai-engineer
Build production-ready LLM applications, RAG systems, and intelligent agents. Use when implementing AI features, chatbots, vector search, or agent orchestration.
8 Updated today
rootcastleco