implementing-llms-litgpt

Featured

Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.

AI & Automation 14,647 stars 1007 forks Updated 1 months ago MIT

Install

View on GitHub

Quality Score: 94/100

Stars 20%
100
Recency 20%
75
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# LitGPT - Clean LLM Implementations ## Quick start LitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows. **Installation**: ```bash pip install 'litgpt[extra]' ``` **Load and use any model**: ```python from litgpt import LLM # Load pretrained model llm = LLM.load("microsoft/phi-2") # Generate text result = llm.generate( "What is the capital of France?", max_new_tokens=50, temperature=0.7 ) print(result) ``` **List available models**: ```bash litgpt download list ``` ## Common workflows ### Workflow 1: Fine-tune on custom dataset Copy this checklist: ``` Fine-Tuning Setup: - [ ] Step 1: Download pretrained model - [ ] Step 2: Prepare dataset - [ ] Step 3: Configure training - [ ] Step 4: Run fine-tuning ``` **Step 1: Download pretrained model** ```bash # Download Llama 3 8B litgpt download meta-llama/Meta-Llama-3-8B # Download Phi-2 (smaller, faster) litgpt download microsoft/phi-2 # Download Gemma 2B litgpt download google/gemma-2b ``` Models are saved to `checkpoints/` directory. **Step 2: Prepare dataset** LitGPT supports multiple formats: **Alpaca format** (instruction-response): ```json [ { "instruction": "What is the capital of France?", "input": "", "output": "The capital of France is Paris." }, { "instruction": "Translate to Spanish: Hello, how are you?", "input": "", "output": "Hola, ¿cómo estás?" } ] ``` Save as `data/my_dataset.json`. **St...

Details

Author
Orchestra-Research
Repository
Orchestra-Research/AI-Research-SKILLs
Created
1 years ago
Last Updated
1 months ago
Language
TeX
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category