local-llm-agentlisted

# Local LLM as Agent Substrate This box is a serious local-inference machine. Use it — a frozen provider is not always the right (or private, or free) substrate. ## Hardware & runtime (verified 2026-06-05) - **2× NVIDIA RTX 5090 = 64 GB VRAM** (Blackwell), 251 GB RAM, ~676 GB disk free. - **ollama already installed & serving** — no install needed: - binary: `hivemind/.local/ollama/bin/ollama` (v0.22.1) - server: `http://127.0.0.1:11434` (used by memoryOS for `nomic-embed-text`) - VRAM budget: a 30B-MoE Q4 (~18 GB) fits ONE 5090. Don't co-load two 30B on the same card; ollama schedules across both GPUs. Check free VRAM: `nvidia-smi --query-gpu=memory.free --format=csv,noheader`. ## Models (pull once, then they're assets) ```bash OLLAMA=hivemind/.local/ollama/bin/ollama $OLLAMA list # what's already local $OLLAMA pull qwen3-coder:30b # BEST agentic coding: 30B MoE/3.3B active, 256K ctx ``` Already cached on this machine: `qwen3-coder:30b` (agentic coding, primary), `qwen3:30b-a3b` (general), `deepseek-coder-v2:16b`, `qwen3:8b`, `nomic-embed-text`. Bigger flagships (Kimi K2.6, GLM-5.1, DeepSeek-V4) are enterprise-scale — do NOT fit 64 GB; use a hosted provider for those. ## How to call it ```bash OLLAMA=hivemind/.local/ollama/bin/ollama # 1. CLI one-shot $OLLAMA run qwen3-coder:30b "Write a Python function that …" # 2. Native HTTP (non-streaming) curl -s http://127.0.0.1:11434/api/generate \ -d '{"model":"qwen3-coder:30b","prompt":"…