bitnetlisted

Microsoft BitNet — 1-bit LLM setup, inference, and benchmarking on CPU. Automates the full workflow: clone bitnet.cpp, create conda env, download GGUF models from HuggingFace, build optimized ternary kernels, and run inference. Supports official Microsoft models (2B) and community models (0.7B-10B). Use when: (1) setting up BitNet/bitnet.cpp for local CPU inference, (2) downloading and running 1-bit/ternary LLMs, (3) benchmarking BitNet vs full-precision models, (4) building edge/agentic inference pipelines without GPU, (5) converting HuggingFace models to GGUF for bitnet.cpp. Triggers on: 'bitnet', '1-bit llm', '1.58-bit', 'ternary model', 'ternary weights', 'edge inference', 'cpu inference', 'bitnet.cpp', 'bitlinear', 'no gpu inference'.
broomva/skills · ★ 3 · AI & Automation · score 72

Install: claude install-skill broomva/skills

# BitNet — 1-Bit LLM Operations Set up and run Microsoft's BitNet (1.58-bit ternary LLMs) for efficient CPU inference. Models use weights of {-1, 0, +1} — no GPU required. ## Quick Start ```bash # Full setup in 5 commands ./scripts/install-bitnet.sh ./scripts/download-model.sh microsoft/BitNet-b1.58-2B-4T-gguf ./scripts/build-bitnet.sh ./scripts/run-inference.sh -p "You are a helpful assistant" -cnv ``` Or manually: ```bash git clone --recursive https://github.com/microsoft/BitNet.git ~/BitNet cd ~/BitNet conda create -n bitnet-cpp python=3.9 -y && conda activate bitnet-cpp pip install -r requirements.txt huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "Hello" -n 128 ``` ## Prerequisites | Tool | Install | Required | |------|---------|----------| | Python 3.9+ | `brew install python@3.9` or conda | Yes | | CMake 3.22+ | `brew install cmake` | Yes | | Clang 18+ | `brew install llvm` | Yes | | conda | `brew install --cask miniconda` | Yes | | huggingface-cli | `pip install huggingface-hub` | Yes | ## Operations ### Install BitNet ```bash ./scripts/install-bitnet.sh [--dir ~/BitNet] ``` Clones the BitNet repo, creates `bitnet-cpp` conda environment, installs Python dependencies. ### Download a Model ```bash ./scripts/download-model.sh <model-id> [--dir ~/BitNet] ``` Downloads a GGUF m