heretic-abliterationlisted

Heretic — fully automatic LLM censorship removal (abliteration) and serving the result via Ollama. Wraps p-e-w/heretic: directional ablation (difference-of-means refusal directions, orthogonalizing attn.o_proj + mlp.down_proj) with an Optuna/TPE search that co-minimizes refusal count and KL divergence. Automates the full local workflow with the dependency + device fixes that make it actually run: clean-venv install (+ kernels/jinja2 fixes), device-correct run (CPU on Apple Silicon — MPS is blocked by a PyTorch GQA matmul bug; CUDA for real models), and the HF→GGUF→Ollama bridge. Includes a no-compute fast path: pull a pre-abliterated GGUF straight into Ollama. Use when: (1) decensoring / uncensoring an open-weight LLM locally, (2) running heretic-llm and hitting install or MPS/CUDA errors, (3) converting a Heretic/abliterated HF model to GGUF and serving it via Ollama CLI, (4) pulling a ready-made abliterated model into Ollama, (5) understanding abliteration as an evaluator-governed (EGRI) optimization loop.
broomva/skills · ★ 3 · AI & Automation · score 72

Install: claude install-skill broomva/skills

# Heretic — Abliteration + Ollama Serving Remove refusals from open-weight LLMs with [`p-e-w/heretic`](https://github.com/p-e-w/heretic), then serve the result locally through the Ollama CLI. Heretic combines directional ablation ("abliteration") with an Optuna/TPE search that **co-minimizes refusals and KL divergence** — decensoring the model while preserving its intelligence, no fine-tuning required. > **The one architectural fact:** Ollama can't be Heretic's *backend*. Heretic edits > raw HF weights and reads activations (PyTorch/transformers); Ollama is inference-only > over GGUF. The link is downstream: **Heretic → GGUF → Ollama**. See > [references/troubleshooting.md](references/troubleshooting.md). ## Quick Start — pick your path **Path A — Serve now, zero compute (recommended on Apple Silicon / no GPU):** ```bash ./scripts/ollama-pull-abliterated.sh huihui_ai/llama3.2-abliterate:3b ``` **Path B — Run Heretic yourself, then to Ollama (needs a GPU for real models):** ```bash ./scripts/heretic-install.sh # clean venv + dependency fixes ./scripts/heretic-doctor.sh # confirm device/tooling ./scripts/heretic-run.sh Qwen/Qwen3-4B-Instruct-2507 # run in a REAL terminal (TTY) to save ./scripts/heretic-to-ollama.sh ./<saved-model> my-heretic Q4_K_M ollama run my-heretic ``` **Path C — Smoke test on Apple Silicon CPU (tiny model, proves the pipeline):** ```bash ./scripts/heretic-install.sh ./scripts/heretic-run.sh Qwen/Qwen3-