WhatisHereticLLM,andhowdoesitdifferfromabliteration?

What is the Heretic LLM tool?

Heretic LLM, usually just Heretic, is a command-line tool by Philipp Emanuel Weidmann that removes refusal behavior from transformer language models without retraining them. Its README describes it as removing “censorship (aka safety alignment)” without expensive post-training, and says it supports most dense models, many multimodal models and several mixture-of-experts architectures. Pure state-space models and some research architectures aren’t supported yet.

The core edit is not new. Heretic credits Arditi et al. (2024) and Maxime Labonne’s abliteration article as its foundation. What it adds is automation: instead of a person inspecting outputs to choose settings, the tool runs a search and scores each candidate.

The project’s license is GNU AGPL-3.0. That covers the tool, not the models it produces, a distinction the licensing section below returns to.

How does classic abliteration work, in one paragraph?

Classic abliteration finds a “refusal direction” in a model’s activations and removes it from the weights. Arditi et al. showed in 2024 that across 13 open chat models up to 72B parameters, refusal is mediated by a one-dimensional subspace of the residual stream. Labonne’s write-up turns that into a recipe: record activations on harmful and harmless prompts, take the difference of the means per layer, rank the candidates, pick one, and orthogonalize the weights that write into the residual stream. The selection step is partly manual. Labonne’s code comments say the layer “needs human evaluation to determine best layer for refusal inhibition,” and his example found that one candidate worked for all four test instructions. For the full background, see what abliteration is and what it costs.

What does Heretic change?

Heretic changes how the ablation settings are chosen: an optimizer searches for them and scores each try on two objectives. According to the README, it combines directional ablation with a TPE-based parameter optimizer powered by Optuna, and it “co-minimizes” the number of refusals and the KL divergence from the original model.

Two details are worth understanding.

The refusal count is a keyword check. In the default config, the scorer marks a response as a refusal if it contains any of 33 marker strings, such as “sorry” or “ethical boundaries,” and responses are capped at 100 generated tokens. That is cheap and fast, and it also means a response can dodge the markers without actually helping. Treat the refusal number as a screen, not a verdict.

KL divergence is the guardrail. The tool measures how far the edited model’s output distribution drifts from the original on harmless prompts (by default a slice of mlabonne/harmless_alpaca). Low KL means the model behaves like the original on ordinary requests. It doesn’t tell you whether coding quality, long-context behavior or math survived. That still needs a real benchmark.

How does Heretic LLM behave in practice, according to developers?

Community reports on Heretic are mixed, and they depend heavily on the model family. The source below is the Hacker News launch thread from November 16, 2025. These are anecdotes from commenters, not our tests, and several come from the tool’s author.

• Family matters. The author said Llama 3.1 is trivial to decensor and the optimizer converges quickly on settings that stop refusals. For gpt-oss and Qwen3, they said most parameter configurations barely have an effect, and it takes much longer to find one that lowers refusals at all.
• Refusal counts can overstate refusals. The author also advised picking a configuration with KL divergence below 1 even when the refusal rate looks high. Reasoning models like gpt-oss talk about refusing inside their chain of thought, and that trips the keyword classifier. That fits the keyword-check caveat above.
• Results on other releases were uneven. One commenter who tried the gpt-oss-20b Heretic release called it “still quite refusey” and reported that on Qwen3 14B, using their own locally modified copy of the tool, the edit either wrecked the model or did nothing. On a Granite model the same setup reached 1/50 refusals at 0.02 divergence. Another commenter converted that release to GGUF, ran it in Ollama, and still got a refusal on a basic harmful-category prompt.
• Quantization may matter. One user reported that a Q4_K_M GGUF of the 20B release failed a first test prompt in llama.cpp while the Q8_0 build passed. That is a single report, but it suggests testing the quant you will actually run.

The pattern is consistent with the README numbers below: a low KL divergence on one model family doesn’t promise the same result on another, or after quantization. Treat a release’s card as a claim to re-check on your own prompts and your own quant.

How do results compare according to the project and the papers?

The head-to-head numbers come from the project itself. An earlier version of the README (read October 2026) had this table, run on PyTorch 2.8 and an RTX 5090, comparing variants of google/gemma-3-12b-it. The current README (checked November 11, 2026) no longer includes it. The Heretic row is still on p-e-w’s model card; the other two rows can no longer be checked against a live source:

All three edited models suppress refusals equally on this test. The difference is drift: the Heretic version stayed closest to the original. The README notes the values might be platform- and hardware-dependent. The current README instead points to independent comparisons on Hugging Face (DreamFast’s Qwen2.5-7B and Gemma 4 E4B abliteration write-ups) and says Heretic compares favorably on MMLU and GSM8K.

Read this table with three caveats. It is one model, one prompt set and one author’s run. The comparison models were made with other tooling and settings, not tuned by the same search. And KL divergence on harmless prompts is not a capability benchmark.

No peer-reviewed paper on Heretic turned up in our search; the project’s citation entry points to the GitHub repository. The closest independent data comes from a different refinement of the manual recipe. Jim Lai’s norm-preserving biprojected abliteration (Nov 6, 2025) argues that plain weight subtraction is “mathematically unprincipled” because it changes weight norms, and reports UGI and NatInt scores on Gemma-3 12B Instruct of 32.08 / 18.64 for standard abliteration versus 32.61 / 21.33 for his method (baseline 19.58 / 18.72). Heretic’s README credits this work too. The point for developers: the field is converging on “remove the direction, but preserve what the model already knew,” and Heretic is one automated take on it.

What are the licensing and responsible-use limits?

Heretic’s AGPL-3.0 license governs the tool; the weights you produce remain subject to the base model’s license. Running Heretic on a Gemma model, for example, leaves you under Google’s Gemma Prohibited Use Policy, which bars “generating content for deceptive or fraudulent activities, scams, phishing, or malware.” The same logic holds for Llama’s acceptable use policy and other community licenses.

The README, as read on 2026-10-26, contains no responsible-use section. That puts the responsibility on whoever runs the tool and publishes the result. Practical rules for teams:

• Check the base model’s license before you modify or redistribute weights.
• Keep the resulting model for authorized work on systems you own or have permission to test.
• Note which tool, version and settings produced a released model. Without that, nobody can reproduce or audit it.
• Remember that removing refusals moves the policy decision to you. A hosted product built on such a model needs its own safeguards.

Should a developer use a Heretic model or a classic abliterated one?

Choose by what the model card proves, not by the tool’s name. A Heretic release is a decent default when the card reports both a refusal count and a KL divergence, because that is the number most likely to predict “behaves like the base model on normal tasks.”

Whatever you pick, evaluate it on your own tasks, at the quantization you plan to run, before it touches a pipeline. Our list of best abliterated models covers specific releases. If you build products on open weights, also read how model makers are defending against abliteration, because a technique that works on today’s models may not on the next generation.

FAQ