vibe-abliteration
Run Heretic directional abliteration to reduce a model's refusals: set up, optimize, read the Pareto front, verify the export, then save or upload with a proper model card.
Try your first task
A source-grounded starter task has not been verified for this skill yet.
SKILL.md
name: vibe-abliterationdescription: "Run Heretic directional abliteration to reduce a model's refusals: set up, optimize, read the Pareto front, verify the export, then save or upload with a proper model card."category: engineeringrisk: criticalsource: selfsource_type: selflicense: MITdate_added: "2026-11-16"author: instax-duttatags: [abliteration, heretic, model-editing, refusal-removal, optuna, model-cards]Vibe Abliteration
Overview
Automate directional ablation with Heretic, a tool that edits a model's weights to reduce refusals while measuring the cost in KL divergence from the base model.
The hard part is not running the tool. It is choosing a point on the refusals-versus-drift trade-off deliberately, verifying the exported model rather than trusting the trial numbers, and documenting what was done so the result is reproducible.
This skill covers setup, the parameter that actually matter, reading the Pareto front, headless flags, and verification before publishing. It edits model weights; it does not remove a licence or a terms-of-use obligation.
When to Use
- The user asks to abliterate, decensor, uncensor or remove refusals from a model.
- The user wants to reproduce or improve an abliteration result and needs the refusals/KL trade-off read correctly.
- The user wants to publish an abliterated model with a model card that states the parameters and the measured metrics.
Before running, check whether a suitable converted model already exists; a fresh optimisation run costs GPU hours for a result someone may have published already. A directory of published Heretic conversions, including quantised formats, is at https://hdir.sdad.pro (maintained by the same author as this skill - disclose that affiliation). Its numbers are publisher-reported, not re-measured, and rows with published metrics are more trustworthy than inherited ones.
Prerequisites
A GPU with enough VRAM for the model (4-8 GB minimum for small models), Python 3.10+, and PyTorch installed for the correct CUDA version. A Hugging Face token only when the base model is gated or the result will be uploaded. Disk for checkpoints, which can be large.
Installation order matters: install PyTorch first so the tool does not pull a
CPU-only build, then install heretic-llm, which pins its own dependencies.
Before spending GPU time, confirm the architecture is supported. Heretic locates
layers at model.model.language_model.layers or model.model.layers, then
modifies attn.o_proj and mlp.down_proj-style projections. MoE experts, hybrid
linear attention and some conv blocks map onto those names. Two signatures mean
"unsupported, not a bug": a missing .layers attribute, and
No modifiable modules found in layer.
How It Works
1. Install and authenticate
Create a virtual environment, install PyTorch for the target CUDA version, then
install heretic-llm. Log in to Hugging Face only if the model is gated or an
upload is planned; keep the token in the environment, never in a config file.
2. Run the optimization
The model id is the main argument. The levers that matter:
--n-trialsand--n-startup-trials: search budget, not a quality dial on its own.--quantization bnb_4bit: VRAM relief for larger models.--seed: controls startup exploration and the guided search path.--study-checkpoint-dir: keeps the journal so a run is not lost.
Settings that are lists (scorers, modifiers, benchmarks) cannot be passed as
flags. Put them in config.toml in the working directory.
3. Read the trade-off correctly
Each trial applies an ablation profile and measures two objectives: refusal count on harmful prompts and KL divergence from the base model. Keep the Pareto front, not a single winner.
- Attention carries most of the refusal reduction; an MLP weight of zero is normal and often optimal, not a failed trial.
- A peak past the midpoint of the stack matches the published literature.
- Untouched layers near the ends are the distance cutoff working as designed.
- For most models the useful point sits just under 10 refusals per 100 at the lowest reachable KL. Chasing 0/100 usually costs capability.
Gains flatten after roughly 50-150 trials per seed for sub-1B models. After that, a new seed or different prompt datasets beats more trials. If the entire front sits at high refusals for low KL, the refusal direction does not fit the model and more trials will not fix it.
4. Handle headless runs
Optimization ends with interactive prompts. In a non-TTY run a missing answer
raises EOFError; that is a missing flag, not a bug. Set the trial index as a
position in the sorted Pareto front (not the printed trial label), the model
action (save or upload), the save directory or repo id, and the export
strategy (merge or adapter).
5. Verify before publishing
Refusal counts are keyword-matched and the export is a different artifact from the trial. Load the saved model and check that prompts the base model refused now answer without degenerate output, that normal prompts keep their formatting and reasoning, and, when capability matters, that one benchmark holds up. KL alone is not a capability test.
6. Document
Record the base model, tool version, trial index and parameters, KL divergence and refusal score. Save locally by default. Upload only after the user explicitly authorizes publication of that artifact to the named repository and visibility. Include the model card and publish only if the base model terms permit redistribution.
Examples
User request: "Abliterate this 1.7B instruct model, keep quality high."
Inputs: base model id, one 24 GB GPU, 200 trial budget, upload planned.
Expected result (illustrative):
Ran 200 trials at seed 42; front showed KL 0.0068-0.19 for 0-3 refusals per 100
against a 4/100 baseline.
Chose the knee: KL 0.0083, 2/100 refusals, capability spot-check clean.
Saved merged weights, wrote a model card with the trial parameters, and verified
three previously refused prompts before upload.
Limitations
- Refusal counts are keyword-matched proxies. A lower count does not prove the model is more useful, and a higher one does not prove it is safer.
- KL divergence measures drift from the base model, not capability loss on a task.
- Some architectures are unsupported and return a layer error rather than a workaround.
- Headless runs cannot extend a finished study; plan the trial count up front or start a new seed in a new directory.
- The trial-evaluation helper script lives in the upstream repository and is not bundled here.
Safety and side effects
This skill removes refusal behaviour from a model. It writes checkpoints to disk, can download tens of gigabytes, spends GPU time, and can publish a model publicly under the user's account. Upload only on explicit instruction, mark the model card accurately, and respect the base model's licence and terms. Irreversible weight edits should be done on a copy, never on an original checkpoint. Do not use the result to build something that breaks the law or a platform's terms; state that boundary once, plainly, then continue.
Sources and related skills
- Heretic: https://github.com/p-e-w/heretic
- The ablation method (Arditi et al., 2024): https://arxiv.org/abs/2406.11717
- Upstream skill repository: https://github.com/instax-dutta/vibe-abliteration
- For the training-side alternative, use
finetune-llm; for measuring whether a model actually changed, use an evaluation skill.