How to Fine-Tune the d1-3B Decision Model on Custom Data: Hands-on with Unsloth and TRL

Leonie (@helloiamleonie) has released two hands-on Jupyter notebooks demonstrating how to fine-tune the d1-3B decision model on custom datasets. Built on the Unsloth and Hugging Face TRL training libraries, the streamlined pipeline can complete fine-tuning in under 30 minutes on a free-tier GPU instance.

Image source: @helloiamleonie / X

The tutorials focus specifically on the d1-3B architecture—a model tailored for structured decision-making and option selection. They walk practitioners through preparing decision datasets, configuring lightweight training runs, and addressing subtle statistical biases that arise during real-world fine-tuning.

Two Practical Notebooks: Unsloth and Hugging Face TRL

To fit different developer environments and stack preferences, the guide provides two independent notebook implementations:

• Data preparation: Formatting structured decision prompts alongside corresponding target option labels.
• Fine-tuning d1-3B: Fine-tuning targeting a single output option-code token representing the decision.
• Dual framework support: Ready-to-run implementations for @UnslothAI and @huggingface TRL.
• Hardware footprint: Fully executable on free-tier GPU environments within approximately 30 minutes.

The notebooks are accessible via the following links:

• Unsloth version notebook: https://t.co/VxwfzlwHXP
• Hugging Face TRL version notebook: https://t.co/yqzvFwVoZ5

Preventing Data Leakage with Case-Based Train/Eval Splits

Following the initial release, a key validation principle was incorporated into the notebooks after technical review within the practitioner community.

As noted by community reviewer @0xYinji, applying a naive random split to decision datasets can introduce severe data leakage between training and evaluation splits. When multiple reworded variations of the same underlying decision case are scattered across both sides of the split, the model risks memorizing context rather than truly learning to generalize decision logic.

Leonie confirmed the validity of this edge case and updated the notebook workflow to enforce splitting by underlying case, ensuring all phrasing variations of a single decision scenario remain strictly contained within either the training or the evaluation set.

Mitigating Positional Bias via Option Code Shuffling and Subsets

Another notable architectural consideration centers on how d1-3B formats options and handles token readout.

The d1-3B model avoids bolting on a fresh classification head, instead reusing its own internal readout token to emit decision choices. This design keeps the training target minimal—a single option-code token.

However, as highlighted by @jmfmdev01, assigning option codes strictly by input position introduces a risk: adding new candidate labels or altering option order later can shift position codes, requiring complete retraining and potentially causing positional shift biases during inference.

To prevent the model from overfitting to positional ordering, the notebooks have been updated to train with shuffled option codes and varying subsets of options per training point.

Original source

• Leonie (@helloiamleonie) on X: https://x.com/helloiamleonie/status/2110994738811715943