← Back
avbiswas

avbiswas/bev-train

Training Jev-like decision models using Qwen3 with choice-order invariance

View on GitHub ↗
Stars
40
Forks
3
Watchers
40
Open issues
0
Contributors
1
Language
Python
License
—
Default branch
main
Created Sep 27, 2026Updated Sep 27, 2026

Star growth

Today—
This week—
This month—

Star history will appear here once this repo has been tracked for a couple of days.

README

Training JEV networks from Qwen

This repo is under construction. This is for the upcoming Neural Breakdown video on training JEV networks from existing autoregressive LMs. The cool part of this approach is that we are making the architecture be choice-order invariant, while still passing multiple options in the same sequence.

Start here: arch.ipynb

If you want to understand how this repo works, open arch.ipynb first. It's the notebook from the livestream, and it builds the core idea step by step: tokenizing the question and each choice separately, giving every choice the same starting position ids, and building the attention mask so no choice can see another. Nothing gets trained in there, it's all about the architecture.

The Python files take that notebook code and turn it into a full pipeline. tokenization.py reuses the notebook functions almost as they are, and dataset.py, network.py, train.py and inference.py build the rest around them.

Dataset: https://huggingface.co/datasets/avbiswas/bev-decision-150K

Livestream: https://youtube.com/live/AzxoU7kxjig