Documentation · Demos · Getting Started · Supported Models · Example · Updates

The same Valen checkpoint answers 24 structured questions about one video, using eight sampled frames and one A800 per runtime. In this live run, Valen's PyTorch reference path took 5.31 s, while vLLM Jev took 1.22 s (4.3× faster); all 24 selected answers agreed. Model loading is excluded. Watch the MP4.

Valen 2B answers six GameQA Maze image questions locally on an Apple M5 using MLX. The display follows live HTTP responses. Watch the MP4.

Both runtimes played the same five levels from Valen's Sokoban evaluation set with the same checkpoint and one A800 each.

The Open-Jev-2B author HTTP server and vLLM Jev received the same 40 text-only Choice requests at once, on one A800 each.

vLLM Jev serves compatible Jev-style checkpoints through vLLM. Give it a question and candidate answers; it returns a label and a probability for each candidate.

• Native vLLM serving on Linux: scheduling, batching, compilation, KV cache, and metrics.
• Apple Silicon preview: run supported text models through MLX or PyTorch MPS, and Valen multimodal decisions through MLX.
• Decision readouts: scalar candidate branches, marker scores, and Laya's trained decision head.
• Structured decisions: Choice, Noul (yes/no), and Score (ordered levels) over HTTP.
• Multimodal inference: Valen and vjev accept text and images; Valen also accepts short videos.
• Automatic setup: give the launcher a supported Hugging Face model ID; it selects and verifies the native protocol. Serving does not train.

On Linux with an NVIDIA GPU, install vLLM Jev with uv from the repository directory:

uv pip install .

For development, use an editable installation.

Start a server with a supported Hugging Face model ID:

vllm-jev serve ZefanCai/Open-Jev-2B

The command downloads and prepares the model, then starts native vLLM at http://127.0.0.1:8795. Later runs reuse the prepared checkpoint.

On an Apple Silicon Mac with macOS 15 or newer, set up once from the repository directory, then serve:

source scripts/install_mac.sh
vllm-jev serve Valen-Team/Valen-Preview-0923

The command downloads and prepares the model on first use. Send text or images to /v1/systemone for Choice, Noul, and Score decisions. For another Mac model, replace the model ID with a supported one from the table below.

CUDA_VISIBLE_DEVICES=0 vllm-jev serve ZefanCai/Open-Jev-2B \
  --gpu-memory-utilization 0.90 --port 8795

Visit our documentation to learn more.

• Installation
• Quickstart
• List of Supported Models

Choose a checkpoint for your platform and run its command. Linux uses native vLLM; Apple Silicon uses MLX, or PyTorch MPS for Laya.

These text-only checkpoints use the same command and /v1/systemone API for Choice, Noul, and Score. Listed platforms have been tested.

JevK5 currently supports 2–16 options per question. In concurrent Linux batches, Kev-0.8B can change its selected label; This-That and Decider-2B can also change the most likely Score level. These consistency limits remain under investigation.

See the model guide for limits and request examples.

With the server running, send a Choice request:

curl -sS http://127.0.0.1:8795/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state":"I was charged twice and want a refund.","questions":{"intent":{"type":"choice","instructions":"Choose the customer intent.","criteria":{"billing":"A payment or refund issue","technical":"A malfunction or setup issue","other":"Another request"}}}}'

The response contains answers.intent.choice and answers.intent.probabilities for the supplied labels.

See the user guide for supported models, serving options, and Choice, Noul, and Score examples. The plugin targets vLLM 0.29.0 and Python 3.12+.

For image questions, start a supported vision model and follow the image request example. Valen also supports short MP4 videos on Linux and Mac.

• Added experimental adapters for Kev, Decider, Mica, This-That, and JevK5, with tested Linux and Mac availability listed above.

• Added experimental video input for Valen on Linux and Apple Silicon, including variable-frame-rate clips. Usage.

• Improved Apple Silicon text serving with adaptive batching, prefix reuse, and bounded memory caching.
• Added optional CUDA Graph readouts for Laya multilingual on Linux.
• Added optional repeated-image reuse for Valen on Mac. See serving options.

• Added all three Laya checkpoints on Linux and Apple Silicon: English, multilingual, and typed decisions.
• Added Tiny-Jev serving on Apple Silicon with its marker readout.
• Added Open-Jev-2B serving on Apple Silicon with its scalar decision head.
• Added local text and image decisions on Apple Silicon with OpenJev-0.6B and Valen.
• Set up with one script, then start a model with the same vllm-jev serve command.
• Kept Mac server health checks responsive during inference.

• Added multimodal decision inference with support for text and images.
• Added a fused Tiny-Jev head: 12.1% higher throughput in the measured 16-option, 16-concurrent workload.
• On 500 Valen image questions, median latency fell from 231.9 to 77.3 ms (3.0×) with 86.8% target-support accuracy on both paths. Results.

Measured on an Apple M5 with 16 GB unified memory, using MLX: 96 text Choice requests for OpenJev and 42 GameQA Maze image questions for Valen. Requests are sequential after warmup; model loading is excluded.

Valen selected the same answers as the Linux reference on all 42 questions; both answered 17 correctly. Probability values differ between backends. Mac executes one model request at a time; concurrent requests queue.

Each row uses the same model, input, and A800 GPU for both paths. Lower latency and higher throughput are better. Measurement methods are noted below.

Paired values: Without vLLM Jev → With vLLM Jev.

Report bugs and feature requests in Issues. Use Discussions for questions and ideas. Submit code changes as pull requests; the maintainer reviews them before merging.

Apache-2.0. The Choice prompt follows MIT-licensed Open-Jev; see third-party notices.