Laya: A 100% Open-Source Local Decision Model Alternative to Jev

In the emerging space of decision models—specialized AI models designed to ingest unstructured state data and typed questions to return discrete probability distributions over predefined choices—a fully open-source local alternative has arrived. Machine learning researcher Akshay Pachaar has released architectural details and benchmarks for Laya, an open-weight local alternative to the hosted decision API Jev.

Image source: @akshay_pachaar

Released under the permissive Apache 2.0 license, Laya runs entirely on local infrastructure using open-weight encoder models. Instead of generating text token by token, it evaluates candidate answers and outputs calibrated probabilities via Softmax, enabling deterministic conditional branching directly in application code.

How the Bidirectional Encoder Decision Pipeline Works

While both Jev and Laya address the identical objective—mapping unstructured state text and typed questions to actionable probability distributions—their underlying architectures and inference strategies differ fundamentally.

• Language-First Routing: Prior to inference, a lightweight Python router selects either an English-specific or multilingual checkpoint based on input text characteristics.
• [MASK] Marker Sequence Creation: For each question, candidate options are assigned [MASK] markers and concatenated with the question and unstructured state context to form comparative evaluation sequences.
• Direct Option Scoring: Bypassing autoregressive token-by-token generation, a bidirectional encoder paired with a compact decision head scores every candidate answer directly, followed by Softmax normalization to produce a final probability distribution.
• Batched Question Evaluation: Multiple questions are grouped and processed in a single model pass, with each question carrying its own replica of the unstructured state context.

35ms Inference Latency and On-Premise Infrastructure Benefits

By removing token-generation loops and external network round-trips, Laya achieves substantial speedups in execution latency.

In independent benchmark tests, Jev required approximately 380ms over standard network API calls, whereas Laya registered an inference latency of roughly 35ms running locally on GPU hardware.

Additionally, self-hosting eliminates per-call API expenses, functions effectively on legacy GPU hardware, and guarantees that sensitive proprietary data never leaves internal infrastructure.

Zero-Shot Generalist vs. Task-Specific Specialization

Laya is not an automatic drop-in substitute across every decision workload. The operational tradeoffs between the two systems are clearly defined:

• Jev (Hosted Generalist): A closed, proprietary API supporting up to 255 options per query, offering robust out-of-the-box zero-shot performance without preliminary training requirements.
• Laya (Local Base to Specialize): Built for environments with stable candidate answer sets and representative domain training data. It delivers maximum efficiency when paired with task-specific fine-tuning and validation-set probability calibration.

In high-concurrency production batching, replicating the unstructured state across multiple questions can lead to sharp memory spikes in encoder contexts. Production deployments are therefore best structured around defined answer sets, routing low-confidence or high-uncertainty decisions to upstream generalist LLMs or human reviewers.

Sources

• Akshay Pachaar (@akshay_pachaar) on X: A faster, 100% open-source alternative to Jev: Laya