LayaSmall models.
Decisions you can inspect.

01 / WHAT LAYA DOESWhat Laya actually does

Laya is an open-source System 1 decision-model family from Convai Innovations. Give it a business state and typed questions; it returns a choice, an ordinal score or a probability. It does not generate a reply token by token, write explanations or execute tools. “System 1” is a product metaphor for fast judgments, not evidence that the model reproduces human cognition.

A generative LLM can analyze a request and draft a response. Laya is closer to a dispatcher: which queue owns the request, whether it asks for a refund, how urgent it is. It can sit before an LLM as a router or after it as a checking component. Business rules, retrieval, a generative model or a person still complete the task. S01S02

The practical assessment, as of September 23, 2026: Laya is worth evaluating as a small, locally deployable decision component. Its usefulness depends on the task, language, option count, fine-tuning data and calibration. The “33 ms” headline does not settle those questions. Performance figures throughout this guide are attributed to their published experiments; they are not new Coachix inference measurements.

Other projects called Laya

This guide covers NandhaKishorM/laya and convaiinnovations/laya. The aayushch/laya desktop notification assistant, the LAYA layer-attention paper at arXiv:2511.12723, and the EEG representation model at arXiv:2603.16281 are different projects. Their results do not belong to this decision model. S36S37S38

02 / OFFICIAL LINKS & TIMELINEOfficial links, versions and timeline

The observed PyPI release is 0.3.6, requiring Python ≥3.10, uploaded on September 22, 2026 at 20:43 UTC (September 23 at 04:43 in Beijing). Dependencies include PyTorch, Transformers, safetensors, huggingface_hub and NumPy. Code-level explanations use GitHub commit c7527708f9f5220c669d8aa385077cd28d04708a. A Git commit and a package version are different identifiers; they are not assumed to contain identical code. S08S09

The website links the author's earlier research to Jev. That is the author's account of provenance and priority. Public material does not establish that Jev borrowed this project's code, weights or methods. Similar terminology is not proof of derivation. S01S10S22

03 / CHOOSE A CHECKPOINTChoosing among the three checkpoints

The same checkpoints can be loaded from the main hub's root, multilingual and typed-decisions subfolders. Downloading one model does not load all three into memory. The model card's approximately 808 MB / 647 MB weight-size figures are not runtime memory promises. FP32, activations, intermediate tensors and request batches add memory use. S03S04S05

The context-length trap

A backbone's longer positional range does not mean Laya feeds an entire 8192-token document into its default model. The actual max_len values are 512 or 1024. Instructions, options and state share that budget. The corresponding head_max_len values are 192 / 256 / 256. The sequence constructor calculates remaining space from the actual question-head length; “about 320 / 768 tokens for state” is only an estimate. Special tokens and the question length change the remaining space. S11S14S15S16

By default, an overlong state keeps its beginning and loses its tail. A customer's final correction—“I already resolved this” or “the previous description was wrong”—may disappear. Raising the configured length requires fresh memory, latency and accuracy measurements. The backbone's maximum length is not a guarantee of useful performance.

An English specialist is still an English specialist

The typed-decisions checkpoint still uses English ModernBERT. Its English workflow advantage does not establish an advantage on Chinese tickets or alerts. Start a Chinese baseline with multilingual, compare it with suitable Chinese models, and evaluate any domain fine-tuning on samples in the target language.

04 / INSIDE A DECISIONInside a single decision

Each question gets its own input sequence, approximately:

[CLS] question type + instructions [SEP]
[MASK] option A [MASK] option B … [SEP]
business state [SEP]

1. Serialize the state. Text, JSON objects and conversation lists become textual representations. JSON describes the situation; it is not a database transaction and triggers no tool automatically.
2. Render the question and options. The three question types have token budgets. The SDK handles literal [MASK] text to prevent interference with option markers.
3. Encode in both directions. Tokens provide context to one another. The encoder does not repeat a next-token decoding loop.
4. Apply type and decision heads. A question-type embedding is added to the encoded representation, followed by a two-layer Transformer decision head. A scorer reads each option marker's representation and produces its logit.
5. Apply temperature and softmax. Temperature is selected by question type and option count, then logits are scaled and normalized into probabilities.
6. Return typed values. choice selects the highest-probability option, score calculates an expected ordinal level, and noul returns the probability of truth. A separate act/escalate auxiliary head does not execute an action. S11S12

One forward pass still has a batch cost

Agent.system_one() builds one state-containing sequence per question, batches those sequences, then calls the model once. It avoids token-by-token generation, but repeats the state across question sequences. Work still depends on question count, length and padding. The official T4 measurements distinguish one question from fifty.

This also explains why usage.input_tokens can exceed the original state's token count: it counts valid input tokens across the batch. output_tokens = 0 means there is no generative output. Many questions or long text still require batch limits, request limits and memory monitoring.

Dynamic options versus a conventional classifier

A conventional classifier usually has a fixed label head established during training. Laya includes the option text in the input, so a request defines its answer space. This is flexible, but it does not make every new task accurate without training. New label meanings, industry vocabulary, rare events and fine distinctions can still exceed the model's ability.

05 / CHOICE, SCORE & NOULChoice, score and noul

For levels [Routine, Time-sensitive, Urgent] with probabilities [0.1, 0.3, 0.6], score = 0×0.1 + 1×0.3 + 2×0.6 = 1.5. The text does not literally belong to “level 1.5”; the number compresses an ordinal distribution. If an application needs an integer, choose and evaluate argmax, a ceiling rule or separate thresholds. Rounding is not automatically the correct policy.

A mutually exclusive choice loses information when a request is both a refund request and a complaint. Two noul questions can preserve those flags; alternatively, route the main intent and identify risk flags separately. Ambiguous questions, overlapping labels and an incomplete “other” category can force a confident but misleading choice.

Confidence is not always the largest probability

In the inspected implementation:

• For choice and score, confidence = 1 − H(p) / log(K): normalized concentration of the distribution.
• For noul, confidence = max(P(true), 1 − P(true)).
• action.act_probability comes from a separate auxiliary head. It is neither the selected option probability nor permission to act. S11S12

For [0.8, 0.2], the maximum probability is 0.8, but normalized-entropy confidence is approximately 0.278. A noul question with the same truth probability has confidence 0.8. Copying a “confidence ≥0.85 means automate” rule across types therefore creates very different behavior. Different types and option counts should not share an unevaluated threshold.

Returned probabilities are commonly rounded to four decimal places, so small sum errors are possible. Account for this precision in calibration calculations and logs. Do not take the logarithm of a rounded zero without an explicit numerical policy.

06 / RLCD & TRAINING DATARLCD, the training objective and data

RLCD means Reinforcement Learning for Calibrated Decisions. Laya's explanation describes a model reporting a probability distribution, exploring with noise on its logits, and receiving rewards from proper scoring rules rather than only from whether the winning class is correct. S03S10

In proper_reward(), targets may be one-hot labels or soft distributions. The reward combines log and spherical scores; ordinal score questions also subtract a Ranked Probability Score based on cumulative distributions. Omitting masks:

log score       = Σ targetᵢ · log(qᵢ)
spherical score = Σ targetᵢ · qᵢ / ||q||₂
RPS             = Σ (CDF(q)ᵢ − CDF(target)ᵢ)² / (K−1)
reward          = log score + wₛ · spherical score − wᵣ · RPS (ordinal)

Proper scoring rules provide an incentive in expectation: under their mathematical assumptions, reporting the true conditional probabilities is optimal. Finite data, incorrect labels, distribution shift, optimization error and implementation clipping break the shortcut from that theory to guaranteed calibration. The code also floors the log score for numerical stability. Every individual output is not thereby mathematically guaranteed to be calibrated. S11S27

What the public fine-tuning notebook actually does

The 2×T4 notebook synchronizes gradients with DDP. It adds zero-mean-projected Gaussian noise to logits to produce groups of candidate distributions, uses the group's mean reward as a baseline, standardizes advantages and computes a policy-gradient term. It also adds soft cross-entropy with weight 1.0: loss_rl + 1.0 * loss_ce. The executable recipe is not an RL-only loss. S17

This detail explains more than the word “reinforcement”: label distributions provide supervision while sampling and rewards shape probability behavior. The notebook is a domain fine-tuning route. It is not automatically a complete, step-by-step reproduction of all three released models from initialization onward.

Keep the generations of data claims separate

A dataset's presence in a training mixture does not prove test-instance leakage. Different splits do not rule out every form of contamination either. Separate source overlap, duplicate examples, template overlap and learning a teacher's style. S17S18S23

07 / PROBABILITY CALIBRATIONCan the probabilities support a decision?

If a model reports 90% confidence on one hundred questions but gets only sixty right, it is overconfident. Accuracy measures selection correctness; calibration concerns the relationship between reported probabilities and long-run frequencies. Low ECE can also accompany uninformative probabilities, so inspect accuracy, Brier score, log loss and coverage together. S26

Temperature scaling and its limits

Temperature scaling computes p = softmax(z/T). A positive scalar T for one question preserves the largest logit's identity; it changes the distribution's sharpness. T above one generally flattens it; T below one sharpens it. It cannot restore truncated evidence or repair a misunderstood label.

Upstream reports English ECE falling from 0.466 to 0.081 and multilingual ECE from 0.314 to 0.106 after fitting temperatures in a particular experiment. These are experiment-specific recalibration results, not promises for unseen production traffic. S06

The English configuration still contains an approximately 0.10058 temperature for choice:11+. The inspected SDK clamps temperatures to [0.5, 5.0] and warns about out-of-range values. Identical weights can therefore yield different probability behavior with different SDK code. Record package version, source commit and calibration file together. S11S12S14

A reproducibility detail that matters

The notebook fits temperature using all_items[::15][:400], where all_items contains training-process data. It does not read a fully independent calibration split for that step. For deployment, split training, calibration and final testing explicitly; do not treat the tutorial's sampling method as independent calibration evidence. S17

Plot threshold against coverage and error rate on the target distribution, choose thresholds from the cost of mistakes, and retain an abstention path. A threshold such as 0.85 is an example, not an industry standard.

08 / ROUTING LOGICHow the Router chooses a model

Router is a Python dispatcher, not another trained large model. The inspected priority is: explicit model → explicit task → opt-in workflow detection → explicit lang → script/language detection → default checkpoint. S13

Version-sensitive detail: auto_task_detection=False by default. Business-looking JSON does not automatically select typed-decisions. Use model="typed-decisions" or task="typed_decisions" explicitly. When automatic workflow detection is enabled, it matches known sets of question IDs; it does not understand which checkpoint is best for an arbitrary new business.

from laya import Router

router = Router()
router.preload(["english", "multilingual"])
# Inspect routing without running the neural network.
print(router.route({"body": "Please investigate a duplicate charge."}).reason)
# If the input language is already known, declare it explicitly.
result = router.predict(state, questions, lang="zh")

The default lazy-loading mode, max_loaded=1, limits resident memory. Alternating English and Chinese traffic can trigger eviction and rebuilding. Upstream measured median reloads around 7.4 seconds on CPU and 10.3 seconds on T4 in some scenarios. Preloading reduces this variability but uses more memory. Loading time and warm inference time are separate measurements. S02S13S21

Language detection can fail on short abbreviations, numbers, code, or English JSON keys mixed with Chinese values. Use lang when the language is known, and log the actual selection and reason from routing metadata. High confidence does not prove the language route was correct.

09 / OFFICIAL BENCHMARKSOfficial benchmarks, with their conditions

The following results come from BENCHMARKS.md and associated public result files. The project states that Laya checkpoints within one experiment are comparable, while quoted Jev numbers come from external runs with different samples and prompts. S06S19S20

Three checkpoints on typed-decisions

This test contains 400 cases and 2,000 decisions. Values below follow BENCHMARKS.md; small README differences such as 0.361/0.362 or 0.061/0.062 are not averaged together.

The specialist's workflow accuracies are approximately 0.804 for invoices, 0.766 for security incidents, 0.764 for customer service and 0.730 for agent traces. The base English and multilingual checkpoints fall below this test's majority baseline. That matters when choosing a checkpoint.

Teacher agreement is not a truth ceiling

The dataset card describes labels as the mean of three distribution samples from a teacher endpoint of roughly 4B-class capability. Scores measure agreement with that teacher. Its 0.735 self-agreement is a reference statistic, not a mathematical accuracy ceiling. Learning the teacher's preferences can exceed that number without proving better real-world decisions. The majority baseline of 0.520 over all 1,600 cases also differs from the 0.461 baseline of the 400-case test split. S18

Application tasks

Strong email results do not establish reliable general content safety. Toxicity accuracy is about 0.53 and Macro-F1 about 0.40, which does not support using the model as the sole safety layer. Domain classification also does not directly identify the cheapest sufficiently capable LLM for any request.

Speed needs a device and a timing boundary

These are Tesla T4 measurements, not a universal “33 ms” for every CPU, phone or server. A local process and a remote API include different network, queue, sequence-length and batching overhead. Speed ratios are scenario-dependent.

Large option sets and option order

The official Banking77 application experiment reports 0.425 for English, 0.425 for multilingual and 0.492 for typed-decisions. Many options share a small question-head token budget, shortening label descriptions and removing distinguishing information. The SDK does not have a hard 20-option interface limit; “roughly twenty or fewer” is practical advice for current defaults.

Possible approaches include a larger head/total budget, coarse-to-fine classification, or candidate retrieval. predict_shortlist uses caller-supplied embeddings to retrieve top-k candidates before deciding. That adds retrieval error and computation; it is no longer an all-options, end-to-end single forward pass. Test shuffled option order too: the official 20-option MASSIVE English experiment reports answer-flip rates of 0.15 / 0.23. S24

10 / INDEPENDENT COMPARISONSIndependent comparisons using the same inputs

instax-dutta/sysone-bench fixes states, questions, seeds and model versions, checking question SHA values for matching inputs. Its September 21, 2026 README v2 table covers 751 states across nine suites. Laya uses the English checkpoint on a local M2 CPU; Jev uses the remote jev-1.13.0 API. S34

In the repository's v3 update, using the then-current Router raised Laya's multilingual intent score to 0.840. That test has only 25 samples. It suggests a direction; neither 0.840 nor 1.000 establishes broad language quality. Those English/Router runs also predate the code revision inspected here.

The results support neither “Laya beats Jev everywhere” nor “Laya has no value.” Laya performs strongly on AG News and MNLI, and its deployment and cost structure differ. The repository's REPORT.md still describes an earlier three-suite run with different figures. This guide keeps the named v2 table separate from that earlier report. S35

Reading a cross-vendor comparison

Control inputs, label counts, prompt lengths, calibration procedures, domain fine-tuning and model versions. Separate hardware execution from remote API latency: identical inputs improve accuracy comparisons but do not remove network or hardware differences. Small public tests still need a target-domain evaluation. The Coachix Jev guide covers the other model's interface and published limits.

11 / CHINESE & MULTILINGUALChinese and multilingual performance

“100+ languages” describes product and backbone coverage. The detailed public MASSIVE sweep covers 51 languages. “45 usable languages” means exceeding three times random accuracy: with twenty choices, random is 0.05 and three times that is only 0.15. This experimental threshold is not a production-quality standard. S04S06

This table and the fourteen-language T4 experiment are different sweeps. Combining their best per-language values would misrepresent the experiments. Above-random English-checkpoint performance on some Chinese examples does not make it the more reliable starting point; the model card recommends multilingual outside English.

For Chinese deployment, start with five sample groups: mixed simplified/traditional script, Chinese with English product names, colloquial support language and typos, domain abbreviations/numbers, and corrections at the end of long messages. Evaluate the language combination of state and instructions/criteria. Chinese state with English questions is not automatically equivalent to Chinese throughout. Preserve original text: translation can alter sentiment, negation and technical terms.

12 / RESEARCH PAPERSPapers: direct history, backbones and theory

As of September 23, 2026, official resources chiefly point to two earlier author preprints, current code, model cards and blogs. No clearly identified standalone technical paper fully documenting the training and evaluation of all three current Laya checkpoints was located in those materials. This is a boundary on the identifiable public material, not proof that no later paper exists.

A. SalesRLAgent: the earlier sales direction

SalesRLAgent: A Reinforcement Learning Approach for Real-Time Sales Conversion Prediction and Optimization. Nandakishor M, March 30, 2025, arXiv:2503.23303. Abstract · Full text · PDF.

The paper treats sales dialogue as a sequence for conversion prediction. It describes about 1.2 million synthetic conversations and 3072-dimensional Azure OpenAI embeddings, and reports results on that sales task. It helps explain the author's interest in probability trajectories and real-time decisions. It does not validate today's ModernBERT-plus-dynamic-option-head architecture. Its reported accuracy or conversion lift must not be relabeled as Laya performance. This interpretation uses the full text; preprint status does not establish peer review. S23

B. Confidence-Aware Routing: the earlier reliability direction

Confidence-Aware Routing for Large Language Model Reliability Enhancement: A Multi-Signal Approach to Pre-Generation Hallucination Mitigation. Nandakishor M, September 23, 2025, arXiv:2510.01237. Abstract · Full text · PDF.

This paper combines semantic alignment, layer convergence and learned confidence estimates to route requests toward local generation, RAG, a larger model or human review. That is a different design layer from choosing a Laya checkpoint by language. Its place in the author's research history does not make it a full training report for current RLCD models. Metrics belong to that paper's own experiments, not an independent reproduction in this guide. This interpretation uses the full text. S25

C. ModernBERT: the English backbone

Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference. Benjamin Warner and colleagues, 2024, arXiv:2412.13663. Paper.

The encoder-only model targets classification and retrieval, reporting 2T pretraining tokens and native 8192-token sequences. It explains why a classifier need not use a large decoder and the speed/memory choices of modern bidirectional encoders. Laya adds its own head and length constraints; backbone results are not Laya results. This account uses the abstract and official model material. S28

D. mmBERT: the multilingual backbone

mmBERT: A Modern Multilingual Encoder with Annealed Language Learning. September 8, 2025, arXiv:2509.06888. Paper · Model card.

The paper studies multilingual encoders and language-learning schedules. Its abstract describes 3T tokens and pretraining coverage exceeding 1,800 languages. That is a corpus-coverage statement, not evidence that Laya passed business evaluations in 1,800 languages. Laya multilingual uses mmBERT-base with additional decision components. This account uses the abstract and model card. S29S49

E. Calibration and reward theory

Also read MASSIVE (arXiv:2204.08582) and XNLI (EMNLP 2018) to understand what intent classification and cross-lingual inference measure. These are benchmark-background papers, not papers authored for Laya. S32S33

13 / BLOGS & ANNOUNCEMENTSReading the blogs and announcements

The official project website brings the checkpoints, performance, routing and deployment motivation together. It contains promotional claims and cannot replace the experimental methods or data. The author's DEV article provides more detail on the early architecture and reward function, but some naming still says “RL Agent.” Use current source code for API and probability-field semantics. S01S10

The article describes ordinary cross-entropy training as prone to overconfidence. That is a practical concern, but “cross-entropy cannot learn calibrated probabilities” is not a general mathematical result: the log score is itself proper. RLCD is this project's training design, not proof that it must be better calibrated than every supervised classifier. S27

TypeSafe's Jev launch post explains the industry context around choice, score, noul, System One and RLCD. Separate interface concepts, implementation and empirical comparison. Shared vocabulary does not establish identical internals. S22

The author's Hugging Face announcement helps identify the model and demo. Dataset sizes and training hardware described in release material and the Reddit post remain author statements, not independent verification. S44S50

Community runtime READMEs can be more useful than reposted summaries. MLX describes its timing boundary, C++ describes GPU backends, and Node documents ONNX weights and memory. The Chinese translation repository is an additional reading aid; resolve conflicts against the corresponding upstream revision. S39S40S41S43

14 / PYTHON QUICKSTARTA complete Python starting example

This example follows the public SDK interface for a local experiment. It prints actual returned values rather than supplying an invented prediction. Installation, weight downloads and inference were not run to produce the figures in this guide.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install 'laya==0.3.6'
USE_TF=0 python demo.py

USE_TF=0 is an upstream compatibility recommendation. In some environments with TensorFlow installed, Transformers' framework detection can interfere with model construction. It is not an acceleration algorithm or a required fix on every machine.

# demo.py
import json
import laya

agent = laya.load("convaiinnovations/laya", subfolder="multilingual")
state = {
    "subject": "Duplicate charge",
    "body": "The same order was charged twice. Please investigate. "
            "I have not requested a refund yet."
}
questions = {
    "department": {
        "type": "choice",
        "instructions": "Which team should handle this message?",
        "criteria": {
            "billing": "Charges, invoices and payment issues",
            "technical": "Application failures and bugs",
            "other": "Requests unrelated to billing or technical support"
        }
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the user explicitly request a refund now?"
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is the request based only on stated facts?",
        "criteria": [
            "Routine", "Time-sensitive",
            "Blocking or explicit immediate deadline"
        ]
    }
}
result = agent.predict(state, questions)
print(json.dumps(result, ensure_ascii=False, indent=2))
print("actual_device:", agent.device)

The example deliberately contains negation: “I have not requested a refund yet.” Check whether the model incorrectly turns a duplicate charge into an explicit refund request. All example text is in English here; to evaluate Chinese, replace the state with original Chinese customer text and compare the instruction-language combinations described above. The example only prints results. It issues no refund, sends no email and changes no account. The first run normally downloads weights; a downloaded local model directory can be used afterward.

Consuming the result

For choice, read answer["choice"] and the matching probabilities[label]; for noul, read answer["noul"]. Read confidence separately when using distribution concentration for routing. Provide explicit handling for missing state, unknown labels, unreliable predictions and exceptions.

Built-in router_questions(), guard_questions(), moderation_questions() and triage_questions() are reusable schemas. They simplify integration but do not establish task quality in your language or data. S48

A similar interface is not an OpenAI chat API

Python predict and system_one are library calls, not Chat Completions or Responses API endpoints. Some community servers expose a Jev-style /v1/systemone; deployment and behavior follow that implementation's documentation. The official Python package is not a ready-made OpenAI-compatible chat service.

15 / DEPLOYMENT & COSTDeployment, hardware, cost and privacy

The official SDK prefers available devices and can fall back to CPU when CUDA/MPS is unavailable or in some memory-error cases. The inspected version uses FP32 on CPU/MPS; CUDA precision also depends on device capabilities. Log the actual device: a cuda startup argument does not prove every request ran on a GPU. S12

CPU support does not mean every throughput requirement suits a CPU. Under concurrency, batching, queue latency, maximum text length and timeouts often matter more than a 33 ms example. Load the intended weights and warm the service before declaring it ready. A running process is not yet a model reliably serving requests.

Free weights do not mean zero total cost

Apache-2.0 code and weights reduce per-call API fees, but hardware, power, memory, operations, monitoring and evaluation remain costs. Compare deployment options using a task-specific estimate:

Monthly total ≈ compute + storage/network + operations time
                + labeling/evaluation + mistakes and human handoffs
Cost per useful decision = monthly total / decisions meeting the quality target

Low traffic and expensive GPU maintenance may favor an API. High volume, internal-only data or existing idle hardware may favor self-hosting. These are deployment trade-offs, not current price promises for a commercial service.

Local inference and network boundaries

Local inference can keep business inputs away from a model-cloud API. Initial weight downloads still use the network, and application logs, monitoring, error reports or subsequent LLM calls may send data elsewhere. An offline environment needs weights, dependencies and tokenizer prepared ahead of time, followed by observation of actual network behavior. Open source alone does not make the whole application offline.

16 / FINE-TUNING & EVALUATIONFine-tuning and designing your own evaluation

The official 2×T4 Kaggle notebook specializes the English model on typed-decisions. It includes mixed precision, gradient accumulation, gradient checkpointing, different encoder/head learning rates, distributed training and post-training temperature fitting. The website's approximately four-hour estimate belongs to a particular setup; it guarantees neither free GPU availability nor completion time for every account. S17

1. Define a specific question, such as whether a ticket explicitly requests a refund, rather than “understand the customer.”
2. Fix label descriptions and an abstention path. Handle multiple intents, missing evidence and contradictory state.
3. Build representative samples. Split by user, time or incident so paraphrases of one event do not cross training and testing.
4. Compare simple rules, a conventional classifier, base Laya and an appropriate generative model using the same inputs.
5. Fine-tune only if needed, checking whether gains extend beyond the training source.
6. Fit temperature and operational thresholds on an independent calibration split. Report final results on a frozen test set.
7. Initially log recommendations beside human outcomes. Observe drift and failure types before expanding automation.

The four typed-decisions workflows each have 300 training and 100 test cases: 1,200 / 400 total, with five questions per case. Two thousand decisions are not two thousand independent user events. Questions sharing a state are correlated. S18

17 / APPLICATIONS & POOR FITSUseful application patterns—and poor fits

Support triage. Supply the message, relevant order state and allowed queues; identify the main intent and urgency before a person or LLM writes a response. Invoice and charge facts should come from the business database. A classification probability is not an accounting lookup.

RAG. Provide a query and candidate passage to judge whether the passage supports a proposition. Evaluate the complete retrieval/reranking system. If a long passage is truncated, “irrelevant” may mean the model never saw the evidence near its end.

Model routing. Classify into bounded categories such as coding, translation and ordinary questions, then use predefined handling paths. Selecting by quality and cost requires task-specific outcome and pricing data; identifying the subject alone is insufficient.

Agent traces. Provide organized tool results and state to flag review needs or stalled work. Aggregate long traces deterministically and locate evidence before inference, instead of burying critical errors at the end of the context.

Security-alert assistance. Prioritize queues or suggest review to analysts alongside deterministic rules, dedicated detectors and human action. Published guardrail/moderation figures do not establish a comprehensive autonomous safety system.

Documents and invoices. A fixed approve/hold/reject recommendation can be one component. OCR, amount calculations, field matching and permission checks still need separate handling. A model score does not authorize a payment.

Poor direct replacements: complex multi-step planning, free writing, open-ended factual question answering, detailed legal interpretation of long documents, live-market fact checking, tool execution and cross-system transactions. Wrapping these jobs in several choice questions does not supply the missing reasoning or execution system.

18 / COMMUNITY RUNTIMESCommunity runtimes and projects

These are third-party implementations, not official performance guarantees. Their original READMEs explain platform constraints, setup, licenses and numerical compatibility.

For a port, check both selected-label agreement and probability-distribution agreement. FP16, quantization, tokenizer differences and temperature handling can alter confidence-based routing even when most argmax labels stay the same. Interface compatibility is not identical behavior.

19 / LIMITS & OPEN QUESTIONSMisconceptions and remaining unknowns

Three documentation boundaries deserve attention. First, BENCHMARKS.md references research/results/app_benchmark.json, which was missing from the inspected main tree and returned 404. The application table is traceable to the summary and script, not a claimed re-computation of every raw result. Second, the historical sales model and dataset are under DeepMostInnovations. Their model-card training-size descriptions differ from the 2025 paper; read each version separately rather than merging them into current Laya facts. Third, rapid repository updates mean old scoreboards do not automatically cover today's temperature clamps, routing and compatibility fixes. S06S20

Open questions include the complete base-training data inventory and splits, independent cross-domain calibration, real Chinese business evaluations, long-context and large-label stability, probability parity across ports, and a full training report for the current family. Missing public information should remain an open question.

20 / FAQ & READING PATHSQuestions and reading paths

Is Laya an LLM?

It uses pretrained language encoders and belongs to the language-model ecosystem. “Chat-style generative large language model” does not accurately describe its interface. A non-autoregressive typed-decision model is more precise.

Can it replace Jev directly?

Some input/output concepts overlap, and compatible community services exist. Behavior, option budgets, latency, deployment and support differ. Replay the same states/questions through both and compare correctness, probability quality and total cost before deciding to migrate.

Does it run on a Mac?

The official code includes MPS/CPU paths, and an MLX port exists. Check the device actually loaded and measure your samples. This guide does not present a benchmark of the reader's or author's computer.

Does it browse, remember or call tools?

Those are application functions. The surrounding system supplies state, retrieval and execution. Laya contributes a judgment within that workflow.

Can it read long chat histories?

Default total budgets are 512/1024 tokens, shared with the question head. Define the relevant time window and evidence, or design chunking and aggregation. Pasting an entire log does not mean every line was processed.

Where should I start reading?

Concept path: sections 1, 3, 5 and 19 → official website → model cards. Learn the input/output contract and the conditions behind the promises.

Engineering path: sections 4, 7, 8, 14 and 15 → agent.py, common.py, router.py → a small integration → your evaluation.

Research path: sections 6, 9, 10, 12 and 16 → the two earlier author papers → ModernBERT/mmBERT → calibration and proper scoring → raw official T4 results and independent sysone-bench.

All three paths lead to one practical question: is Laya better suited than your current system under your labels, languages, latency budget and cost of mistakes? For the larger assistant design, continue with Coachix's model-selection module, knowledge layer and tool connections. Open the Coachix console to work on your assistant; the console link is not a hosted Laya inference endpoint.

21 / GLOSSARYGlossary

22 / SOURCESSources and further reading

The fifty references below distinguish official material, implementation, papers, independent experiments and community ports. Each entry explains its relevance and reading scope. Citation numbers in the article jump to the matching entry. Ten papers provide research history or technical background; two other same-name papers are included only for disambiguation.

50 of 50 references

No references match. Try a shorter term or select all types.

1. S01 · Official · Full text
   Official Laya website

   Project positioning, checkpoint family and deployment motivation. Read performance claims with the experimental conditions.

2. S02 · Official · Full text
   Official Laya GitHub repository

   SDK, routing, presets, license and limits. The main branch changes over time.

3. S03 · Models and data · Full text
   Laya English and family model card

   Three checkpoints, weights, architecture, training summary and limitations.

4. S04 · Models and data · Full text
   Laya Multilingual model card

   mmBERT-based checkpoint, language sweep and configuration.

5. S05 · Models and data · Full text
   Laya Typed Decisions model card

   Domain specialization; its scores must not be attributed to the base checkpoint.

6. S06 · Benchmarks · Full text
   Official BENCHMARKS.md

   Language, application, latency, calibration and limitation tables from several experiments.

7. S07 · Official · Entry and metadata
   Laya demo Space

   Online interface; shared-resource waits and cold starts differ from warm inference.

8. S08 · Official · Package metadata
   PyPI laya

   Official package; observed release 0.3.6 and Python requirement of at least 3.10.

9. S09 · Code and configuration · Full text
   PyPI release and dependency JSON

   Upload times, Python compatibility, dependencies and release history.

10. S10 · Blogs and discussion · Full text
    Author's DEV article: I Built Non-Autoregressive Decision Models…

    September 18, 2026 article on the early architecture and RLCD. Compare naming and marketing claims with current code.

11. S11 · Code and configuration · Full text
    common.py: sequences, decision heads, rewards and confidence

    Computation path, truncation, entropy confidence and temperature limits.

12. S12 · Code and configuration · Full text
    agent.py: loading and typed output

    Device fallback, batching, choice/score/noul fields and usage accounting.

13. S13 · Code and configuration · Full text
    router.py: selection priority

    Workflow detection is disabled by default; language routing, preloading and eviction.

14. S14 · Code and configuration · Full text
    English rl_agent_config.json

    512/192 token budgets and temperatures, interpreted with the SDK's current clamps.

15. S15 · Code and configuration · Full text
    Multilingual rl_agent_config.json

    1024/256 budgets; type temperatures initially one, without fitted option-count buckets.

16. S16 · Code and configuration · Full text
    Typed-decisions rl_agent_config.json

    Specialist length limits, temperature and training metadata.

17. S17 · Code and configuration · Full text
    2×T4 Kaggle fine-tuning notebook

    DDP, noisy policy gradients, soft cross-entropy, calibration and testing. Code was read, not executed for this guide.

18. S18 · Models and data · Full text
    LocalLLaMA/typed-decisions dataset card

    Four workflows, splits, soft teacher labels, baselines and metric definitions.

19. S19 · Benchmarks · Full text
    Official raw T4 results

    JSON for the 17,416-question experiment, including samples and metrics.

20. S20 · Benchmarks · Full text
    bench_apps.py: application benchmark

    Dataset sources and task construction. The cited app_benchmark.json was missing from the inspected tree.

21. S21 · Benchmarks · Full text
    bench_latency.py: latency and loading

    Separates language detection, warm calls, cold switching and model residency.

22. S22 · Blogs and discussion · Full text
    TypeSafe: Introducing System One Models and Jev

    First-party explanation of a related product, not evidence of Laya implementation or priority.

23. S23 · Papers · Full paper
    SalesRLAgent, 2025

    Earlier sales-conversion RL work; the full HTML was read. Not the current Laya technical report.

24. S24 · Code and configuration · Full text
    shortlist.py: candidate retrieval

    Caller-provided embeddings narrow candidates before classification; consider recall loss and added computation.

25. S25 · Papers · Full paper
    Confidence-Aware Routing, 2025

    Earlier multi-signal reliability routing; different from today's language-checkpoint Router.

26. S26 · Papers · Abstract and method overview
    Guo et al.: On Calibration of Modern Neural Networks

    ICML 2017 background on temperature scaling, calibration and overconfidence.

27. S27 · Papers · Abstract and definitions
    Gneiting and Raftery: Strictly Proper Scoring Rules

    JASA 2007 mathematical foundations for probabilistic prediction and proper scores.

28. S28 · Papers · Abstract and model material
    ModernBERT: Smarter, Better, Faster, Longer

    English backbone paper; encoder context length is not Laya's default total length.

29. S29 · Papers · Abstract and model material
    mmBERT: A Modern Multilingual Encoder with Annealed Language Learning

    Multilingual backbone; pretraining language counts are not business-task coverage.

30. S30 · Papers · Abstract
    BERT: Pre-training of Deep Bidirectional Transformers

    Bidirectional encoding and MLM background; option-marker inference is described in Laya code.

31. S31 · Papers · Abstract
    Proximal Policy Optimization Algorithms

    PPO background; do not equate the current notebook's entire training recipe with PPO.

32. S32 · Papers · Abstract
    MASSIVE: A 1M-Example Multilingual NLU Dataset

    Background for the 51-language intent task and its evaluation scope.

33. S33 · Papers · Abstract
    XNLI: Evaluating Cross-lingual Sentence Representations

    Cross-lingual natural-language inference, not open-ended question answering.

34. S34 · Benchmarks · Full text
    sysone-bench README: independent same-input comparison

    Nine-suite v2 table and v3 Router update; local and API latency still include different overhead.

35. S35 · Benchmarks · Full text
    sysone-bench REPORT: earlier three-suite run

    An older run than README v2; preserve its sample and uncertainty boundaries.

36. S36 · Name disambiguation · Project description
    The other Laya desktop assistant

    Notification management and action cards, not the decision model covered here.

37. S37 · Name disambiguation · Abstract
    LAYA: Layer-wise Attention Aggregation

    Gennaro Vessio's output-head aggregation research; a different model identity.

38. S38 · Name disambiguation · Abstract
    Laya: A LeJEPA Approach to EEG

    An EEG representation model with different authors and tasks.

39. S39 · Community runtimes · Full text
    mizorewww/laya-mlx

    Apple Silicon runtime; inspect M3 Max setup, timing boundaries, versions and probability parity.

40. S40 · Community runtimes · Full text
    lkarlslund/laya.cpp

    Native ggml inference with CUDA/Vulkan; distinguish this repository from similarly named ports.

41. S41 · Community runtimes · Full text
    receptron/laya

    Node/TypeScript ONNX implementation with its own weight and memory requirements.

42. S42 · Community runtimes · Full text
    AXERA-TECH/Laya

    NPU3 package: batch one, sequence 256, up to four options and dedicated hardware.

43. S43 · Community runtimes · Entry and description
    yangshun2005/laya-cn

    Chinese reading aid; its revision can lag upstream.

44. S44 · Blogs and discussion · Announcement
    Author's Hugging Face Laya announcement

    First-party model description and demo links; training-size claims remain author statements.

45. S45 · Models and data · Full text
    Historical SalesRLAgent model card

    DeepMostInnovations' earlier PPO sales model has different interfaces, licensing and data claims.

46. S46 · Models and data · Dataset card
    Historical SaaS sales-conversations dataset

    Earlier project data, not a complete training inventory for today's three checkpoints.

47. S47 · Benchmarks · Full text
    Laya research README

    Experiment scripts, raw results and explicit limits of non-simultaneous Jev comparisons.

48. S48 · Code and configuration · Full text
    presets.py: built-in question sets

    Model routing, guardrail, moderation and triage schemas; presets do not guarantee task quality.

49. S49 · Models and data · Full text
    jhu-clsp/mmBERT-base model card

    Multilingual backbone configuration, pretraining background and usage.

50. S50 · Blogs and discussion · Announcement and retrieved post
    Author's r/LocalLLaMA Laya announcement

    Author statement of 25,000+ human-annotated examples, not independent verification of complete training data. The original research used the announcement excerpt; the post was reopened for this English edition.