Tacet α
Our first structured decision model. It reads a state and a typed question, then reads the answer straight from its output logits. One forward pass per question, no generated text.
| Base | Gemma 4 26B A4B IT (mixture of experts, 4B active) |
|---|---|
| Method | LoRA, option cross-entropy plus a small KL penalty to the base |
| Access | Tacet API, model id tacet-alpha |
| Calibration | Built in. Probabilities come back calibrated. |
| Options per question | 2 to 26 |
| Get started | Quickstart |
Training results
| Measurement | Base | Option CE | Option CE + KL 0.1 |
|---|---|---|---|
| Accuracy | 74.22% | 80.30% | 80.48% |
| Brier score (lower is better) | 0.486 | 0.289 | 0.287 |
| Text drift vs base (lower is better) | 0 | 0.803 | 0.690 |
Fine-tuning adds 6.26 accuracy points. The KL penalty cuts measured text drift by 14% compared with cross-entropy alone, while accuracy stays about the same.
Context: published Jev results
Accuracy by task, next to third-party measurements of TypeSafe’s Jev published in the Kev benchmark repo. Samples, prompts, and option order differ, so treat this as context, not a head-to-head ranking.
| Task | Tacet α | Published Jev |
|---|---|---|
| BoolQ | 92.00% | 92.50% |
| QNLI | 94.33% | 92.50% |
| PAWS | 94.00% | 78.75% |
| MMLU | 83.00% | 90.00% |
| SciQ | 100.00% | 98.75% |
| HellaSwag | 90.33% | 89.33% |
| CLINC | 94.33% | 97.33% |
| DBpedia | 100.00% | 95.00% |
| SST-5 | 56.00% | 63.75% |
| Yelp | 62.00% | 66.25% |
| When2Call | 66.33% | 76.00% |
Known limits
- Tool-use decisions are weak. On When2Call it scores 66.3%, below the base model’s 70%.
- Speed and cost aren’t measured yet. Skipping text generation helps, but each question still re-reads the full state.
- Calibration was measured on our test set. Check it on your own traffic before you rely on a threshold.