Vyakti Research
Intelligence can be replaced. A relationship cannot. We study the state, memory, perception and evaluation systems that make continuity testable.
One completed web preprint and one research note still collecting its primary comparison. Their status is part of the result.
Web preprint
Six frontier judges were backtested against verdicts that had already decided which model serves a live product. All six failed, and the bar itself turned out to sit above the ground truth's own measured ceiling.
Research note in progress
The question it is built to answer: does a byte-identical compiled context survive a model swap, or does the serving model set the ceiling on whether a companion stays recognisable?
No result is published. The primary comparison remains incomplete.
Read the research status| Judge | Agreement | 95% CI (cluster-bootstrap) | Verdict |
|---|---|---|---|
| gpt-5.6-terra | 54.2% (52/96) | [43.8%, 64.6%] | FAIL |
| grok-4.3 | 34.4% (33/96) | [25.0%, 43.8%] | FAIL |
| DeepSeek-V4-Pro | 30.9% (29/94) | [20.7%, 41.5%] | FAIL |
| Mistral-Large-3 | 29.2% (28/96) | [20.8%, 38.5%] | FAIL |
| DeepSeek-V4-Flash | 28.1% (27/96) | [18.8%, 39.6%] | FAIL |
| Cohere command-a-plus | 0 scorable units | not plottable | DISQUALIFIED |
| claude-opus-4.8 (ceiling, not a candidate) | 77.1% (74/96) | [67.7%, 84.4%] | CEILING |
| claude-opus-5 (invalid, not a result) | 100.0% (17/17) | parse-selected denominator | INVALID |
Pre-registered bar ≥80%. Chance baselines: uniform-random 30.5%, pure slot-A 21.9%, both derived from the archived verdict distribution. Cohere command-a-plus is absent from the plot because 158 of 192 calls failed to parse, which is a disqualification for cause rather than a low score.
Five connected questions define the relational layer. We treat them as one system, not a list of companion features.
Character that holds.
Preferences are easy. A coherent identity is harder: a voice, point of view, values and contradictions that remain recognisable without preventing growth.
History with meaning.
Memory should do more than retrieve facts. It should recognise what mattered, to whom and why, so a relationship can continue instead of restart.
More than the words.
Conversation also lives in tone, timing, expression, context and what goes unsaid. We build models that reason across the signals people choose to share.
One state. Many signals.
Language, voice, gaze, facial motion and reaction should feel like expressions of the same underlying moment, not separate models performing beside one another.
Initiative within boundaries.
A companion should be capable of curiosity, reflection and initiative without becoming controlling. Agency must remain legible and interruptible.
Three empirical findings and one systems measurement. Each value keeps its sample, method, date and source attached.
Empirical finding
0 / 31,122A retrieval-time database predicate produced zero leaks across 31,122 checks. The same privacy rule expressed only as a prompt instruction leaked in most tested scenarios.
Empirical finding
20.4% → 41.7%engaged-on-a-stop (+21.3 pp)
A matched-arm study found that more proactive screen commentary roughly doubled engagement, with no statistically detected rise in fabrication in the powered follow-up.
Empirical finding
74 / 96 = 77.1%95% CI [67.7%, 84.4%]
A trusted AI judge reproduced only 74 of its own 96 prior verdicts. The measured ceiling sat below the qualification bar fixed before the run.
Systems measurement
9.2×cheaper with caching enabled ($0.0017 against $0.0160 per turn, identical turn)
On measured production calls, prompt caching reduced the cost of an otherwise identical turn by roughly nine times.
Method
Vyakti is a three-person team. Our research output is small by design and every entry in it is load-bearing: pre-registered before we saw the data, retracted in public when our own controls proved us wrong, and released with the harness that produced it.
A judge favours its own vendor's model, by roughly 16×.
A between-judge control killed it. A judge with no vendor conflict at all showed a larger effect, which is the opposite of what the favoritism explanation predicts. The agreement failure stands; only the causal attribution is withdrawn.
context/measurements.md grok43-favoritism-retracted; docs/paper/CAMERA.md §4.4Six frontier judges failed because the material is code-switched.
The identical 96 units, machine-translated to monolingual English and re-judged: −3.1 to +6.6 pp, every interval overlapping its Hinglish counterpart. Inside our own 13.6 pp noise floor. It is not the code-switching.
context/measurements.md r4-english-control; docs/paper/CAMERA.md §4.5Artifact package
The full judge-qualification protocol, harness and dataset behind the paper “It's Not the Code-Switching”. Released so other teams evaluating LLM judges on affective or open-ended preference tasks can qualify their own candidates against a real backtest rather than assuming trustworthiness.
Built and internally gated; public repository URL pending alongside the paper's arXiv posting.
We want to work with researchers in speech, conversation analysis, memory, evaluation and long-horizon systems.
Work with Vyakti