UNIMATRIx - The standard v1 first benchmarks

Many single-answer benchmarks are crap at evaluating agents. In thestandard-v1 recipe of UNIMATRIx every decision changes a world that keeps moving. And this is a good measure of what a model is capable of.

In UNIMATRIx each LLM are tested in eight domains: forecasting, negotiation, cooperation, shared resources, relationships, delegation, knowledge and recovery.

Each case produces three event-derived normalized metrics. The case points are calculated with this formula:

case points = max(0, 100 × (model outcome − passive) / (anchor − passive))

A first session of menchmarking#

To extract the first bunch of benchmark, I used four models in parallel on the same LM Studio instance, temperature 0, inference seed 42.

On gslf.it all the table with this results.