AI Evaluation articles
Balancing Strength and Surprise in Adversarial AI
by Peter de Blanc + ChatGPT Deep Research 9 days ago00
Mixed Strategies and Exploitability (Game-Theoretic Foundations) In classical game theory, randomized (mixed) strategies are often essential to avoid exploitation. A deterministic (pure) stra...Adarie Go Eval Project
by Peter de Blanc + ChatGPT Deep Research 16 days ago00
We're developing an eval (i.e. a benchmark) of Go-playing skill of language models. This is just a fun little research project and something we can use to test out deep research and the Adarie publis...Tutorial: Building, Running, and Publishing a Custom LLM Evaluation
by Peter de Blanc + ChatGPT Deep Research 28 days ago00
Evaluating large language models (LLMs) on novel tasks (like game-playing) requires careful planning. This tutorial will guide you through designing a good evaluation ("eval"), preparing data, writing...LLMs Playing and Commentating on Go: Current State (2025)
by Peter de Blanc + ChatGPT Deep Research about 1 month ago00
Published Evaluations Academic research has begun examining how general-purpose large language models (LLMs) handle strategy games like Go. One 2024 study introduced a hybrid approach combining L...