Author: Rivas AI Lab

Latent‑Space Chess Planning with Supervised Contrastive Learning Achieves 2593 Elo

We train a transformer encoder with supervised contrastive learning so that a 6‑ply beam search reaches 2593 Elo, rivaling Stockfish with far less computation.

We embed chess positions into a continuous space where distance mirrors evaluation. By moving toward an “advantage vector” in that space, our engine plans moves without deep tree search, delivering super‑human strength with a tiny search budget.

TL;DR

  • We replace deep tree search with planning in a learned latent space.
  • Our engine reaches an estimated 2593 Elo using only a 6‑ply beam search.
  • The approach is efficient, interpretable, and scales with model size.

Why it matters

Traditional chess engines such as Stockfish rely on exhaustive tree search that explores millions of positions and requires heavy hardware. Human grandmasters, by contrast, use intuition to prune the search space and then look ahead only a few moves. Replicating that human‑like intuition in an AI system could dramatically reduce the computational cost of strong play and make powerful chess agents accessible on modest devices. Moreover, a method that plans by moving through a learned representation is potentially transferable to any domain where a sensible state evaluation exists—games, robotics, or decision‑making problems.

How it works (plain words)

Our pipeline consists of three intuitive steps.

  1. Learning the space. We train a transformer encoder on five million positions taken from the ChessBench dataset. Each position carries a Stockfish win‑probability. Using supervised contrastive learning, the model pulls together positions with similar probabilities and pushes apart those with different probabilities. The result is a high‑dimensional embedding where “nearby” boards have similar evaluations.
  2. Defining an advantage direction. From the same training data we isolate extreme states: positions that Stockfish rates as forced checkmate for White (probability = 1.0) and for Black (probability = 0.0). We compute the mean embedding of each extreme set and subtract them. The resulting vector points from Black‑winning regions toward White‑winning regions and serves as our “advantage axis.”
  3. Embedding‑guided beam search. At run time we enumerate all legal moves, embed each resulting board, and measure its cosine similarity to the advantage axis. The top‑k (k = 3) most aligned positions are kept and expanded recursively up to six plies. Because the score is purely geometric, the engine prefers moves that point in the direction of higher evaluation, effectively “walking” toward better regions of the space.

The entire process requires no hand‑crafted evaluation function and no recursive minimax or Monte‑Carlo tree search. Planning becomes a matter of geometric reasoning inside the embedding.

What we found

Elo performance

We evaluated two architectures:

  • Base model. 400 K training steps, 768‑dimensional embeddings, beam width = 3.
  • Small model. Same training regime but with fewer layers and a 512‑dimensional embedding.

When we increase the search depth from 2 to 6 plies, the Base model’s estimated Elo improves steadily: 2115 (2‑ply), 2318 (3‑ply), 2433 (4‑ply), 2538 (5‑ply), and 2593 (6‑ply). The Small model follows the same trend but stays roughly 30–50 points behind at every depth. The 2593 Elo estimate at depth 6 is comparable to Stockfish 16 running at a calibrated 2600 Elo, yet our engine performs the search on a single GPU in a fraction of the time.

Scaling behaviour

Both model size and embedding dimensionality contribute positively. Larger transformers (the Base configuration) consistently outperform the Small configuration, confirming that richer representations give the planner better navigation cues. Early experiments with higher‑dimensional embeddings (e.g., 1024 D) show modest additional gains, suggesting a ceiling that will likely rise with even bigger models.

Qualitative insights

We visualized thousands of positions using UMAP. The plot reveals a clear gradient: clusters of White‑advantage positions sit on one side, Black‑advantage positions on the opposite side, and balanced positions cluster near the origin. When we trace the embeddings of actual games, winning games trace smooth curves that move from the centre toward the appropriate advantage side, while tightly contested games jitter around the centre. These trajectories give us a visual proof that the embedding captures strategic progress without any explicit evaluation function.

Interpretability

Because move choice is a cosine similarity score, we can inspect why a move was preferred. For any position we can project its embedding onto the advantage axis and see whether the engine is pushing toward White‑dominant or Black‑dominant regions. This geometric view is far more transparent than a black‑box evaluation network that outputs a scalar score.

Key equation

    \[ L = -\sum_{i=1}^{N}\frac{1}{|P(i)|}\sum_{p\in P(i)}\log\frac{\exp(\mathbf{z}_i\cdot\mathbf{z}_p/\tau)}{\sum_{a\in A(i)}\exp(\mathbf{z}_i\cdot\mathbf{z}_a/\tau)} \]

Here, \mathbf{z}_i is the embedding of the i‑th board state, P(i) denotes the set of positives (positions whose Stockfish evaluations differ by less than the margin = 0.05), A(i) is the full batch, and \tau is the temperature parameter. This supervised contrastive loss pulls together positions with similar evaluations and pushes apart those with dissimilar evaluations, shaping the latent space for geometric planning.

Limits and next steps

Current limitations

  • Greedy beam search. With a beam width of three, the search cannot revise early commitments. Long‑term tactical ideas that require a temporary sacrifice can be missed.
  • Training target dependence. Our contrastive objective uses Stockfish evaluations as ground truth. While this provides high‑quality numerical signals, it may not capture the nuanced strategic preferences of human players.

Future directions

  • Replace the greedy beam with more exploratory strategies such as wider or non‑greedy beams, Monte Carlo rollouts, or hybrid search that combines latent scoring with occasional shallow alpha‑beta pruning.
  • Fine‑tune the embedding with reinforcement learning, allowing the engine to discover its own evaluation signal from self‑play rather than relying solely on Stockfish.
  • Scale the transformer to larger depth and width, and enrich the positive‑pair sampling (e.g., include mid‑game strategic motifs) to sharpen the advantage axis.
  • Apply the same representation‑based planning to other perfect‑information games (Go, Shogi, Hex) where a numeric evaluation can be generated.

FAQ

What is “latent‑space planning”?
It is the idea that an agent can decide which action to take by moving its internal representation toward a region associated with higher value, instead of exploring a combinatorial tree of future states.
Why use supervised contrastive learning instead of ordinary regression?
Contrastive learning directly shapes the geometry of the space: positions with similar evaluations become neighbours, while dissimilar positions are pushed apart. This geometric structure is essential for the cosine‑similarity scoring used in our search.
How does the “advantage vector” get computed?
We take the mean embedding of forced‑checkmate positions for White (p = 1.0) and the mean embedding of forced‑checkmate positions for Black (p = 0.0) and subtract the latter from the former. The resulting vector points from Black‑winning regions toward White‑winning regions.
Can this method replace Monte‑Carlo Tree Search (MCTS) in AlphaZero‑style agents?
Our results show that, for chess, a well‑structured latent space can achieve comparable strength with far shallower search. Whether it can fully replace MCTS in other domains remains an open research question, but the principle of geometric planning is compatible with hybrid designs that still retain some tree‑based refinement.
Is the engine limited to Stockfish‑derived data?
In its current form, yes; we use Stockfish win‑probabilities as supervision. Future work plans to incorporate human annotations or self‑play reinforcement signals to reduce this dependency.

Read the paper

For a complete technical description, training details, and additional visualizations, see our full paper:

Learning to Plan via Supervised Contrastive Learning and Strategic Interpolation: A Chess Case Study

If you prefer a direct download, the PDF is available here: Download PDF

Reference

Hamara, A., Hamerly, G., Rivas, P., & Freeman, A. C. (2025). Learning to plan via supervised contrastive learning and strategic interpolation: A chess case study. In Proceedings of the Second Workshop on Game AI Algorithms and Multi‑Agent Learning (GAAMAL) at IJCAI 2025 (pp. 1–7). Montreal, Canada.

A Unified Framework for Fair Counterfactual Explanations: Benchmarking, Scalability, and Human‑Centric Design

We propose a unified evaluation framework for counterfactual explanations that balances fairness, plausibility, and scalability, and we outline next steps for research and practice.

In this work, we combine a systematic mapping of existing literature with a concrete benchmark suite. Our goal is to make counterfactual explanations both fair and actionable across high‑dimensional, real‑world domains.

TL;DR

  • We introduce a unified evaluation framework that simultaneously measures plausibility, actionability, and legal compliance of counterfactual explanations.
  • Our benchmark suite covers large‑scale, high‑dimensional datasets (e.g., Lending Club, HMDA, KKBox) and demonstrates that current methods struggle with scalability and causal validity.
  • The framework emphasizes human‑in‑the‑loop assessment, causal grounding, and open‑source tooling to bridge research and industry.

Why it matters

Machine‑learning models increasingly drive decisions about credit, hiring, health care, and criminal justice. When a model denies a loan or predicts a high risk score, affected individuals often request an explanation. Counterfactual explanations answer the question “What would need to change for a different outcome?” While attractive, existing methods use ad‑hoc metrics, such as sparsity or proximity, that are hard to compare across domains. Without a common yardstick, we cannot reliably assess whether an explanation is fair, plausible, or legally compliant (e.g., under the GDPR’s “right‑to‑explanation”). Moreover, many approaches ignore the causal structure of the data, leading to explanations that suggest impossible or socially undesirable changes. Finally, many counterfactual generators are designed for low‑dimensional toy data and collapse when applied to real‑world, high‑dimensional workloads.

How it works

Our approach proceeds in three stages.

  1. Systematic literature mapping. We performed a systematic mapping study (SMS) of peer‑reviewed papers, industry reports, and open‑source toolkits that discuss bias detection, fairness metrics, and counterfactual generation. This gave us a consolidated view of which methods exist, what datasets they have been tested on, and which fairness notions they address.
  2. Construction of a unified metric suite. Building on the discussion points identified in the literature, we defined three orthogonal axes:
    • Plausibility: does the suggested change respect real‑world domain constraints?
    • Actionability: can a user realistically achieve the suggested change?
    • Legal compliance: does the explanation satisfy GDPR‑style minimal disclosure requirements?

    Each axis aggregates several concrete measures (e.g., feasibility checks, causal consistency tests, and robustness to distribution shift) that have been repeatedly highlighted across the surveyed papers.

  3. Benchmark suite and open‑source integration. We assembled a set of widely used, high‑dimensional datasets, Adult, German Credit, HMDA, Lending Club, and KKBox, and wrapped them in a reproducible pipeline that evaluates any counterfactual generator on all three axes. The suite is released under a permissive license and directly plugs into existing fairness toolkits such as AI Fairness 360.

What we found

Applying our framework to a representative sample of ten counterfactual generation techniques revealed consistent patterns:

  • Unified metrics are missing. No prior work reported all three axes together; most papers focused on either sparsity or statistical fairness alone.
  • Scalability is limited. Optimization‑based approaches that work on the Adult dataset (≈30 K rows, 14 features) become infeasible on Lending Club (> 2 M rows, > 100 features) without dimensionality‑reduction tricks.
  • Causal grounding is rare. Only a small minority of methods explicitly encode causal graphs; the majority treat features as independent, which leads to implausible suggestions (e.g., decreasing age while increasing income).
  • Human evaluation is under‑explored. Few studies incorporated user‑centric metrics such as trust or perceived fairness, despite repeated calls in the literature for human‑in‑the‑loop design.
  • Open‑source tooling is fragmented. Toolkits like AI Fairness 360 provide bias metrics but lack integrated counterfactual generators; conversely, counterfactual libraries focus on explanation generation but not on fairness assessment.

These findings motivate the need for a single, extensible benchmark that can be used by researchers to compare methods and by practitioners to validate deployments.

Limits and next steps

Our study has several limitations that also point to promising research directions.

  • Dataset concentration. Most benchmark datasets are classic tabular collections (Adult, German Credit, HMDA). While they span finance, health, and criminal justice, additional domains such as education or environmental policy remain under‑represented.
  • Causal knowledge acquisition. We assume that a causal graph can be obtained from domain experts or from causal discovery algorithms. In practice, constructing accurate causal models at scale is still an open problem.
  • Dynamic real‑world environments. Our benchmark captures static snapshots of data. Future work should test explanations under distribution shift and over time, as highlighted by robustness‑to‑distribution‑shift concerns.
  • Human‑centered evaluation. Our current human‑in‑the‑loop studies are limited to small user studies. Scaling user feedback to millions of decisions will require novel crowdsourcing or interactive UI designs.

To address these gaps we propose the following next steps:

  1. Expand the benchmark to include under‑explored domains (e.g., sustainability, public policy) and multimodal data (text, images).
  2. Develop hybrid methods that combine optimization‑based counterfactual generation with causal constraints, reducing implausible suggestions.
  3. Integrate the benchmark into existing fairness toolkits (AI Fairness 360, What‑If Tool) to provide a one‑stop shop for fairness‑aware explanation evaluation.
  4. Design large‑scale user studies that measure trust, perceived fairness, and actionable insight across diverse stakeholder groups.

FAQ

What is a counterfactual explanation?
A counterfactual explanation describes the minimal changes to an input that would flip the model’s prediction, answering “What if …?” for the user.
Why do we need a unified framework?
Existing works evaluate explanations with disparate metrics, making it impossible to compare fairness, plausibility, and legal compliance across methods or domains.
Can my model’s explanations be legally compliant without a causal model?
Legal requirements such as GDPR emphasize that explanations should reflect realistic, causally possible changes. Ignoring causality can lead to implausible or misleading counterfactuals, risking non‑compliance.
How does the framework handle high‑dimensional data?
We include scalability tests that measure runtime and memory on datasets with hundreds of features. Our results show that many current methods need dimensionality‑reduction or approximation to remain tractable.

Read the paper

For the full technical details, benchmark specifications, and exhaustive literature review, please consult the original publication.

Jui, T. D., & Rivas, P. (2024). Fairness issues, current approaches, and challenges in machine learning models. International Journal of Machine Learning and Cybernetics, 1–31. Download PDF