Hands-On Explainable AI

Interpreting, Evaluating, and Trusting Large Language Models

A lab-first book about opening language models up: feature attribution, attention, probing, chain-of-thought faithfulness, mechanistic interpretability, behavioral audits, and what it takes to run all of that in production. Every number in the text comes from a notebook you can run yourself.

working draft, v0.2 · 13 chapters · 424 pages · 123 figures · private page, please don’t share the link

13 / 13

Chapters with executed labs

Each chapter is written from a notebook that actually ran. The labs fit a free Colab GPU, and the book quotes their real outputs, including the messy ones.

0.95 → 0.515

Honest results, kept in

Chapter 1 ships a model at 95% accuracy that collapses to coin-flipping after deployment. The ten-line explanation that would have caught it is the point of the book.

32 / 32

Audit your judges

Our LLM judge preferred whichever answer came first, in all 32 trials. Chapter 8 shows how to catch that before a leaderboard quietly inherits it.

Read the draft

The manuscript itself is private. Enter the access key to decrypt and read it here.

AES-256-GCM · decrypted only in your browser · nothing is sent anywhere

draft Open in new tab Download PDF

If the preview stays blank on your device, use Download or Open in new tab.

What’s inside

Part I

The Lay of the Land

Why accuracy is not understanding, a working map of every method family, and a tour of the transformer with the places explanations attach.

Part II

The Explanation Toolkit

SHAP, LIME, and gradients on one shared model; attention read honestly; probes and counterfactuals; then a case study that breaks its own pipeline, and the chapter that scores the explanations themselves.

Part III

Explaining the Model Itself

Whether chain-of-thought reasoning is faithful, what circuits and sparse autoencoders actually recover, and how to audit model values against real survey data.

Part IV

Building Explainable LLM Systems

Training for faithfulness with DPO, then production: RAG with citation audits, drift monitors that fire before accuracy falls, and the EU AI Act mapped to the tools you now have.

About the author

I work on explainable NLP: how to interrogate language models, how to score the answers, and how to tell when an explanation is lying. This book turns that research into something you can run. Its thirteen labs are already public and free, along with a self-paced course built on them, Opening the Black Box, and a two-minute playground, Poke the Black Box.