Poke the Black Box
Type a sentence. Watch a model decide whether it sounds positive or negative, see exactly which words tipped it, test whether that explanation actually holds, and then try to break it. This is explainable AI (XAI) in five minutes, live.
Everything runs in your browser. Nothing you type leaves this page.Watch it decide
The model below reads movie-review sentences and predicts positive or negative. It is deliberately simple: a logistic regression over single words, trained on 67,349 sentences from the SST-2 dataset (81.8% validation accuracy, seed 42). Simple is the point. For this model the explanation is exact: every word carries a learned weight, the weights add up, and that sum is the decision. Type anything, or start from an example.
POSITIVE · 77%
negativepositive
Which words did it?
pushes positive
pushes negative
neutral or unknown
Show the arithmetic (the whole model, no hand-waving)
Weights add to a score, the score goes through a sigmoid, and that is the probability above. This table is the complete computation, which is why this model is called a glass box.
Now try the third example, The film is not good. The model calls it positive. It sees a bag of words: good pushes up harder than not pushes down, and word order does not exist here. Hold that thought for Part 4. Deeper: chapter 4 of the book covers attribution for models where nothing is exact anymore.
Test the explanation
An explanation is a claim: these words carried the decision. Claims get tested. The standard check is a deletion test: remove the words the explanation says matter most, strongest first, and watch the model’s confidence. If the explanation is honest, confidence should collapse quickly, much faster than deleting random words. Evaluating explanations like this is what chapter 8 of the book, and my own research, is about.
For this glass-box model the test looks almost rigged: the explanation and the model are the same object, so it cannot fail. Keep that as your baseline. For deep models, Part 4, the explanation is a separate estimate, and this same test is the main honesty check we have.
Flip it
The sharpest question you can ask a model is: what is the smallest change that would change your mind? That change is called a counterfactual explanation. Edit the sentence above and try to flip the verdict in as few word changes as you can. The counter tracks how far you have drifted from your starting point.
One more experiment: starting from a positive sentence, try flipping it by only adding not in front of the nicest word. On this model it usually fails, because a bag of words has no idea what the not is attached to. Counterfactuals are chapter 6 of the book; using them to attack models is chapter 7.
Now the real thing
Real systems are not glass boxes. DistilBERT is a transformer with 66 million parameters, fine-tuned on the same task (its reported validation accuracy is 91%, against our 81.8%). Load it and it runs locally in your browser through WebAssembly; your text still goes nowhere. But there is no weight table to show you now. To explain it we have to probe from outside: remove each word, rerun the model, and measure what moved. That is occlusion, one of the attribution methods from chapter 4.
How it all fits together
This page is the two-minute version of a larger body of work on explaining language models.
You are here · 2 minutes · free
The playground
See, test, and break an explanation, live in your browser.
Self-paced · free
The course
Opening the Black Box: a guided five-day path through thirteen hands-on labs on a free GPU.
The code · MIT
The labs
Thirteen open-source notebooks: the evidence base, runnable and citable.
The reference
The book
Hands-On Explainable AI: the full treatment, written lab-first.
For institutions
The school
The instructor-led five-day summer school, hosted at your university or company.
Model card, honest limitations, and provenance
- Glass box. Logistic regression over unigram counts, SST-2 (GLUE split), 67,349 training and 872 validation sentences, seed 42. Shipped vocabulary pruned to the 6,000 strongest-weight terms, which scores 81.8% on validation (the full 13,244-term model scores 81.7%). The weights file is 105 KB and you are welcome to read it in your browser’s network tab.
- Deep model. distilbert-base-uncased-finetuned-sst-2-english (Apache-2.0), downloaded by your browser from the Hugging Face Hub only if you click load, and run locally.
- Domain. Both models were trained on English movie-review sentences. On other topics they still answer, but they are out of their depth; that is a lesson, not a bug.
- Quirks are real. Some learned weights are dataset accidents, like a positive weight on the word calls. And in movie reviews, devastating is praise.
- Estimates are estimates. The transformer’s occlusion heatmap is a probe from outside, not a readout of its internals. Chapter 8 of the book is about how such estimates are evaluated, and chapter 5 about why attention maps are not the answer either.
- Demo, not a product. Nothing here is a moderation or decision tool.
- Every number on this page comes from a logged training run (2026-08-07). The page verifies its shipped weights against reference outputs on load and refuses to run if they drift.