Hadi Mohammadi
Menu
Blog

Blog

Field notes on building and evaluating language-model systems, written from production work and from research, plus the open guides I keep up to date for people studying AI and data science. Everything here is free to read, takes corrections by pull request, and is archived with a DOI so it can be cited.

01Field notes

Recent writing

Short empirical pieces, usually built around one measurement on public data: what an LLM judge does when nobody is checking, what a benchmark’s annotation layer looks like underneath, how small a gap a benchmark can resolve, what a model means when it says it is 95% sure. Each one comes with the code that produced its numbers.

  1. What 106,134 arena votes can and cannot order

    A leaderboard is a ranked list, and the ranked list is what everyone reads. On the public LMArena sample, the votes separate one neighbouring pair out of 54.

    6 min read

  2. What a context window forgets

    A sliding window at 2,048 tokens has the supporting turn in front of the model for 8% of 1,527 questions. Searching the transcript first takes that to 60% for the same money.

    5 min read

  3. Ask a model how sure it is

    Three local models answered 1,000 MMLU questions each and rated their own confidence from 0 to 100. In 3,000 answers, a number under 80 came up eight times.

    5 min read

  4. What a benchmark can actually tell apart

    HumanEval catches a true three-point gap about one time in seven. The floor for five familiar benchmarks, worked out from their item counts.

    4 min read

  5. How much text a watermark needs before it shows up

    A circulating simulation of a simplified watermark puts detection at roughly two hundred words. The arithmetic checks out. What it assumes about prose does not.

    6 min read

  6. The labels under the benchmark

    211,225 crowd ratings sit under GoEmotions. On 27 of its 28 emotions, the raw annotation layer falls below the reliability floor the field normally requires.

    5 min read

  7. What an LLM judge actually does when you are not looking

    Three local models graded the same arithmetic, with one answer provably correct. One picked whichever answer was shown first 91% of the time.

    5 min read

02Series

Modern AI Engineering

Six chapters on the stack I use daily, from calling models well to serving them cheaply, with the trade-offs stated plainly and the failure modes named. They come from shipping LLM systems in production, mostly matching, ranking and content generation for a recruitment marketplace, and from doctoral research on explainable NLP and LLM evaluation.

Each chapter stands alone. If you already build with LLMs, start with Evaluation: it is the discipline most teams add last and need first.

  1. Chapter 1

    Calling LLMs well

    Structured outputs, streaming, retries, caching and cost control: the unglamorous 80%.

    4 min read

  2. Chapter 2

    Retrieval-augmented generation

    Chunking, hybrid search, rerankers, and how RAG actually fails.

    4 min read

  3. Chapter 3

    Agents and MCP

    Tool loops, the Model Context Protocol, and when not to build an agent.

    4 min read

  4. Chapter 4

    Fine-tuning and preference optimization

    Prompting vs. RAG vs. LoRA vs. DPO/GRPO: a decision guide, then the mechanics.

    4 min read

  5. Chapter 5

    Evaluation and LLM-as-judge

    The chapter closest to my research: evals that predict production, judges you can trust.

    4 min read

  6. Chapter 6

    Serving and inference optimization

    vLLM, quantization, KV caches, and latency and cost budgets that hold.

    4 min read

The series on GitHub DOI RSS feed

03Guides

Guides I keep up to date

These are living documents rather than posts: lists and paths that only stay useful if somebody maintains them. Each one has its own repository, checks its links weekly, and takes corrections by issue or pull request.

Guide

AI & Data Science Master’s Programmes in the Netherlands

Every master’s in artificial intelligence, machine learning and data science at a Dutch research university, with how admissions actually work, what it costs, and how to choose. I did my PhD in the Netherlands and stayed; this is the overview I wanted when I was comparing options. Every programme links to its official page, checked weekly.

mohammadi-hadi/ai-masters-netherlands · MIT

Read the guide DOI
Guide

Machine Learning Summer Schools in Europe

Which schools exist, how to get in, how to fund it, and how to get real value out of the week. Written from four of them: the Explainable AI Summer School at TU Delft, OxML, the Generative Modeling Summer School in Eindhoven, and a LERU doctoral school.

mohammadi-hadi/ml-summer-schools-europe · MIT

Read the guide DOI
Guide

ML Learning Paths

The question is never “what courses exist”. It is “I want this job, what do I do next”. Five ordered paths, for ML scientist, ML engineer, LLM engineer, data engineer and data scientist. Every entry says why it is there, and every path can be completed for free.

mohammadi-hadi/ml-learning-paths · MIT

Read the guide DOI
Reading list

Awesome Explainable NLP

A curated list of papers, tools and resources on explainability and interpretability for NLP and large language models: feature attribution, probing, counterfactuals, faithfulness, and evaluation. Companion to my survey of explainable NLP across domains.

mohammadi-hadi/awesome-explainable-nlp · CC0 1.0

Browse the list DOI The survey
04Elsewhere

Related

Opening the Black Box is a free self-paced course in explainable AI for large language models: thirteen labs that run in the browser. The same material is taught as a five-day summer school for universities and teams. The playground lets you poke at a model’s explanations without writing any code.

NL Student Support is a separate free directory I maintain of the support services at 49 Dutch higher education institutions: psychologists, counsellors, study advisers, disability and financial support.

Weekend Learning is the LinkedIn newsletter that goes with this blog: one issue each Saturday, built around one post or one package from here, with an exercise you can finish in about an hour.

Corrections and additions to any of this are welcome as an issue or a pull request on the repository. New writing appears in the RSS feed.