Blog

Field notes on building and evaluating language-model systems, written from production work and from research, plus the open guides I keep up to date for people studying AI and data science. Everything here is free to read, takes corrections by pull request, and is archived with a DOI so it can be cited.

Field notes

Recent writing

Short empirical pieces, usually built around one measurement on public data: what an LLM judge does when nobody is checking, what a benchmark’s annotation layer looks like underneath, how small a gap a benchmark can resolve, what a model means when it says it is 95% sure. Each one comes with the code that produced its numbers.

  1. Conference map

    NeurIPS and EMNLP moved closer in 2026

    Both conferences, 2025 and 2026, on one map. Under a quarter of the papers sat in ground both share in 2025, almost a third in 2026, and agents drove the biggest part of the rise.

  2. Conference map

    What 5,534 EMNLP 2026 papers are about

    Every EMNLP 2026 paper on one map, with the room and time of the talks and posters in Budapest. One in four papers now mentions agents, up from one in eight last year.

  3. Conference map

    What 9,110 NeurIPS 2026 papers are about

    I put every NeurIPS 2026 paper on one map you can search and click, and compared it with last year. Papers on LLM agents more than doubled their share.

  4. A/B tests

    4,873 experiments and the winner’s curse

    Upworthy ran thousands of headline A/B tests and published the raw data. Nearly one in six fails a basic sanity check on its own traffic split.

  5. Leaderboards

    What 106,134 arena votes can and cannot order

    A leaderboard is a ranked list, and the ranked list is what everyone reads. On the public LMArena sample, the votes separate one neighbouring pair out of 54.

  6. Long context

    What a context window forgets

    A sliding window at 2,048 tokens has the supporting turn in front of the model for 8% of 1,527 questions. Searching the transcript first takes that to 60% for the same money.

  7. Calibration

    Ask a model how sure it is

    Three local models answered 1,000 MMLU questions each and rated their own confidence from 0 to 100. In 3,000 answers, a number under 80 came up eight times.

  8. Benchmarks

    What a benchmark can actually tell apart

    HumanEval catches a true three-point gap about one time in seven. The floor for five familiar benchmarks, worked out from their item counts.

  9. Watermarks

    How much text a watermark needs before it shows up

    A circulating simulation of a simplified watermark puts detection at roughly two hundred words. The arithmetic checks out. What it assumes about prose does not.

  10. Annotation

    The labels under the benchmark

    211,225 crowd ratings sit under GoEmotions. On 27 of its 28 emotions, the raw annotation layer falls below the reliability floor the field normally requires.

  11. LLM judges

    What an LLM judge actually does when you are not looking

    Three local models graded the same arithmetic, with one answer provably correct. One picked whichever answer was shown first 91% of the time.

Before the blog

Research since 2023

The research behind the field notes goes back to 2023: papers, talks and code from my PhD on explainable NLP at Utrecht University, each linked to its public record.

  1. Jun 2023

    Introduction to Transformers, BERT and Explainable NLP

    A hands-on workshop at the Skills Lab of the M&S Research Hay Day, Utrecht University.

  2. Jul 2023

    Code for online sexism detection at EXIST 2023

    BERT, XLM-RoBERTa and DistilBERT for the EXIST 2023 tasks, archived on Zenodo; the paper is in the CLEF 2023 working notes.

  3. Sep 2023

    AI-generated text detection using ensemble and combined model training

    A talk at CLIN33, the Computational Linguistics in the Netherlands meeting in Antwerp.

  4. Sep 2024

    A transparent pipeline for identifying sexism in social media

    Explainability combined with model prediction, in Applied Sciences.

  5. Feb 2025

    Explainability in practice: a survey of explainable NLP across domains

    Seven application domains, from medicine and finance to human resources: what kind of explanation each needs, which methods are used and how they are evaluated.

  6. Jun 2025

    Explainability-based token replacement on LLM-generated text

    SHAP and LIME find the tokens an AI-text detector leans on, four replacement strategies rewrite them, and an ensemble detector is built to withstand it.

  7. Jul 2025

    The reliability of LLM annotations, demographic bias and model explanation

    At the GeBNLP workshop, ACL 2025.

  8. Oct 2025

    EvalMORAAL: chain-of-thought and LLM-as-judge evaluation of moral alignment

    The moral alignment of 20 language models against the World Values Survey and Pew surveys; published at *SEM 2026.

  9. Dec 2025

    Best Oral Presentation Award, IOPS Winter Conference

    For work on cultural moral alignment in LLMs.

  10. Dec 2025

    Evaluating GRPO and DPO for faithful chain-of-thought reasoning

    Whether two fine-tuning methods make a model’s chain of thought reflect how it actually reached its answer; published at INLG 2026.

The full list is on publications, and the talks, posters and grants are on research.

Series

Modern AI Engineering

Six chapters on the stack I use daily, from calling models well to serving them cheaply, with the trade-offs stated plainly and the failure modes named. They come from shipping LLM systems in production, mostly matching, ranking and content generation for a recruitment marketplace, and from doctoral research on explainable NLP and LLM evaluation.

Each chapter stands alone. If you already build with LLMs, start with Evaluation: it is the discipline most teams add last and need first.

  1. Chapter 1

    Calling LLMs well

    Structured outputs, streaming, retries, caching and cost control: the unglamorous 80%.

  2. Chapter 2

    Retrieval-augmented generation

    Chunking, hybrid search, rerankers, and how RAG actually fails.

  3. Chapter 3

    Agents and MCP

    Tool loops, the Model Context Protocol, and when not to build an agent.

  4. Chapter 4

    Fine-tuning and preference optimization

    Prompting vs. RAG vs. LoRA vs. DPO/GRPO: a decision guide, then the mechanics.

  5. Chapter 5

    Evaluation and LLM-as-judge

    The chapter closest to my research: evals that predict production, judges you can trust.

  6. Chapter 6

    Serving and inference optimization

    vLLM, quantization, KV caches, and latency and cost budgets that hold.

Guides

Guides I keep up to date

These are living documents rather than posts: lists and paths that only stay useful if somebody maintains them. Each one has its own repository, checks its links weekly, and takes corrections by issue or pull request.

Guide

AI & Data Science Master’s Programmes in the Netherlands

Every master’s in artificial intelligence, machine learning and data science at a Dutch research university, with how admissions actually work, what it costs, and how to choose. I did my PhD in the Netherlands and stayed; this is the overview I wanted when I was comparing options. Every programme links to its official page, checked weekly.

mohammadi-hadi/ai-masters-netherlands · MIT

Guide

Machine Learning Summer Schools in Europe

Which schools exist, how to get in, how to fund it, and how to get real value out of the week. Written from four of them: the Explainable AI Summer School at TU Delft, OxML, the Generative Modeling Summer School in Eindhoven, and a LERU doctoral school.

mohammadi-hadi/ml-summer-schools-europe · MIT

Guide

ML Learning Paths

The question is never “what courses exist”. It is “I want this job, what do I do next”. Five ordered paths, for ML scientist, ML engineer, LLM engineer, data engineer and data scientist. Every entry says why it is there, and every path can be completed for free.

mohammadi-hadi/ml-learning-paths · MIT

Reading list

Awesome Explainable NLP

A curated list of papers, tools and resources on explainability and interpretability for NLP and large language models: feature attribution, probing, counterfactuals, faithfulness, and evaluation. Companion to my survey of explainable NLP across domains.

mohammadi-hadi/awesome-explainable-nlp · CC0 1.0

Elsewhere

Related

Opening the Black Box is a free self-paced course in explainable AI for large language models: thirteen labs that run in the browser. The same material is proposed as a five-day summer school for universities and teams in 2027. The playground lets you poke at a model’s explanations without writing any code.

NL Student Support is a separate free directory I maintain of the support services at 49 Dutch higher education institutions: psychologists, counsellors, study advisers, disability and financial support.

Weekend Learning is the LinkedIn newsletter that goes with this blog: one issue every other Saturday, built around one post or one package from here, with an exercise you can finish in about an hour. Every issue so far is listed with the commands to rerun it.

«یادگیری آخر هفته» is its Persian sister, for beginners: one basic idea from machine learning and language models every other Friday, as a LinkedIn newsletter and as lessons you can read, play with, run in plain Python or download for offline use.

Corrections and additions to any of this are welcome as an issue or a pull request on the repository. New writing appears in the RSS feed.