Explainability in Practice: A Survey of Explainable NLP

Abstract

The rapid advancement of Natural Language Processing (NLP) models, particularly advanced models like GPT and BERT, has led to their widespread adoption in critical decision-making across healthcare, finance, and customer relationship management. However, the black-box nature of these advanced NLP models has created an urgent need for transparency and trustworthiness.

This survey examines how explainable NLP (XNLP) can be deployed in real-world contexts while addressing domain-specific challenges. We focus on underexplored areas including real-world applicability, evaluation metrics, and human interaction in model assessment.

Our work aims to bridge knowledge gaps in XNLP literature through domain-specific exploration, addressing practical deployment challenges, and suggesting future research directions for broader XNLP application.

Key Contributions

Domain-Specific Exploration

Examination of explainable NLP across critical domains including healthcare, finance, and customer relationship management.

Real-World Applicability

Focus on practical deployment challenges and how XNLP methods can be effectively implemented in real-world contexts.

Evaluation Metrics

Analysis of underexplored areas in evaluation metrics for explainable NLP systems.

Future Directions

Suggestions for future research directions to enable broader XNLP application and address current knowledge gaps.

Application Domains

Healthcare

Clinical decision support and medical diagnosis

Finance

Risk assessment and fraud detection

Customer Management

Sentiment analysis and service optimization

Key Findings

1 Advanced NLP models (GPT, BERT) are increasingly used in high-stakes domains where transparency is essential for trust and accountability.
2 The black-box nature of these models creates an urgent need for explainability methods in real-world deployments.
3 Key gaps exist in real-world applicability of XNLP methods, standardized evaluation metrics, and human interaction in model assessment.
4 Different application domains require tailored explainability approaches to address their unique requirements and constraints.

Citation

@article{mohammadi2025explainability,
  title={Explainability in Practice: A Survey of Explainable NLP Across Various Domains},
  author={Mohammadi, Hadi and Bagheri, Ayoub and Giachanou, Anastasia and Oberski, Daniel L.},
  journal={arXiv preprint arXiv:2502.00837},
  year={2025}
}