Recognising Text, Recognising Processes

eXplainable Automatic Text Recognition for Scottish Spiritualist Newspapers

In October 2025, Dr Joe Nockels began his role as the National Library of Scotland’s Digital Research Fellow for 2025/26. His project explores how AI-enabled transcription can work with “eXplainable” developmental principles to help non-technical users understand how transcription models reach results [1].

Recent reports have indicated a steady rise of AI adoption within libraries, although this remains mostly exploratory [2].At the same time, there is a need for greater AI literacy and confidence among library practitioners to extend their understanding beyond the notion of programming towards broader socio-technical issues of usability, system development and maintenance [3].To reach this technical fluency, AI outputs – like digital transcriptions – need further contextualisation, especially considering libraries’ anxieties around legitimising inadequate systems due to their status as trustworthy information repositories. With this context in mind, Joe’s project asks how far such “eXplainability” should be applied to the seemingly innocuous process of semi-automatic transcription? This will include familiar tools such as Handwritten Text Recognition (HTR) and advanced Optical Character Recognition (OCR).

Moving beyond AI’s proven ability to make collections more readable, the project will assess to what degree eXplainable AI (XAI) is required for national libraries to automatically transcribe collections at scale. It will also explore whether added “eXplainability” impacts transcription model performance. This follows technical papers citing increasing efforts to make their processes more interpretable as compromising precision and scalability [4].

The project will use Data Foundry samples of The Spiritualist Newspaper (1869–1882), a key record of those who believed they could communicate with the deceased, to test six predominant AI transcription tools for accuracy. These tools range from community-built workflows to commercial products. The Spiritualist is a fitting example: just as mediums claimed spirits produced “automatic writing” during séances, AI transcription can appear equally abstract in its “creative” process. Alongside this technical experimentation, Joe will examine whether HTR/OCR developers provide clear public documentation that aligns with XAI principles. These findings will direct staff training at the National Library of Scotland, likely to occur in January, in a preferred transcription approach and real-world recommendations for how to select, implement and test XAI on library collections.

Spiritualist images, as well as the resulting transcriptions, will be interactively available for further comparison via the Digital Humanities Institute. Joe will also be presenting his initial findings at AI4LAM’s Fantastic Futures Conference, hosted at the British Library in December.

Project Team

  • Dr. Joe Nockels (National Library of Scotland Digital Research Fellow, DHI)
  • Dr Sarah Ames (Digital Scholarship Librarian, National Library of Scotland)
  • Jamie McLaughlin (Senior Research Software Engineer, DHI)

References

[1] J. Van Wessel (2020). AI in Libraries: Seven Principles, https://zenodo.org/records/3865344

[2] A. Cox (2021). The impact of AI, machine learning, automation and robotics on the information profession. CILIP. 1-56. https://www.cilip.org.uk/page/researchreport; L. Dalgleish (2022). Artificial Intelligence, cultural heritage and the National Library of Scotland, https://data.nls.uk/projects/artificial-intelligence-report/; Clarivate (2025), https://clarivate.com/pulse-of-the-library/

[3] T. Padilla (2019). Responsible Operations: Data Science, Machine Learning, and AI in Libraries. OCLC, https://www.oclc.org/research/publications/2019/oclcresearch-responsible-operations-data-science-machine-learning-ai.html [4] Z.C. Lipton (2016) ‘The Mythos of Model Interpretability’, Paper presented at 4th ICML Workshop on Human Interpretability in Machine Learning, June 23, 2016, New York. 1-8. doi: 10.48550/arXiv.1606.03490. S. Ali et al. (2023) ‘Explainable Artificial Intelligence (XAI): What We Know and What Is Left to Attain Trustworthy Artificial Intelligence’, Information Fusion, 99: 1-52. doi: 10.1016/j.inffus.2023.101805.