Creating Bespoke HTR Solutions for Complex Historical Manuscripts with Generative AI

Started in 1998, the Newton Project (https://www.newtonproject.ox.ac.uk/) was one of the first initiatives to provide full texts of historical documents online and in open access. Ever since, the Project has amassed around 12 million words that were authored by Isaac Newton or closely related to him. In connection to the tercentenary of Isaac Newton’s death in 2027, an ambitious new project has started under the aegis of the Newton Project: the production of an AI-enabled digital edition of the three versions of Newton’s masterpiece Naturalis Philosophiae Principia Mathematica (1687, 1713, 1726), together with material relating to their conception and revision. One of the chief milestones of the Newton Principia Project is the transcription of manuscripts related to the Principia and housed at the Cambridge University Library, and in particular MS 3965. This manuscript comprises c. 1530 pages, most of it handwritten by Isaac Newton himself. While some of these are written in relatively fair hand, the majority are drafts that contain heavy corrections, tables and calculations, some of which are jotted down almost chaotically on any piece of free paper that Newton could find. 

The proposed presentation focusses on our design of the optimal MS 3965 transcription workflow, involving both automatic transcription and manual correction. The design involved the empirical testing of different transcription AI-based models before settling on a bespoke solution involving two generative AI (Anthropic Claude and Google Gemini Pro 3.0), with incorporation of the LaTex-language outputs of a math-specialised LLM (MathPix). Claude was employed as virtual assistant and computer vision specialist, while Gemini was used for its transcription capacity. By interacting with Claude and Gemini, we were able to create a 5-tier classification system for the pages in the manuscript, together with specific workflow for each of the tiers. The complexity of the transcription task and the possibilities of generative AI further led to the AI-assisted construction of a bespoke transcription platform that optimises the interactions between human and the AIs and models used, while capturing the complexity of the manuscript pages (multiple corrections, changes, stains and blots, deteriorated paper, complex mathematical notation). The platform is designed in view of creating TEI XML code that includes genetic markup and genetic encoding. We consider that the classification system and the transcription platform could in due course provide a methodology for triaging manuscript pages, transcribing and capturing them in complex TEI-XML standard.