This paper presents and evaluates two computational approaches for parsing A Dictionary of Hiberno-English (Terence Patrick Dolan)(Dolan, 2006) into a structured, machine-readable lexical resource. Although the dictionary exhibits a high degree of editorial consistency, its entries remain computationally challenging to structure (Atkins & Rundell, 2008) . Lexical information is distributed across semi-regular templates that are frequently disrupted by redirects, variant spellings, homonym numbering, multi-part grammatical labels, numbered senses, phrase entries, and entries with missing or atypical pronunciation markers. Further complexity arises because region codes, informant identifiers, editorial asides, etymologies, examples, and cross-references are often embedded within running prose or parenthetical material rather than isolated in stable positions. As a result, extraction requires field recognition and careful boundary resolution, normalization, and disambiguation (Ide & Véronis, 1995).
To address these challenges, we implement and compare two complementary approaches: a rule-based parser and a large language model (LLM)-assisted extraction pipeline(Brown et al., 2020; Jurafsky & Martin, 2023). The rule-based system uses deterministic segmentation and pattern recognition to identify core lexical components, including headwords, pronunciation, variants, part-of-speech labels, definitions, etymologies, examples, and cross-references, while explicitly handling recurrent irregularities such as redirect entries, variant forms, numbered senses, compound grammatical labels, and embedded metadata. The generative pipeline, by contrast, employs schema-driven prompting, encoding the target data model within the prompt itself targeted at field-consistent outputs and reducing ambiguity in the model’s interpretation of semi-structured inputs and edge cases(Ji et al., 2023).
To enable systematic comparison, we develop an evaluation framework that captures multiple dimensions of extraction quality, including (1) field-level presence and absence, (2) exact string agreement between methods, (3) token-level similarity using both ordered and unordered matching, and (4) source coverage through token- and character-level containment checks targeted at detection of overgeneration and omissions. These measures function as lightweight proxies for identifying hallucination-like behaviour in generative outputs, without requiring exhaustive manual annotation.
Our evaluation reveals distinct performance profiles across the two approaches. The rule-based system achieves high precision and strong source fidelity, producing outputs that remain consistently traceable to the input text, but is sensitive to formatting variation and requires substantial manual rule engineering. By contrast, the LLM-based system demonstrates greater robustness to irregular structures and achieves higher recall on complex entries but exhibits variability in output length and occasional divergence from the source text. These behaviours are reflected in overgeneration patterns and reduced containment scores, highlighting trade offs between flexibility and control.
By systematically comparing these approaches, we show that LLM-based extraction can produce usable lexical data when guided by schema-driven prompting and evaluated through explicit, field-aware metrics. Rather than treating generative outputs as opaque, we demonstrate how they can be assessed through multi-dimensional measures, including field
level validation, token similarity, and source coverage. In the wake of recent LLM-driven approaches, our findings suggest that some aspects of lexical structuring can be achieved with reduced reliance on extensive rule engineering, opening new possibilities for working with semi-structured historical resources.
References
Atkins, B. T. S., & Rundell, M. (2008). The Oxford Guide to Practical Lexicography. Oxford University Press.
Brown, T. B., Mann, B., Ryder, N., & al, et. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS). Dolan, T. P. (2006). A Dictionary of Hiberno-English: The Irish Use of English (2nd ed.). Gill & Macmillan.
Ide, N., & Véronis, J. (1995). Encoding Dictionaries. In Text Encoding Initiative: Background and Context. Kluwer Academic Publishers.
Ji, Z., Lee, N., Frieske, R., & others. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys.
Jurafsky, D., & Martin, J. H. (2023). Speech and Language Processing.