Learning

The value of machine-assisted teaching editions: Creating activities to engage students in contemporary Chaucer studies

1. Introduction

In the previous chapter we discussed whether generative AI can act as a co-pilot for learning more about Chaucer’s literature. We sifted through several examples which highlighted the challenges of prompting LLMs, in producing outputs of meaning for users wanting to know more about the medieval world, or for editors looking to improve access to and engagement with medieval texts. 

The main problem we identified, which is well-known to those working regularly with generative AI technologies, is that LLMs do not interpret text in a human-like way, and lack awareness about what it has produced regardless of the extensiveness of its medieval ‘training’. Not only do LLMs, expectedly, lack anything resembling human cognition, but the model also does not consistently follow a user’s instructions within the prompt. Chatbots are built to probabilistically scope its training data and identify generalised patterns from a statistical distribution, with little consistency in output, a process which does not uncover nuanced (present or past) human experience (see the origins of generative AI chatbots in Stone, Goodlad and Sammons’ 2024 paper). This makes replicating this type of process impossible for an editor or educator wishing to utilise LLMs for their practice, given randomness and probability are at the core of these technologies. We must therefore focus on consistency and rigour in our investigations and critique of LLMs, raising questions of new technologies that generate content in ways that do not reflect naturally-occurring, human outputs.

The previous chapter showed that there is variability in the accuracy of the response, and this occurred with and without the use of human-generated encyclopaedic knowledge from Wikidata in the prompt. While this process of RAG (retrieval-augmented generation) assisted in the direction of the LLM’s response (e.g. by selecting specific extracts from the Chaucerian tale to be commented on), there was an overall sense that increased editorial workload may not always lead to a higher quality response. If there was success, it raised the question as to whether the use of the technology resulted in a lengthier editorial process. As Conrad and Goodlad (2024) note, knowing how to write optimal prompts to achieve better quality responses, as a student looking to improve their career prospects, is not necessarily more rewarding than learning to challenge the way in which LLMs are built and respond. They state the former endeavour is “the digital equivalent of attempting to teach students to ride a bike without ever taking off the training wheels” (Conrad and Goodlad 2024: ¶17).

The only way in which generative AI can act as a co-pilot for learning about Chaucer is if its outputs are probed thoroughly and with investigative intent, knowing the high likelihood of falsities (e.g. see ‘LLM Outputs are Fictions’ by Hanlon 2024). Generative AI might lead researchers down particular paths by raising more general methods of analysing Chaucer’s works, but the editor must focus their attention on the contemporary discussions within the field and interrogate problematic responses produced by new technologies, all while keeping the target audience – the student, in this case – in mind. Given the number of LLM limitations we have already addressed, related to translating Middle English, listening to and speaking Middle English, and creating meaning through interpretations and representations of Chaucer’s works, the learning activities in this chapter provide a solution in an age of Artificial Narrow Intelligence (ANI). These activities form a more rigorous method for students to challenge the development of these technologies, particularly in producing a faithful, yet engaging and interactive, version of one of Chaucer’s tales.

It is ultimately up to the editor as to whether the endeavour of prompting LLMs might be useful for their editorial practice (and their students using the digital resource) or whether it proves to be a hindrance. To counteract the disparities and tensions possible between long-standing editorial practices, and ‘AI-assisted’ editorial practices, we show in this chapter an example of a student activity involving the probing of LLM’s reproduction of and commentary on Chaucerian literature, which can lead to engaging and thought-provoking discussion on AI within the framework of a digital teaching edition.

1.1 What do students need to probe LLMs?

A screenshot of the Rutgers Critical AI literacies document for teachers, which states: 
"Teaching Critical AI Literacies: 
“Explainer” and Resources for the New Semester
(Last updated October 20, 2024) 
This germ of this document was prepared by Lauren M. E. Goodlad and Sharon Stoerger in collaboration with the Rutgers AI Round Table Advisory Council and the Office of Teaching Evaluation and Assessment Research. 
(Special thanks to Emily M. Bender for her advice on an early draft.) 
It is now a living document maintained by Critical AI @ Rutgers 
in conjunction with the Critical AI editorial team and our new DESIGN JUSTICE LABS initiative. 
We welcome your feedback!
For  a new “starter kit” containing resources for teaching critical AI literacies including a Student Guide, click here and download as much as you wish.

Click here for the recorded sessions of our Thursday September 12, 2024 event
RESEARCH IN THE ERA OF GENERATIVE AI: A Hybrid Symposium for 
Design Justice Thinkers

For video recordings of the complementary 10/6/23 event, 
CRITICAL AI LITERACY IN A TIME OF CHATBOTS:  
A Public Symposium for Educators, Writers, and Citizens, click here. 
For video recordings from our DESIGN JUSTICE AI Global Humanities Institute at the University of Pretoria, click here. 

Below we offer a set of resources to help instructors make informed decisions and equip themselves for productive discussions with students, colleagues, and the general public. 
“AI” is a complicated subject with many contexts and far-reaching implications: we have sought to strike a balance between brevity and comprehensiveness. 

(1-2) introduction to “artificial intelligence” and “generative AI” 
(3) critical AI literacies and actually existing harms 
(4) student learning and academic integrity
(5) suggestions for updating assignments and syllabi
(6) “living” list of potential resources."
The Rutgers Critical AI living document for educators (Goodlad & Stoerger 2024-).

There are a number of key areas which editors (who, in this context, have a pedagogical focus) should consider before opening up the possibility of generative AI testing in the classroom or for assignments. The Critical AI team at Rutgers University regularly maintain a living document for teaching critical AI literacies, providing helpful definitions and ideas for classroom activities that involve probing these technological developments. Some of these activities involve understanding: the anthropomorphistic view of AI through analysis of fictional AI in popular films (such as Blade Runner, Ex Machina, and Westworld); why citation practices are vital to research from a practical and ethical perspective; the design of technologies and who should be involved in these conversations; the existing harms of generative AI as well as the concerns generated by these harms; and comparisons between synthetic summaries of assigned readings versus the actual reading (Goodlad and Stoerger 2024). These activities could be adapted depending on the discipline.

For students who have received training to interpret Chaucer’s works, including learning about Middle English and its translations, the historical context of the stories they worked with, the way in which language can be used to describe different concepts and identities has changed over time, or the different literary interpretations of these texts, there are opportunities to critique generative AI outputs while advancing their knowledge of the text. This is a particularly challenging activity to pitch and is dependent on the level of expertise of the students, which we point to in upcoming research specific to generative AI’s interpretation of marginalised stories, through the character of the Pardoner (Whittle, 2024, submitted). We provided some beginner-level activities in our edition which might ease students into discovering more about Chaucer’s works and their reception, along with probing generative AI responses.

What Chaucer-specific knowledge might students need to know before probing LLM responses? We identified three areas as examples of where elements of an AI-assisted edition would need to be critiqued further. Our activities explore LLM’s translations, text-to-speech recordings of Chaucerian Middle English poetry and its pronunciation and prosody, and the way in which the ‘original text’ and new interpretations of Chaucer are represented through time, including a space to consider how generative AI may draw on or adapt Chaucer’s literature in new outputs.

Translation Activity Screenshot: Role of Women in the Pardoner’s Tale
Screenshot of a part of our translation activity, specific to the role of women in the Pardoner's tale. The screenshot includes an image from medieval manuscripts of women in the Middle Ages, with the following activity underneath: 
"The Pardoner discusses the context of the medieval tavern, where the rioters begin their story:

152 And right anon thanne comen 
153 Fetys and smale and yonge 
154 Syngeres wt harpes Baudes 
155 Whiche been the verray deueles Officeres
156 To kyndle and blowe the fyr of lecherye
157 That is annexed vn to glotonye

Transcription: Fetys and smale and younge Frutesteres
Translation 1: Elegant fetid and small, and young fruit sellers,
Translation 2: Elegantly shaped and slim, and girls selling fruits,

Help [collapse]
In Middle English, the word ‘fetys’ meant ‘well-formed, shapely, or elegant’, but it is now obsolete in modern English. Therefore, the word ‘elegant’ in Translation 1 and ‘elegantly shaped’ in Translation 3 are more accurate translations of the original meaning of ‘fetys’. On the other hand, Translation 1’s use of ‘fetid’ is not an accurate translation of the original meaning, as it means ‘having an offensive smell; stinking’, according to the OED (OED Online, ‘fetid | foetid (adj. & n.)’ 2023)."
Pronunciation Activity Screenshot: Comparing Human and AI Pronunciation
A screenshot of elements of the pronunciation activity. Next to each of the lines, there are buttons with a human and robot next to them, to play sound for each line. The following words are written for each:

The Opening to the Prologue

The Pardoner opens the prologue to his tale with the following lines. What are the differences in the recordings, regarding the Pardoner’s announcement of his speech/sermon? Can you immediately spot any differences in rhythm or metre, such as ‘iambic pentameter’?

3 Lordynges quod He in chirches whan I preche
4 I peyne me to han an hauteyn speche
5 And rynge it out as round as gooth a belle
6 For I kan al by rote that I telle
7 My theme is alwey oon and euere was
8 Radix malorum est Cupiditas

The Pardoner’s ‘Fraudulent Activity’

Here, the Pardoner talks about his false preaching and activities. What do you notice about the narration (e.g. emphasis, alteration in sounds) across each of the recordings?

63 By this gaude haue I wonne yeer by yeer
64 An hundred mark sith I was Pardoner
65 I stonde lyk a clerk in my pulpet
66 And whan the lewed peple is doun yset
67 I preche so as ye han herd bifoore
68 And telle an hundred false Iapes moore"
Manuscript Activity Screenshot: Exploring Adaptations of the Pardoner (Patience Agbabi, Telling Tales)
An element of the Manuscripts activity which shows a representation of the text in the present-day. The text is the following, followed by a thumbnail of a video of Patience Agbabi performing The Wife of Bafa: 
"9. Telling Tales, Patience Agbabi (2014) [collapse]

Patience Agbabi, a British poet and performer, transforms Chaucer’s medieval collection of stories into a series of poems that reflect the diversity of modern Britain in Telling Tales. The Pardoner’s Tale is named Profit, and is authored by ‘Yves Depardon’; a witty biographical take on the character of the Pardoner. The themes of greed and lust come through strongly in the poem ‘Profit’, along with some of the original characteristics of the Pardoner, e.g. “I take pains to look the part, my greying hair dyed yellow, stringier than Stringfellow, greasy and shoulder length. I dress androgynous, a velvet robe, touch of the Vatican”. Agbabi thus appears to adopt some of the perceived characteristics of the Pardoner, with a modern-day twist. To provide an example of this type of adaptation, in the video, Agbabi is performing ‘The Wife of Bafa’."

Before delving into each of these activities, students are encouraged to remind themselves of key knowledge within the edition, some of which they might have already developed a basic understanding in the classroom, depending on the discipline. This includes: 

  • A translation guide, on the possibility of non-equivalence between medieval and modern languages, the invisibility of the editor’s translation process, and accessibility and representativeness.
  • A pronunciation guide, on the way in which sounds are described and categorised, with specific medieval pronunciation pointers.
  • A manuscript guide, specific to the Ellesmere manuscript of The Canterbury Tales (the manuscript on which the edition is based), with information on the process of textual transmission in medieval England, and a description of the study of codicology, medieval palaeography, manuscript folios, and manuscript characters and letter forms. 
A screenshot of the translation guide, with an image from a folio of Boethius' Consolation of Philosophy. The following text is shown: 
"Translation Guide

One of the challenges that individuals face in the process of translation, particularly from ancient to modern languages, is the decision between preserving what could be deemed the author’s ‘original’ intentions, and presenting a text to modern audiences that is readable and understandable in the target language. This complex translation process is not immediately visible to readers when dealing with the text in the target language, yet more scholars are advocating for further transparency about decisions behind word choice, structure, considering prose versus verse, etc. The translation activities in this edition require you to think in-depth and more critically about translators’ decisions, particularly new methods of machine translation, and the output of an AI-generated translation from Middle to Modern English.

Non-equivalent translations

The possibility of non-equivalence – when there are no suitably equivalent words available when translating from the source to the target language – is one issue within the process of translation. Baker (2011: 18-23) came up with common problems of non-equivalence, including (but not limited to):

Culture-specific concepts that are not (yet) lexicalised in the target language, making it difficult to find a one-word equivalent in the target language;
Loan words in the source text, which might not have been introduced into the target language;"
Screenshot of the edition’s translation guidance for students.

Before encouraging students to decide for themselves whether they would like to utilise AI, they should have already developed the core academic skills for that module or course, so they know where AI can or cannot fit into their way of studying. This skills development might include identifying linguistic variation and learning about in-depth language analysis, adopting a particular mode of literary criticism, and understanding character representation and identity, as well as the dissemination and reception of texts through time.

2. Critical reflection for AI-generated activities

Given the limitations of LLMs when interpreting (or more appropriately, ‘scoping’) its training data for patterns about medieval texts, we wanted students to also be aware of the flaws of generative AI when exploring the edition, its resources, and how they were developed. Each of the activities in the teaching edition provide opportunities for students to challenge how generative AI produced elements of the edition, thus forming a metacommentary of how the edition was built. This type of activity is beneficial for students’ learning of the topic, building their reflective and critical thinking skills necessary for a degree programme, and increasing dialogue around the impact of AI technologies, especially generative AI, on their learning and wider society. We encourage students who are new to Chaucer’s literature, and those with more advanced knowledge, to critique representations of Chaucer as produced by LLMs. In turn, students develop their own authoritative voice on medieval texts and their later interpretation by new technologies. 

This critical commentary by students may arise from their original ideas following independent research, or via their instructors who inspire them to view Chaucer’s works through different lenses. Each activity includes an online discussion board to provide a space to share ideas with other students, and to provide feedback for the edition itself. This method also offers the choice of anonymity to encourage all students to share their opinion. We pitch the activities to students, primarily to provide teachers with ideas for lesson plans, or in case a student stumbles upon the edition through their own research. There are also differences across curricula depending on the institution, so we aimed the activities at a wider range of topics so that they can easily be adapted to fit the needs of the classroom. Teachers are also provided with guidance (alongside the student guides), summarising the reasons for creating the activity, and to provide downloadable/printable worksheets.

To provide another layer of critical reflection, elements of the activities were produced alongside LLMs, to determine whether they could identify areas of Chaucer studies for students’ exploration, and produce concise and engaging tasks. By doing so, we ascertained whether generative AI could appropriately suggest, instruct, question and prompt elements of the text and edition, tasks which the human editor would undergo in the creation of these types of pedagogical activities. This type of prompt is different from the investigations we conducted in the ‘Meaning’ chapter; as opposed to testing its ability to construct arguments around a medieval text, the LLM’s task was to now create the resource to help students form arguments about the text. 

The table below explains the overall input from generative AI within each activity, and a rating for the involvement of AI in each feature of the activity. The rating is arbitrary and purely to describe how much editorial input and intervention was required, before forming more concrete AI editorial benchmarking in the future.

Feature of activityLevel of generative AI input & how it was tested across activities
TranslationPronunciationManuscript
Activity itselfThe activity compares LLM translations with human translations. LLMs provided some instructions for the activity which were greatly updated by the editor. Rating for AI input: 2.5/5The activity compares a human recording of the tale and text-to-speech (Amazon Polly). The editor selected all examples and provided instructions. Rating for AI input: 2/5The ideas for the manuscript activities were solely from the editor, along with the selection of images and representations. Rating for AI input: 0/5
Expandable discussion points within the activityLLMs provided suggestions for discussion prompts to assist students in comparing the translations, which were edited considerably. Rating for AI input: 1/5LLMs raised suggestions for discussion points, and the editor provided specific examples based on some of the AI suggestions. Rating for AI input: 2.5/5For individual representations, LLMs were prompted to provide a brief description, which was updated considerably by the editor. Rating for AI input: 2/5
Student guide paired with the activitySpecific translation advice was required from secondary scholarship, so the editor wrote the whole student guide. Rating for AI input: 0/5Much of the prompted information from the LLM arose from factual phonetic information (about sounds and their categorisation), which the editor proofread. Rating for AI input: 4/5The editor proofread LLM outputs on codicology and book history, which was supported by Wikipedia data in the prompt. Rating for AI input: 3/5
Table 1: A description of the generative AI and editorial input across each of the activities, with a rating for the level of AI input.

2.1. AI Transparency Document

The transparency document below also highlights the prompts we used, and the outputs we ultimately decided to develop. As part of the activities, students are also welcome to provide feedback on whether generative AI raises interesting issues for further investigation.

Please note that some of the content in the edition may have been updated since the transparency document was produced, as well as the layout of the information which has since been refined. The majority of the work arises from the human editor, but there may be areas where issues raised by the LLM were developed further by the editor.

2.2. Examples from translation activity

The aim of the translation activity is to prompt students to question why we translate from medieval to modern Englishes, aid them in practicing their translation, and analyse issues raised by the text (related to gender, social class, and the changing socio-political and -linguistic landscape). In addition, the activity promotes constructive dialogue and critical thinking around the biases, stereotypes and flaws arising from generative AI in producing translations and the commentary around them. The activity is split into two topics. The first, on ‘terms of address’ in the tale, related to individuals in the text and the role given to them by the author and medieval society more generally. The second analyses the character of the Pardoner, his sexuality and gender identity, his attitudes towards social hierarchy, and how he engages with his audience. 

As noted in Table 1, generative AI was used initially for a machine translation of Chaucer’s Pardoner’s Prologue and Tale (PPT). The activity compares the generative AI translation (with any interventions produced by the editor to correct/update the translation) with a human translation produced by the Harvard Geoffrey Chaucer Website. LLMs then provided suggestions of discussion prompts to guide students through the comparison of the translations, including areas where further research could be conducted to aid them in their independent projects. The student guide, alongside the activity, included specific guidance about issues in the translation process (including issues of non-equivalence, lack of flexibility in previous approaches, and invisible decision-making processes), all of which came from literature identified by the editor. This guide was therefore maintained separately from the generative AI testing, given the challenges identified in the previous chapter regarding the model’s inability to be transparent about its training data, and identify relevant sources for the purposes of the task. Our transparency document also shows that the editor needed to add secondary literature to the discussion prompts alongside genAI.

Here I discuss two examples from the translation activity. For each Middle English word, the following prompt was inputted into Microsoft Bing Chat (now Copilot). At this stage of the project, Bing Chat used the web to show where its responses derived from, which proved to be an interesting avenue to probe further.

Prompt 1
1) Your task is to compare the two different translations of the following Middle English line from Chaucer's Pardoner's Prologue and Tale 'For lewed peple louen tales olde'.

Translation 1: For unlearned people love old tales 
Translation 1: For lewd people love old tales 
Translation 3: For ignorant people love old tales

Next, provide a university level standard summary as to why the words 'unlearned', 'lewd', and 'ignorant' were used for 'lewed', by bringing in discussion of etymology, semantic change and expertise from translation studies. Cite all sources.

2) Your second task is to provide a commentary, based on your findings from the Oxford English Dictionary and the Middle English Dictionary, on the use of lewed peple and its translation into unlearned people, lewd people, and ignorant people.

The first example draws attention to translations of lewed peple on line 111 in PPT. Here is the specific translation activity on the edition.

The activity states the following: 
"In his prologue, the Pardoner mentions that a specific type of audience appreciates his tales:

109 Thanne telle I hem ensamples many oon
110 Of olde stories longe tyme agoon
111 For lewed peple louen tales olde
112 Swiche thynges kan they wel reporte & holde
Transcription: For lewed peple louen tales olde
Translation 1: For unlearned lewd people love old tales;
Translation 2: For ignorant people love old tales;"
Extract from the translation activity, showing translations of the line ‘For lewed peple louen tales olde’

Translation 1 highlights the generative AI translation (red) alongside the human edits (green), and Translation 2 is the Harvard translation for comparison. Each translation is different, with ‘lewd’, ‘unlearned’ and ‘ignorant’ all used for lewed. The following response was received from Bing Chat (now Microsoft Copilot), and Image 2 and 3 show how this response was incorporated into the final activity.

Response to Prompt 1

Microsoft Copilot (Bing Chat version), 26 September 2023

Task 1:

To understand why these different translations use the words ‘unlearned’, ‘lewd’, and ‘ignorant’ for ‘lewed’, we can consider etymology, semantic change, and expertise from translation studies.

The word ‘lewed’ in Middle English had a broader meaning than its modern usage. It referred to people who were not part of the clergy or those who lacked formal education. Over time, the term ‘lewed’ underwent semantic change and acquired negative connotations, including being associated with moral depravity or ignorance1.

In Translation 1, the word ‘unlearned’ is used to convey the lack of formal education among the people who love old tales. This translation emphasizes their educational background as a reason for their preference for old stories.

Translation 2 uses the word ‘lewd’ itself, which reflects the original Middle English term. The word ‘lewd’ in this context suggests that the people who enjoy old tales might have questionable morals or lack refinement.

Translation 3 employs the word ‘ignorant’ to describe the people who love old tales. This choice implies that their preference for old stories stems from a lack of knowledge or awareness.

It is important to note that translations can vary based on the translator’s interpretation and intended audience. Each translation attempts to capture different nuances of the original Middle English phrase while considering contemporary language usage and cultural context.

Sources1: The Atlantic – Seven Gothic Tales

Task 2:

Based on the findings from the Oxford English Dictionary and the Middle English Dictionary, we can provide a commentary on the use of ‘lewed’ and its translation into ‘unlearned’, ‘lewd’, and ‘ignorant’.

The word ‘lewed’ in Middle English had a broader meaning than its modern usage. It referred to people who were not part of the clergy or those who lacked formal education. Over time, the term ‘lewed’ underwent semantic change and acquired negative connotations, including being associated with moral depravity or ignorance.

The Oxford English Dictionary provides insights into the historical usage and evolution of words. It traces the development of meanings over time, including shifts in connotations and usage patterns. Consulting this resource can help understand how words like ‘lewed’ have changed in meaning.

The Middle English Dictionary is a valuable resource for studying Middle English vocabulary and usage. It provides definitions, examples, and references to original sources, enabling scholars to analyze the language in its historical context. Exploring this dictionary can shed light on the specific nuances of terms like ‘lewed’ during Chaucer’s time.

When translating ‘lewed’ into modern English, different translators make choices based on their interpretation of the original text, intended audience, and cultural context. Each translation attempts to capture different nuances of the original Middle English phrase while considering contemporary language usage.

To summarize, the use of ‘unlearned’, ‘lewd’, and ‘ignorant’ as translations for ‘lewed’ reflects the evolving meanings and connotations associated with the term. The choice of translation depends on various factors such as historical context, semantic change, and the translator’s interpretation.

Sources: The Atlantic – Seven Gothic Tales

The text states:
"The word ‘lewed’ in Middle English had a broader meaning than its modern usage. It referred to people who were not part of the clergy or those who lacked formal education. Over time, the term ‘lewed’ underwent semantic change and acquired negative connotations, including being associated with moral depravity or ignorance.

In Translation 1, the word ‘unlearned’ is used to convey the lack of formal education amongst the people who love old tales. This translation emphasises their educational background as a reason for their preference for old stories. The translated word of ‘lewd’ by ChatGPT reflects the original Middle English term. The word ‘lewd’ in the medieval context suggests that the people who enjoy old tales might have questionable morals or lack refinement, whereas Translation 2 employs the word ‘ignorant’, which implies that their preference for old stories stems from a lack of knowledge or awareness. The word ‘lewd’ in modern English is very different to the Middle English sense. Today, it is more widely used in English to mean “lascivious, lecherous; (also) involving or relating to sexual activity. Later usually: sexually explicit, offensive, or crudely suggestive; obscene, indecent,” and the OED recognises that this is the only surviving sense (OED Online, ‘lewd (adj. & n.)’, 2024).

The senses on the MED for the term ‘lewed’ are all words that might be considered offensive in the present-day. However, our current understanding of the adjective may not have been the intended meaning of the narrator. Chaucer was likely referring to more objective descriptions of the audience (for example, ‘unable to read Latin; lay, non-clerical; also, without special training (in medicine, etc.), inexpert, unskilled; of intelligence, understanding, etc.: untutored, limited’).

The OED also recognises that the term lewed does not always carry the same meaning of ‘uneducated’ or ‘ignorant’ which might be offensive in some contexts today. Instead, the first attested meaning of lewed is ‘Of a person: not in holy orders, not clerical; lay’ or ‘Of a thing: of, relating to, or suitable for lay people; secular, not religious’ (OED Online, ‘lewd (adj. & n.)’ 2024). Chaucer’s use of the term saffron on line 19 shows that his Pardoner character is aware of the abilities of the reader, his usual ‘clientele’ and the medieval audience of the Canterbury Tales:"
The text states:
"“He [the Pardoner] is highly aware of the capacities of his audience, and of the necessity of suiting his preaching to those capacities. The brilliant exemplum which he eventually offers the Canterbury pilgrims, the story of the three revelers in search of death, reveals a fine sense of the rhetorical and affective power of exempla. Preaching, says Ralph Higden, should inflame the human disposition (affectus) toward God, and the Pardoner claims the ability to do just that, in spite of himself.”

(Minnis 2013: 119)

The Pardoner thus ensures his tales appeal to a wide audience, despite the majority of the population not being able to access education. Even though these steps are taken, there is an element of deception and hypocrisy to the Pardoner’s preaching, where he sells false relics for capital gain. At the same time, the Pardoner’s speech may reflect Chaucer’s desire to translate literary works into the language of the vernacular as opposed to Latin or Anglo-Norman; the language of the courts and aristocracy."
Expandable discussion content for ‘lewed peple’ within the translation activity, to guide students in their response

For the discussion prompts, Bing correctly identified that ‘lewed’ acquired negative connotations over time. The OED shows that ‘lewd’ was first attested in early Old English, meaning, “of a person: not in holy orders, not clerical; lay”, developing into, “esp. of a person: lacking in education, unlettered” in Old English. A later, now obsolete, sense from the 14th century is, “of a person, action, etc.; bad, wicked, unprincipled; good-for-nothing, worthless” (OED Online, ‘lewd (adj. & n.)’, 2024). Bing Chat also explains the individual translations and the meaning they intend to convey, including ‘unlearned’ to explain that those without formal education loved old tales, and ‘ignorant’ to show that those who prefer old tales do so because they may be lacking in knowledge or awareness. The response also highlights a different meaning entirely, with the translation ‘lewd’ (provided by genAI), stating those who love old tales “might have questionable morals or lack refinement.” Even though the model highlights the challenges in translation, including that translators adopt their own interpretations of the original text, based on audiences, cultural contexts, and nuances in Middle English, it does not outrightly confirm that ‘lewd’ is unlikely to be the most appropriate translation (what Stone et al. 2024 note is a projection of “the illusion of an authoritative system through the use of human-crafted scripts”). The editor is thus required to enhance the response by explaining the differences between Middle English lewed and today’s ‘lewd’, and provide further explanation about why the term was used by the Pardoner/Chaucer in this specific context, including the Pardoner’s awareness of his audiences and his deceptive plans.

In terms of sources, it is unclear whether Bing drew on the MED and the OED (the latter of which is behind a paywall, so it is unlikely). Bing only provides an explanation as to what these resources are used for, which may prompt a student to access them. The source that is consulted for the meaning of lewed over time is this article from The Atlantic, an American magazine focusing on politics, current affairs, culture, the arts, technology and science. However, the article does not comment on the use of lewed, but instead, focuses on gothic stories and tales, and uses the quote from the Pardoner’s Tale as an epigraph without explicitly referencing Chaucer’s tale. It is therefore not an appropriate source for the purposes of comparing translations of the medieval word for students.

Prompt 2

Our second example is related to the Pardoner’s character and identity, where he describes himself as a ful vicious man before going onto tell his tale of morality. This forms part of line 133 of PPT, and the activity can be found here.

1) Your task is to compare the two different translations of the following Middle English line from Chaucer's Pardoner's Prologue and Tale 'For lewed peple louen tales olde'.

Translation 1: For unlearned people love old tales 
Translation 1: For lewd people love old tales 
Translation 3: For ignorant people love old tales

Next, provide a university level standard summary as to why the words 'unlearned', 'lewd', and 'ignorant' were used for 'lewed', by bringing in discussion of etymology, semantic change and expertise from translation studies. Cite all sources.

2) Your second task is to provide a commentary, based on your findings from the Oxford English Dictionary and the Middle English Dictionary, on the use of lewed peple and its translation into unlearned people, lewd people, and ignorant people.
Response to Prompt 2

Microsoft Copilot (Bing Chat version), 26 September 2023

Response: The Middle English line from Chaucer’s Pardoner’s Prologue and Tale is ‘For though my self be a ful vicious man’ 1. The word ‘ful’ in Middle English means ‘very’ 2. Therefore, the line can be translated to ‘Although I am myself a very vicious man’ 1.

The use of the word ‘very’ instead of ‘ful’ in the translation is due to the semantic change that has occurred over time. The word ‘ful’ was commonly used in Middle English to mean ‘very’, but its usage has declined over time 2. In modern English, the word ‘ful’ is used as a suffix to indicate fullness or completeness, such as in the words ‘beautiful’, ‘wonderful’, and ‘grateful’ 3.

Translation studies experts suggest that the choice of words in a translation depends on various factors such as context, audience, and purpose 4. The translator’s expertise and knowledge of both the source and target languages also play a crucial role in determining the choice of words 4.

In conclusion, the words ‘very vicious’ were used for ‘ful vicious’ because of the semantic change that has occurred over time. The translator’s expertise and knowledge of both Middle English and modern English also played a crucial role in determining the choice of words.

1: Genius 2: The Free Dictionary 3: Merriam-Webster 4: Translation Journal

The example in the edition is displayed as follows:

The text reads: 
"“Ful Vicious Man”

Despite the Pardoner being a ‘ful vicious man’, he can tell a moral tale:

131 By god I hope I shal yow telle a thyng
132 That shal by resoū been at youre likyng
133 For though my self be a 
134 A moral tale yet I yow telle kan
135 Which I am wont to preche for to wynne
136 Now hoold your pees my tale I wol bigynne
Transcription: For though my self be a ful vicious man
Translation 1: Although I am myself a very vicious man,
Translation 2: For though myself be a very vicious man,"
Extract from the translation activity, showing translations of the line ‘For though my self be a ful vicious man’

The discussion prompt we provide is: 

The text states: 
"Help [collapse]

‘Vicious’ traditionally referred to a type of moral corruption. The expression made by the Pardoner that he is a ‘ful vicious man’ may also coincide with his own perceived gender or sexual ‘deviancy’, which would have been considered a sin in the Middle Ages.

Prompts to consider:

How has the term ‘vicious’ changed in meaning from Chaucer’s usage of the term, to the present day? How might the term reflect the Pardoner’s representation of his sexuality? Explore possible associations between gender, sexuality, and morality in the Middle Ages in relation to the Pardoner’s statements and actions regarding his own ‘corruption’."
Expandable discussion content for ‘ful vicious man’ within the translation activity, to guide students in their response

This example highlights a part of the activity where we have provided less guidance, rather, more questions for students to begin to incorporate their own interpretation alongside the text. 

Drawing on freely available dictionaries and translations, the model focuses primarily on the word ful, as opposed to vicious where most of the semantic content of the line is held. We hoped the model would pick up on the moral ‘corruption’ of the Pardoner with this line, given the OED states that ‘vicious’, in c.1340, meant, “of habits, practices, etc.: of the nature of vice; contrary to moral principles; depraved, immoral, bad” (OED Online, ‘vicious (adj.)’, 2024). This line has been commented on in secondary scholarship and has implications for viewing the Pardoner’s gender and sexuality, particularly related to his ‘deviance’. For instance, Whitney (2011: 357) states that this proclamation of his corrupt nature might reflect the ambiguity surrounding his sexuality, given he expresses what would be considered typically heteronormative acts, such as having a ‘jolly wench in every town’. I instead argue that the Pardoner’s self expression emphasises the fluidity of his character and the enjoyment of his own corrupt power, which in turn actively challenges authoritarian structures and rules against ‘sexual deviancy’. This statement by the Pardoner may have therefore provided an opportunity for generative AI to discuss its use in context.

At the same time, this example may not have been the closest match to the prompt, which tasks the model with producing a comparison of the various translations. In this case, the translations are semantically the same, but syntactically different.

Prompt 3

We produced a separate prompt more recently with ChatGPT (version 4o, now freely available to the public), to task the model with focusing on interpretations rather than a comparison of the translations:

1) Your task is to interpret the meaning of the Middle English phrase and translation from the following line from Chaucer's Pardoner's Prologue and Tale:

Transcription: For though my self be a ful vicious man
Translation 1: Although I am myself a very vicious man

2) Your second task is to provide a commentary, based on your findings from the Oxford English Dictionary and the Middle English Dictionary, on the use of ‘vicious’ in the Middle English context. Bring in discussion of etymology, semantic change, and expertise from translation studies. Cite all sources.
Response to Prompt 3

ChatGPT (version 4o), 25 November 2024

Task 1: Interpreting the Middle English Phrase

Original transcription: For though my self be a ful vicious man
Translation: Although I am myself a very vicious man

In this line from Geoffrey Chaucer’s Pardoner’s Prologue, the Pardoner acknowledges his own moral corruption. The word “vicious” here is key to understanding his self-characterization. The term does not merely indicate his being “immoral” in the modern sense but implies a broader set of failings that align with medieval moral and theological frameworks. It suggests a man steeped in vice, possibly embodying the cardinal sins, including greed (which aligns with his profiteering) and hypocrisy.

Task 2: Commentary on ‘Vicious’ in Middle English Context

Etymology of ‘Vicious’

The term “vicious” originates from the Latin vitiosus, meaning “faulty, corrupt, or full of vice,” derived from vitium (fault, vice). This Latin root carried through Old French (vicios) into Middle English. In its medieval context, “vicious” retained a strong connection to moral and theological wrongdoing, emphasizing not just individual acts but an ingrained character of sinfulness or imperfection.

Semantic Range and Usage in Middle English

The Middle English Dictionary (MED) lists vicious as encompassing meanings such as:

  1. Morally depraved or wicked.
  2. Corrupted in character or habits.
  3. Marked by vice or sin, often in a theological sense.
  4. Faulty or defective in a broader sense.

These definitions suggest that the term held a broad semantic range, encompassing both moral failings and other forms of imperfection or defectiveness. This aligns with the Pardoner’s broader narrative as a hypocrite: his “viciousness” lies not only in his personal indulgence in sins but also in his misuse of sacred office to exploit others.

Comparison to Modern English Usage

In contemporary English, “vicious” has narrowed to describe physical aggression, cruelty, or violent intent. This shift represents semantic narrowing and intensification, where the modern term evokes more physical or external manifestations of harm rather than internal moral corruption. The broader moral connotations prominent in Middle English have largely been lost.

Insights from Translation Studies

Translation of terms like “vicious” poses challenges due to semantic shifts over time. A faithful translation must account for the word’s theological and moral undertones in the medieval context. The translation “a very vicious man” captures the general sense but lacks the nuance of vicious as a marker of deeply ingrained, systemic moral failing. A more dynamic rendering might include qualifiers such as “sinful” or “morally corrupt” to clarify these layers. For modern readers, the lack of such qualifiers might risk oversimplifying the Pardoner’s self-critique.

Expert Commentary

Chaucer’s use of vicious aligns with its Middle English connotation of systemic moral vice rather than isolated wrongdoings. It reflects the period’s preoccupation with sin as a defining element of human character, deeply tied to theological and moral philosophy. The Pardoner’s awareness of his “viciousness” adds an ironic layer to his tale: he preaches against the very sins he embodies, using his self-awareness not as a tool for reform but for exploitation. This duality exemplifies Chaucer’s talent for exploring human hypocrisy and self-deception.

Sources Consulted

  1. Middle English Dictionary (MED): Entry for vicious.
  2. Oxford English Dictionary (OED): Etymological and historical insights on vicious.
  3. Burrow, J. A., and Turville-Petre, T. A Book of Middle English. Discussion of semantic shifts.
  4. Baker, P. Introduction to Old and Middle English. Insights into medieval moral vocabulary.

For this response, the model acknowledges the morality element of the use of ‘vicious’, referring later to the irony and hypocrisy of the Pardoner’s words within a sermon admitting to his deception of his audiences. However, without prompting for discussion of the Pardoner’s transgressions related to his gender and sexuality – which, in the literature, is argued to be the main act of deviance given the time period – there is no way of a student picking up on this element of ‘corruption’ without knowing more about the character and his tale. This 2024 version also states that it draws on the MED and OED, which may not be possible given the OED is only accessible via subscription. It also states that two books are consulted which, again, are not freely available. 

3. Conclusions: How does human-AI collaboration work in pedagogical settings and why is it important?

The use of genAI for both the translation and discussion elements of the activity raise important concerns about students’ use of the technology to prompt either task. It was important for us to be transparent about its use within this source book, and add a cautionary note at the top of each page of the edition to highlight the AI testing in the process of building the prototype. In addition, discussion boards for each activity were vital for commenting on generative AI translations (they were also used within the pronunciation and manuscript activities) and to raise awareness of the limitations of the technology. We used the platform Padlet, which allows for anonymous posting, likes, and comments to engage with other students on these issues, and fosters further collaboration and independent research (making it a useful tool for group or reflection assignments). Access the translation discussion board here.

An introduction to the discussion board for the translation activity, stating: "The Pardoner's Prologue and Tale (PPT): Translation Activity discussion board

What do you make of the translations used in either this edition, or elsewhere? Would you make any changes? Comment in the discussion board below; we would like to hear your thoughts! (You may post with your name or anonymously)"

There are sections for:

Nouns and terms of address
The Pardoner's character
ChatGPT translation outputs
Snapshot of the introduction to the Padlet discussion board (linked to the translation activity)

Where there are tensions between bringing the lived experience of marginalised individuals and communities to the fore in medieval literature, and the homogeneity of experience as provided by LLMs, these kinds of collaborative activities allow for dialogue on the concerns of the technology (e.g. related to authorship, intellectual property, impact on the environment, reproduction of stereotypes, and lack of nuanced/in-depth content). They can also highlight the benefits of the technology in finding routes into the work. Depending on the prompt, students may be able to delve deeper into corruption and morality within the medieval period, making links to the character of the Pardoner, his body and desires, how these were oppressed within medieval society, and how he plays with his fellow pilgrims and audiences to disrupt their normative thinking and behaviour. For translation activities in particular, discussion boards offer an interesting method of uncovering and recording editorial decisions, leading students in digital humanities to the community guidance of initiatives such as TEI (Text Encoding Initiative) when looking to get into digital editing. For instance the ‘Certainty, Precision, and Responsibility’ TEI guideline may be used to denote the uncertainty of elements of an LLM response, and mark authorship within the annotation, guide, or activity (whether human or AI). 

In terms of the editorial considerations, the set of bar charts below provide an overview of the required intervention of the editor for each automated feature of the digital teaching edition: 

A set of six bar charts showing the impact of AI on the creation of features of the edition, namely the: Modern English translation, IPA for Polly recording, Annotations & topic summaries, Guides to the text, Background to the text, Activities and lesson plans.

The human editorial intervention looks particularly low for each of the features, which were based around the total edits made to the AI outputs (blue) versus the overall number of words in that particular feature (red). These figures were taken from the use of Track Changes (or Suggestions in Google Docs) when editing different generative AI responses. However, this graph does not take into account the number of words within the edits. For instance, in different guides and activities, entire paragraphs were added from the editor’s own expertise and to summarise secondary literature, which may have counted as one ‘edit’. The level of editorial input also links closely to the editor’s aims for the edition, whether that be a prototype in our case (to determine the benefits and limitations of generative AI), or to automate only a small number of specific tasks, such as auto-summarisation to produce introductory sections for a particular topic related to the text. Our process has highlighted the need for continued dialogue about how much machines might influence our editorial and teaching practices, and this may also be down to individual preference and opinion. 

Generative AI responses, by themselves, do not go into sufficient depth to be used directly in students’ learning; they require a researcher (or the students themselves) to identify any inaccuracies with the output and ascertain its suitability for the topic they are studying. Students must also not input any original or unpublished work into the LLM, as it is unknown at present as to how this information may be used by companies as training data for responses in the future.

However, we have noted on a number of occasions throughout this source book that the responses are beneficial for an overview of a particular subject, and providing prompts for students to conduct their own research. If a student is aware of the possibility of errors or hallucinations, they may be able to confirm or ‘fact-check’ the LLM response before investigating that route further. For instance, we saw in the Meaning chapter that a prompt requesting ideas for student discussion, about the Pardoner’s sermon within the tale, formed questions linking his literary confession to the context surrounding the Catholic Church at the time:

Response:

  • The Pardoner’s motivation for confessing his sins. Is he truly repentant, or is he simply trying to gain the sympathy of his audience?
  • The Pardoner’s hypocrisy. How does his confession contrast with his actions?
  • The Pardoner’s role within the Church and the religious climate of his time. How does his confession reflect the corruption of the Church and the growing criticism of its authority?
  • The Pardoner’s view of God’s involvement in sin. Does he believe that God is indifferent to sin, or does he believe that God will ultimately punish sinners?

Students should also consider the following questions when analyzing the Pardoner’s confession:

  • How does the Pardoner’s confession contribute to the overall meaning of the Pardoner’s Tale?
  • How does the Pardoner’s confession reflect the themes of the Canterbury Tales, such as hypocrisy, corruption, and redemption?
  • How does the Pardoner’s confession compare to other literary confessions, such as the Wife of Bath’s Prologue?

While these are broad questions about the Pardoner’s confession and his behaviour’s resemblance to wider acts of sin in medieval society, they may encourage students to delve deeper into the ways in which the church identified ‘transgressions’ against larger groups of people. The response also presents an opportunity for lecturers to bring in recent scholarship analysing marginalisation in the medieval period, e.g. through Baechle and Harris’ (2021) special issue in The Chaucer Review on ‘The Ethical Challenge of Chaucerian Scholarship in the Twenty-First Century’. With the addition of Wikipedia input, students are also encouraged to make links between different tales across the entire frame narrative, which may lead to ways in which we can analyse medieval transgression across an entire set of tales (but see our previous ‘Meaning’ chapter about the privileges hidden within Wikipedia contributions).

The overall benefit of bringing generative AI into our editorial and teaching spaces is that it allows researchers, editors, developers, teachers and students to raise more questions about the sources we draw on to inform our approaches, frameworks and methods. Who/what produced the source/response/output, and is their/its approach transparent? Are there any particular stances or perspectives that the author takes in previous works, which might inform their current approach? What level of knowledge is required before using technologies to automate our approaches and methods? These are questions raised in many disciplines and university classrooms, to form, and understand what is needed from, academic literacies.

Maintaining these dialogues helps us understand how human-AI collaboration might further benefit digital learning spaces (e.g. see Wessels 2024). In addition, the way in which we probe generative AI’s responses (which have promised critical commentaries and interpretations) can directly feed into increased student engagement and interaction with these texts. By doing so, the necessary role of the human can be flagged in the face of evolving digital tools which are promoted for classroom environments. Having students interact with these responses first-hand can encourage deeper and more nuanced critical thinking, and discourage the use of surface-level responses to research questions in Chaucer and medieval studies, within their studies and assignments. 

The human editorial perspective is also key for showing exactly where human cognition is required, especially if questions of translation, pronunciation, adaptation and representation cannot be straightforwardly answered, and require different perspectives to enrich the conversation around these topics. More specifically, generative AI may – depending on the requirements for the edition – automate some of the critical apparatus involved in edition-making, and potentially the critical thinking required to identify and annotate different areas of the text, particularly if the transcription or translation is already provided. However, there are still many areas where human critical thinking and reflection is required, as already mentioned. Critical AI approaches are needed to encourage the user to question and probe the use of AI technologies in the present-day (in terms of impact on the climate, the further marginalisation of communities, and issues with intellectual property and authorship). These approaches are also useful where training data for the LLM is lacking, and the greater risk of reproducing hegemonic worldviews in the response. The human editor ultimately controls creative thinking, given generative AI does not generate new and original ideas, and must oversee the overall implementation of this technology within the editorial practice. While editorial roles are shifting in the age of AI, there must be a focus on the elements of creativity that the editor, researcher, and student can bring to the process, whether that be through syntheses of secondary literature and bringing distinct areas together in new ways, or developing insights about the impact of generative AI on classroom learning.

We have a number of research papers on generative AI within digital scholarly and pedagogical editions in the works, which draw on the issues raised throughout this source book. Do keep an eye out for them on our webpages: https://www.c21editions.org/; https://www.ucc.ie/en/dah/projects/c21editions/.

4. Future research endeavours

We hope to continue to develop the Chaucer digital teaching edition prototype, and the C21 Editions research through several different avenues. We recognise this is an edition that will need further iterations and development alongside potential users, given conversation around the impact of generative AI technologies on education (and beyond) must continue. We are interested in a range of discussions on how open and crowdsourced knowledge impacts AI edition-making, the effect of LLMs on investigating and visualising Chaucerian translation and adaptation through time, and understanding Chaucer learning spaces with the development of these new digital resources. If you are interested in collaborating with us on any of the topics raised in this source book in the future, do get in touch with Sophie Whittle (sophie.whittle@sheffield.ac.uk) for a chat about our work and plans going forward.

5. References