Learning

The value of machine-assisted teaching editions: Creating activities to engage students in contemporary Chaucer studies

1. Introduction

In the previous chapter we discussed whether generative AI can act as a co-pilot for learning more about Chaucer’s literature. We sifted through several examples which highlighted the challenges of prompting LLMs, in producing outputs of meaning for users wanting to know more about the medieval world, or for editors looking to improve access to and engagement with medieval texts. 

The main problem we identified, which is well-known to those working regularly with generative AI technologies, is that LLMs do not interpret text in a human-like way, and lack awareness about what it has produced regardless of the extensiveness of its medieval ‘training’. Not only do LLMs, expectedly, lack anything resembling human cognition, but the model also does not consistently follow a user’s instructions within the prompt. Chatbots are built to probabilistically scope its training data and identify generalised patterns from a statistical distribution, with little consistency in output, a process which does not uncover nuanced (present or past) human experience (see the origins of generative AI chatbots in Stone, Goodlad and Sammons’ 2024 paper). This makes replicating this type of process impossible for an editor or educator wishing to utilise LLMs for their practice, given randomness and probability are at the core of these technologies. We must therefore focus on consistency and rigour in our investigations and critique of LLMs, raising questions of new technologies that generate content in ways that do not reflect naturally-occurring, human outputs.

The previous chapter showed that there is variability in the accuracy of the response, and this occurred with and without the use of human-generated encyclopaedic knowledge from Wikidata in the prompt. While this process of RAG (retrieval-augmented generation) assisted in the direction of the LLM’s response (e.g. by selecting specific extracts from the Chaucerian tale to be commented on), there was an overall sense that increased editorial workload may not always lead to a higher quality response. If there was success, it raised the question as to whether the use of the technology resulted in a lengthier editorial process. As Conrad and Goodlad (2024) note, knowing how to write optimal prompts to achieve better quality responses, as a student looking to improve their career prospects, is not necessarily more rewarding than learning to challenge the way in which LLMs are built and respond. They state the former endeavour is “the digital equivalent of attempting to teach students to ride a bike without ever taking off the training wheels” (Conrad and Goodlad 2024: ¶17).

The only way in which generative AI can act as a co-pilot for learning about Chaucer is if its outputs are probed thoroughly and with investigative intent, knowing the high likelihood of falsities (e.g. see ‘LLM Outputs are Fictions’ by Hanlon 2024). Generative AI might lead researchers down particular paths by raising more general methods of analysing Chaucer’s works, but the editor must focus their attention on the contemporary discussions within the field and interrogate problematic responses produced by new technologies, all while keeping the target audience – the student, in this case – in mind. Given the number of LLM limitations we have already addressed, related to translating Middle English, listening to and speaking Middle English, and creating meaning through interpretations and representations of Chaucer’s works, the learning activities in this chapter provide a solution in an age of Artificial Narrow Intelligence (ANI). These activities form a more rigorous method for students to challenge the development of these technologies, particularly in producing a faithful, yet engaging and interactive, version of one of Chaucer’s tales.

It is ultimately up to the editor as to whether the endeavour of prompting LLMs might be useful for their editorial practice (and their students using the digital resource) or whether it proves to be a hindrance. To counteract the disparities and tensions possible between long-standing editorial practices, and ‘AI-assisted’ editorial practices, we show in this chapter an example of a student activity involving the probing of LLM’s reproduction of and commentary on Chaucerian literature, which can lead to engaging and thought-provoking discussion on AI within the framework of a digital teaching edition.

1.1 What do students need to probe LLMs?

There are a number of key areas which editors (who, in this context, have a pedagogical focus) should consider before opening up the possibility of generative AI testing in the classroom or for assignments. The Critical AI team at Rutgers University regularly maintain a living document for teaching critical AI literacies, providing helpful definitions and ideas for classroom activities that involve probing these technological developments. Some of these activities involve understanding: the anthropomorphistic view of AI through analysis of fictional AI in popular films (such as Blade Runner, Ex Machina, and Westworld); why citation practices are vital to research from a practical and ethical perspective; the design of technologies and who should be involved in these conversations; the existing harms of generative AI as well as the concerns generated by these harms; and comparisons between synthetic summaries of assigned readings versus the actual reading (Goodlad and Stoerger 2024). These activities could be adapted depending on the discipline.

For students who have received training to interpret Chaucer’s works, including learning about Middle English and its translations, the historical context of the stories they worked with, the way in which language can be used to describe different concepts and identities has changed over time, or the different literary interpretations of these texts, there are opportunities to critique generative AI outputs while advancing their knowledge of the text. This is a particularly challenging activity to pitch and is dependent on the level of expertise of the students, which we point to in upcoming research specific to generative AI’s interpretation of marginalised stories, through the character of the Pardoner (Whittle, 2024, submitted). We provided some beginner-level activities in our edition which might ease students into discovering more about Chaucer’s works and their reception, along with probing generative AI responses.

What Chaucer-specific knowledge might students need to know before probing LLM responses? We identified three areas as examples of where elements of an AI-assisted edition would need to be critiqued further. Our activities explore LLM’s translations, text-to-speech recordings of Chaucerian Middle English poetry and its pronunciation and prosody, and the way in which the ‘original text’ and new interpretations of Chaucer are represented through time, including a space to consider how generative AI may draw on or adapt Chaucer’s literature in new outputs.

Translation Activity Screenshot: Role of Women in the Pardoner’s Tale

Pronunciation Activity Screenshot: Comparing Human and AI Pronunciation

Manuscript Activity Screenshot: Exploring Adaptations of the Pardoner (Patience Agbabi, Telling Tales)

Before delving into each of these activities, students are encouraged to remind themselves of key knowledge within the edition, some of which they might have already developed a basic understanding in the classroom, depending on the discipline. This includes: 

• A translation guide, on the possibility of non-equivalence between medieval and modern languages, the invisibility of the editor’s translation process, and accessibility and representativeness.
• A pronunciation guide, on the way in which sounds are described and categorised, with specific medieval pronunciation pointers.
• A manuscript guide, specific to the Ellesmere manuscript of The Canterbury Tales (the manuscript on which the edition is based), with information on the process of textual transmission in medieval England, and a description of the study of codicology, medieval palaeography, manuscript folios, and manuscript characters and letter forms. 

Before encouraging students to decide for themselves whether they would like to utilise AI, they should have already developed the core academic skills for that module or course, so they know where AI can or cannot fit into their way of studying. This skills development might include identifying linguistic variation and learning about in-depth language analysis, adopting a particular mode of literary criticism, and understanding character representation and identity, as well as the dissemination and reception of texts through time.

2. Critical reflection for AI-generated activities

Given the limitations of LLMs when interpreting (or more appropriately, ‘scoping’) its training data for patterns about medieval texts, we wanted students to also be aware of the flaws of generative AI when exploring the edition, its resources, and how they were developed. Each of the activities in the teaching edition provide opportunities for students to challenge how generative AI produced elements of the edition, thus forming a metacommentary of how the edition was built. This type of activity is beneficial for students’ learning of the topic, building their reflective and critical thinking skills necessary for a degree programme, and increasing dialogue around the impact of AI technologies, especially generative AI, on their learning and wider society. We encourage students who are new to Chaucer’s literature, and those with more advanced knowledge, to critique representations of Chaucer as produced by LLMs. In turn, students develop their own authoritative voice on medieval texts and their later interpretation by new technologies. 

This critical commentary by students may arise from their original ideas following independent research, or via their instructors who inspire them to view Chaucer’s works through different lenses. Each activity includes an online discussion board to provide a space to share ideas with other students, and to provide feedback for the edition itself. This method also offers the choice of anonymity to encourage all students to share their opinion. We pitch the activities to students, primarily to provide teachers with ideas for lesson plans, or in case a student stumbles upon the edition through their own research. There are also differences across curricula depending on the institution, so we aimed the activities at a wider range of topics so that they can easily be adapted to fit the needs of the classroom. Teachers are also provided with guidance (alongside the student guides), summarising the reasons for creating the activity, and to provide downloadable/printable worksheets.

To provide another layer of critical reflection, elements of the activities were produced alongside LLMs, to determine whether they could identify areas of Chaucer studies for students’ exploration, and produce concise and engaging tasks. By doing so, we ascertained whether generative AI could appropriately suggest, instruct, question and prompt elements of the text and edition, tasks which the human editor would undergo in the creation of these types of pedagogical activities. This type of prompt is different from the investigations we conducted in the ‘Meaning’ chapter; as opposed to testing its ability to construct arguments around a medieval text, the LLM’s task was to now create the resource to help students form arguments about the text. 

The table below explains the overall input from generative AI within each activity, and a rating for the involvement of AI in each feature of the activity. The rating is arbitrary and purely to describe how much editorial input and intervention was required, before forming more concrete AI editorial benchmarking in the future.

Feature of activity	Level of generative AI input & how it was tested across activities
	Translation	Pronunciation	Manuscript
Activity itself	The activity compares LLM translations with human translations. LLMs provided some instructions for the activity which were greatly updated by the editor. Rating for AI input: 2.5/5	The activity compares a human recording of the tale and text-to-speech (Amazon Polly). The editor selected all examples and provided instructions. Rating for AI input: 2/5	The ideas for the manuscript activities were solely from the editor, along with the selection of images and representations. Rating for AI input: 0/5
Expandable discussion points within the activity	LLMs provided suggestions for discussion prompts to assist students in comparing the translations, which were edited considerably. Rating for AI input: 1/5	LLMs raised suggestions for discussion points, and the editor provided specific examples based on some of the AI suggestions. Rating for AI input: 2.5/5	For individual representations, LLMs were prompted to provide a brief description, which was updated considerably by the editor. Rating for AI input: 2/5
Student guide paired with the activity	Specific translation advice was required from secondary scholarship, so the editor wrote the whole student guide. Rating for AI input: 0/5	Much of the prompted information from the LLM arose from factual phonetic information (about sounds and their categorisation), which the editor proofread. Rating for AI input: 4/5	The editor proofread LLM outputs on codicology and book history, which was supported by Wikipedia data in the prompt. Rating for AI input: 3/5

2.1. AI Transparency Document

The transparency document below also highlights the prompts we used, and the outputs we ultimately decided to develop. As part of the activities, students are also welcome to provide feedback on whether generative AI raises interesting issues for further investigation.

Please note that some of the content in the edition may have been updated since the transparency document was produced, as well as the layout of the information which has since been refined. The majority of the work arises from the human editor, but there may be areas where issues raised by the LLM were developed further by the editor.

2.2. Examples from translation activity

The aim of the translation activity is to prompt students to question why we translate from medieval to modern Englishes, aid them in practicing their translation, and analyse issues raised by the text (related to gender, social class, and the changing socio-political and -linguistic landscape). In addition, the activity promotes constructive dialogue and critical thinking around the biases, stereotypes and flaws arising from generative AI in producing translations and the commentary around them. The activity is split into two topics. The first, on ‘terms of address’ in the tale, related to individuals in the text and the role given to them by the author and medieval society more generally. The second analyses the character of the Pardoner, his sexuality and gender identity, his attitudes towards social hierarchy, and how he engages with his audience. 

As noted in Table 1, generative AI was used initially for a machine translation of Chaucer’s Pardoner’s Prologue and Tale (PPT). The activity compares the generative AI translation (with any interventions produced by the editor to correct/update the translation) with a human translation produced by the Harvard Geoffrey Chaucer Website. LLMs then provided suggestions of discussion prompts to guide students through the comparison of the translations, including areas where further research could be conducted to aid them in their independent projects. The student guide, alongside the activity, included specific guidance about issues in the translation process (including issues of non-equivalence, lack of flexibility in previous approaches, and invisible decision-making processes), all of which came from literature identified by the editor. This guide was therefore maintained separately from the generative AI testing, given the challenges identified in the previous chapter regarding the model’s inability to be transparent about its training data, and identify relevant sources for the purposes of the task. Our transparency document also shows that the editor needed to add secondary literature to the discussion prompts alongside genAI.

Here I discuss two examples from the translation activity. For each Middle English word, the following prompt was inputted into Microsoft Bing Chat (now Copilot). At this stage of the project, Bing Chat used the web to show where its responses derived from, which proved to be an interesting avenue to probe further.

Prompt 1

1) Your task is to compare the two different translations of the following Middle English line from Chaucer's Pardoner's Prologue and Tale 'For lewed peple louen tales olde'.

Translation 1: For unlearned people love old tales 
Translation 1: For lewd people love old tales 
Translation 3: For ignorant people love old tales

Next, provide a university level standard summary as to why the words 'unlearned', 'lewd', and 'ignorant' were used for 'lewed', by bringing in discussion of etymology, semantic change and expertise from translation studies. Cite all sources.

2) Your second task is to provide a commentary, based on your findings from the Oxford English Dictionary and the Middle English Dictionary, on the use of lewed peple and its translation into unlearned people, lewd people, and ignorant people.

The first example draws attention to translations of lewed peple on line 111 in PPT. Here is the specific translation activity on the edition.

Translation 1 highlights the generative AI translation (red) alongside the human edits (green), and Translation 2 is the Harvard translation for comparison. Each translation is different, with ‘lewd’, ‘unlearned’ and ‘ignorant’ all used for lewed. The following response was received from Bing Chat (now Microsoft Copilot), and Image 2 and 3 show how this response was incorporated into the final activity.

Response to Prompt 1

Microsoft Copilot (Bing Chat version), 26 September 2023

Task 1:

To understand why these different translations use the words ‘unlearned’, ‘lewd’, and ‘ignorant’ for ‘lewed’, we can consider etymology, semantic change, and expertise from translation studies.

The word ‘lewed’ in Middle English had a broader meaning than its modern usage. It referred to people who were not part of the clergy or those who lacked formal education. Over time, the term ‘lewed’ underwent semantic change and acquired negative connotations, including being associated with moral depravity or ignorance¹.

In Translation 1, the word ‘unlearned’ is used to convey the lack of formal education among the people who love old tales. This translation emphasizes their educational background as a reason for their preference for old stories.

Translation 2 uses the word ‘lewd’ itself, which reflects the original Middle English term. The word ‘lewd’ in this context suggests that the people who enjoy old tales might have questionable morals or lack refinement.

Translation 3 employs the word ‘ignorant’ to describe the people who love old tales. This choice implies that their preference for old stories stems from a lack of knowledge or awareness.

It is important to note that translations can vary based on the translator’s interpretation and intended audience. Each translation attempts to capture different nuances of the original Middle English phrase while considering contemporary language usage and cultural context.

Sources¹: The Atlantic – Seven Gothic Tales

Task 2:

Based on the findings from the Oxford English Dictionary and the Middle English Dictionary, we can provide a commentary on the use of ‘lewed’ and its translation into ‘unlearned’, ‘lewd’, and ‘ignorant’.

The word ‘lewed’ in Middle English had a broader meaning than its modern usage. It referred to people who were not part of the clergy or those who lacked formal education. Over time, the term ‘lewed’ underwent semantic change and acquired negative connotations, including being associated with moral depravity or ignorance.

The Oxford English Dictionary provides insights into the historical usage and evolution of words. It traces the development of meanings over time, including shifts in connotations and usage patterns. Consulting this resource can help understand how words like ‘lewed’ have changed in meaning.

The Middle English Dictionary is a valuable resource for studying Middle English vocabulary and usage. It provides definitions, examples, and references to original sources, enabling scholars to analyze the language in its historical context. Exploring this dictionary can shed light on the specific nuances of terms like ‘lewed’ during Chaucer’s time.

When translating ‘lewed’ into modern English, different translators make choices based on their interpretation of the original text, intended audience, and cultural context. Each translation attempts to capture different nuances of the original Middle English phrase while considering contemporary language usage.

To summarize, the use of ‘unlearned’, ‘lewd’, and ‘ignorant’ as translations for ‘lewed’ reflects the evolving meanings and connotations associated with the term. The choice of translation depends on various factors such as historical context, semantic change, and the translator’s interpretation.

Sources: The Atlantic – Seven Gothic Tales

For the discussion prompts, Bing correctly identified that ‘lewed’ acquired negative connotations over time. The OED shows that ‘lewd’ was first attested in early Old English, meaning, “of a person: not in holy orders, not clerical; lay”, developing into, “esp. of a person: lacking in education, unlettered” in Old English. A later, now obsolete, sense from the 14th century is, “of a person, action, etc.; bad, wicked, unprincipled; good-for-nothing, worthless” (OED Online, ‘lewd (adj. & n.)’, 2024). Bing Chat also explains the individual translations and the meaning they intend to convey, including ‘unlearned’ to explain that those without formal education loved old tales, and ‘ignorant’ to show that those who prefer old tales do so because they may be lacking in knowledge or awareness. The response also highlights a different meaning entirely, with the translation ‘lewd’ (provided by genAI), stating those who love old tales “might have questionable morals or lack refinement.” Even though the model highlights the challenges in translation, including that translators adopt their own interpretations of the original text, based on audiences, cultural contexts, and nuances in Middle English, it does not outrightly confirm that ‘lewd’ is unlikely to be the most appropriate translation (what Stone et al. 2024 note is a projection of “the illusion of an authoritative system through the use of human-crafted scripts”). The editor is thus required to enhance the response by explaining the differences between Middle English lewed and today’s ‘lewd’, and provide further explanation about why the term was used by the Pardoner/Chaucer in this specific context, including the Pardoner’s awareness of his audiences and his deceptive plans.

In terms of sources, it is unclear whether Bing drew on the MED and the OED (the latter of which is behind a paywall, so it is unlikely). Bing only provides an explanation as to what these resources are used for, which may prompt a student to access them. The source that is consulted for the meaning of lewed over time is this article from The Atlantic, an American magazine focusing on politics, current affairs, culture, the arts, technology and science. However, the article does not comment on the use of lewed, but instead, focuses on gothic stories and tales, and uses the quote from the Pardoner’s Tale as an epigraph without explicitly referencing Chaucer’s tale. It is therefore not an appropriate source for the purposes of comparing translations of the medieval word for students.

Prompt 2

Our second example is related to the Pardoner’s character and identity, where he describes himself as a ful vicious man before going onto tell his tale of morality. This forms part of line 133 of PPT, and the activity can be found here.

1) Your task is to compare the two different translations of the following Middle English line from Chaucer's Pardoner's Prologue and Tale 'For lewed peple louen tales olde'.

Translation 1: For unlearned people love old tales 
Translation 1: For lewd people love old tales 
Translation 3: For ignorant people love old tales

Next, provide a university level standard summary as to why the words 'unlearned', 'lewd', and 'ignorant' were used for 'lewed', by bringing in discussion of etymology, semantic change and expertise from translation studies. Cite all sources.

2) Your second task is to provide a commentary, based on your findings from the Oxford English Dictionary and the Middle English Dictionary, on the use of lewed peple and its translation into unlearned people, lewd people, and ignorant people.

Response to Prompt 2

Microsoft Copilot (Bing Chat version), 26 September 2023

Response: The Middle English line from Chaucer’s Pardoner’s Prologue and Tale is ‘For though my self be a ful vicious man’ ¹. The word ‘ful’ in Middle English means ‘very’ ². Therefore, the line can be translated to ‘Although I am myself a very vicious man’ ¹.

The use of the word ‘very’ instead of ‘ful’ in the translation is due to the semantic change that has occurred over time. The word ‘ful’ was commonly used in Middle English to mean ‘very’, but its usage has declined over time ². In modern English, the word ‘ful’ is used as a suffix to indicate fullness or completeness, such as in the words ‘beautiful’, ‘wonderful’, and ‘grateful’ ³.

Translation studies experts suggest that the choice of words in a translation depends on various factors such as context, audience, and purpose ⁴. The translator’s expertise and knowledge of both the source and target languages also play a crucial role in determining the choice of words ⁴.

In conclusion, the words ‘very vicious’ were used for ‘ful vicious’ because of the semantic change that has occurred over time. The translator’s expertise and knowledge of both Middle English and modern English also played a crucial role in determining the choice of words.

¹: Genius ²: The Free Dictionary ³: Merriam-Webster ⁴: Translation Journal

The example in the edition is displayed as follows:

The discussion prompt we provide is: 

This example highlights a part of the activity where we have provided less guidance, rather, more questions for students to begin to incorporate their own interpretation alongside the text. 

Drawing on freely available dictionaries and translations, the model focuses primarily on the word ful, as opposed to vicious where most of the semantic content of the line is held. We hoped the model would pick up on the moral ‘corruption’ of the Pardoner with this line, given the OED states that ‘vicious’, in c.1340, meant, “of habits, practices, etc.: of the nature of vice; contrary to moral principles; depraved, immoral, bad” (OED Online, ‘vicious (adj.)’, 2024). This line has been commented on in secondary scholarship and has implications for viewing the Pardoner’s gender and sexuality, particularly related to his ‘deviance’. For instance, Whitney (2011: 357) states that this proclamation of his corrupt nature might reflect the ambiguity surrounding his sexuality, given he expresses what would be considered typically heteronormative acts, such as having a ‘jolly wench in every town’. I instead argue that the Pardoner’s self expression emphasises the fluidity of his character and the enjoyment of his own corrupt power, which in turn actively challenges authoritarian structures and rules against ‘sexual deviancy’. This statement by the Pardoner may have therefore provided an opportunity for generative AI to discuss its use in context.

At the same time, this example may not have been the closest match to the prompt, which tasks the model with producing a comparison of the various translations. In this case, the translations are semantically the same, but syntactically different.

Prompt 3

We produced a separate prompt more recently with ChatGPT (version 4o, now freely available to the public), to task the model with focusing on interpretations rather than a comparison of the translations:

1) Your task is to interpret the meaning of the Middle English phrase and translation from the following line from Chaucer's Pardoner's Prologue and Tale:

Transcription: For though my self be a ful vicious man
Translation 1: Although I am myself a very vicious man

2) Your second task is to provide a commentary, based on your findings from the Oxford English Dictionary and the Middle English Dictionary, on the use of ‘vicious’ in the Middle English context. Bring in discussion of etymology, semantic change, and expertise from translation studies. Cite all sources.

Response to Prompt 3

ChatGPT (version 4o), 25 November 2024

Task 1: Interpreting the Middle English Phrase

Original transcription: For though my self be a ful vicious man
Translation: Although I am myself a very vicious man

In this line from Geoffrey Chaucer’s Pardoner’s Prologue, the Pardoner acknowledges his own moral corruption. The word “vicious” here is key to understanding his self-characterization. The term does not merely indicate his being “immoral” in the modern sense but implies a broader set of failings that align with medieval moral and theological frameworks. It suggests a man steeped in vice, possibly embodying the cardinal sins, including greed (which aligns with his profiteering) and hypocrisy.

Task 2: Commentary on ‘Vicious’ in Middle English Context

Etymology of ‘Vicious’

The term “vicious” originates from the Latin vitiosus, meaning “faulty, corrupt, or full of vice,” derived from vitium (fault, vice). This Latin root carried through Old French (vicios) into Middle English. In its medieval context, “vicious” retained a strong connection to moral and theological wrongdoing, emphasizing not just individual acts but an ingrained character of sinfulness or imperfection.

Semantic Range and Usage in Middle English

The Middle English Dictionary (MED) lists vicious as encompassing meanings such as:

1. Morally depraved or wicked.
2. Corrupted in character or habits.
3. Marked by vice or sin, often in a theological sense.
4. Faulty or defective in a broader sense.

These definitions suggest that the term held a broad semantic range, encompassing both moral failings and other forms of imperfection or defectiveness. This aligns with the Pardoner’s broader narrative as a hypocrite: his “viciousness” lies not only in his personal indulgence in sins but also in his misuse of sacred office to exploit others.

Comparison to Modern English Usage

In contemporary English, “vicious” has narrowed to describe physical aggression, cruelty, or violent intent. This shift represents semantic narrowing and intensification, where the modern term evokes more physical or external manifestations of harm rather than internal moral corruption. The broader moral connotations prominent in Middle English have largely been lost.

Insights from Translation Studies

Translation of terms like “vicious” poses challenges due to semantic shifts over time. A faithful translation must account for the word’s theological and moral undertones in the medieval context. The translation “a very vicious man” captures the general sense but lacks the nuance of vicious as a marker of deeply ingrained, systemic moral failing. A more dynamic rendering might include qualifiers such as “sinful” or “morally corrupt” to clarify these layers. For modern readers, the lack of such qualifiers might risk oversimplifying the Pardoner’s self-critique.

Expert Commentary

Chaucer’s use of vicious aligns with its Middle English connotation of systemic moral vice rather than isolated wrongdoings. It reflects the period’s preoccupation with sin as a defining element of human character, deeply tied to theological and moral philosophy. The Pardoner’s awareness of his “viciousness” adds an ironic layer to his tale: he preaches against the very sins he embodies, using his self-awareness not as a tool for reform but for exploitation. This duality exemplifies Chaucer’s talent for exploring human hypocrisy and self-deception.

Sources Consulted

1. Middle English Dictionary (MED): Entry for vicious.
2. Oxford English Dictionary (OED): Etymological and historical insights on vicious.
3. Burrow, J. A., and Turville-Petre, T. A Book of Middle English. Discussion of semantic shifts.
4. Baker, P. Introduction to Old and Middle English. Insights into medieval moral vocabulary.

For this response, the model acknowledges the morality element of the use of ‘vicious’, referring later to the irony and hypocrisy of the Pardoner’s words within a sermon admitting to his deception of his audiences. However, without prompting for discussion of the Pardoner’s transgressions related to his gender and sexuality – which, in the literature, is argued to be the main act of deviance given the time period – there is no way of a student picking up on this element of ‘corruption’ without knowing more about the character and his tale. This 2024 version also states that it draws on the MED and OED, which may not be possible given the OED is only accessible via subscription. It also states that two books are consulted which, again, are not freely available. 

3. Conclusions: How does human-AI collaboration work in pedagogical settings and why is it important?

The use of genAI for both the translation and discussion elements of the activity raise important concerns about students’ use of the technology to prompt either task. It was important for us to be transparent about its use within this source book, and add a cautionary note at the top of each page of the edition to highlight the AI testing in the process of building the prototype. In addition, discussion boards for each activity were vital for commenting on generative AI translations (they were also used within the pronunciation and manuscript activities) and to raise awareness of the limitations of the technology. We used the platform Padlet, which allows for anonymous posting, likes, and comments to engage with other students on these issues, and fosters further collaboration and independent research (making it a useful tool for group or reflection assignments). Access the translation discussion board here.

Where there are tensions between bringing the lived experience of marginalised individuals and communities to the fore in medieval literature, and the homogeneity of experience as provided by LLMs, these kinds of collaborative activities allow for dialogue on the concerns of the technology (e.g. related to authorship, intellectual property, impact on the environment, reproduction of stereotypes, and lack of nuanced/in-depth content). They can also highlight the benefits of the technology in finding routes into the work. Depending on the prompt, students may be able to delve deeper into corruption and morality within the medieval period, making links to the character of the Pardoner, his body and desires, how these were oppressed within medieval society, and how he plays with his fellow pilgrims and audiences to disrupt their normative thinking and behaviour. For translation activities in particular, discussion boards offer an interesting method of uncovering and recording editorial decisions, leading students in digital humanities to the community guidance of initiatives such as TEI (Text Encoding Initiative) when looking to get into digital editing. For instance the ‘Certainty, Precision, and Responsibility’ TEI guideline may be used to denote the uncertainty of elements of an LLM response, and mark authorship within the annotation, guide, or activity (whether human or AI). 

In terms of the editorial considerations, the set of bar charts below provide an overview of the required intervention of the editor for each automated feature of the digital teaching edition: 

The human editorial intervention looks particularly low for each of the features, which were based around the total edits made to the AI outputs (blue) versus the overall number of words in that particular feature (red). These figures were taken from the use of Track Changes (or Suggestions in Google Docs) when editing different generative AI responses. However, this graph does not take into account the number of words within the edits. For instance, in different guides and activities, entire paragraphs were added from the editor’s own expertise and to summarise secondary literature, which may have counted as one ‘edit’. The level of editorial input also links closely to the editor’s aims for the edition, whether that be a prototype in our case (to determine the benefits and limitations of generative AI), or to automate only a small number of specific tasks, such as auto-summarisation to produce introductory sections for a particular topic related to the text. Our process has highlighted the need for continued dialogue about how much machines might influence our editorial and teaching practices, and this may also be down to individual preference and opinion. 

Generative AI responses, by themselves, do not go into sufficient depth to be used directly in students’ learning; they require a researcher (or the students themselves) to identify any inaccuracies with the output and ascertain its suitability for the topic they are studying. Students must also not input any original or unpublished work into the LLM, as it is unknown at present as to how this information may be used by companies as training data for responses in the future.

However, we have noted on a number of occasions throughout this source book that the responses are beneficial for an overview of a particular subject, and providing prompts for students to conduct their own research. If a student is aware of the possibility of errors or hallucinations, they may be able to confirm or ‘fact-check’ the LLM response before investigating that route further. For instance, we saw in the Meaning chapter that a prompt requesting ideas for student discussion, about the Pardoner’s sermon within the tale, formed questions linking his literary confession to the context surrounding the Catholic Church at the time:

Response:

• The Pardoner’s motivation for confessing his sins. Is he truly repentant, or is he simply trying to gain the sympathy of his audience?
• The Pardoner’s hypocrisy. How does his confession contrast with his actions?
• The Pardoner’s role within the Church and the religious climate of his time. How does his confession reflect the corruption of the Church and the growing criticism of its authority?
• The Pardoner’s view of God’s involvement in sin. Does he believe that God is indifferent to sin, or does he believe that God will ultimately punish sinners?

Students should also consider the following questions when analyzing the Pardoner’s confession:

• How does the Pardoner’s confession contribute to the overall meaning of the Pardoner’s Tale?
• How does the Pardoner’s confession reflect the themes of the Canterbury Tales, such as hypocrisy, corruption, and redemption?
• How does the Pardoner’s confession compare to other literary confessions, such as the Wife of Bath’s Prologue?

While these are broad questions about the Pardoner’s confession and his behaviour’s resemblance to wider acts of sin in medieval society, they may encourage students to delve deeper into the ways in which the church identified ‘transgressions’ against larger groups of people. The response also presents an opportunity for lecturers to bring in recent scholarship analysing marginalisation in the medieval period, e.g. through Baechle and Harris’ (2021) special issue in The Chaucer Review on ‘The Ethical Challenge of Chaucerian Scholarship in the Twenty-First Century’. With the addition of Wikipedia input, students are also encouraged to make links between different tales across the entire frame narrative, which may lead to ways in which we can analyse medieval transgression across an entire set of tales (but see our previous ‘Meaning’ chapter about the privileges hidden within Wikipedia contributions).

The overall benefit of bringing generative AI into our editorial and teaching spaces is that it allows researchers, editors, developers, teachers and students to raise more questions about the sources we draw on to inform our approaches, frameworks and methods. Who/what produced the source/response/output, and is their/its approach transparent? Are there any particular stances or perspectives that the author takes in previous works, which might inform their current approach? What level of knowledge is required before using technologies to automate our approaches and methods? These are questions raised in many disciplines and university classrooms, to form, and understand what is needed from, academic literacies.

Maintaining these dialogues helps us understand how human-AI collaboration might further benefit digital learning spaces (e.g. see Wessels 2024). In addition, the way in which we probe generative AI’s responses (which have promised critical commentaries and interpretations) can directly feed into increased student engagement and interaction with these texts. By doing so, the necessary role of the human can be flagged in the face of evolving digital tools which are promoted for classroom environments. Having students interact with these responses first-hand can encourage deeper and more nuanced critical thinking, and discourage the use of surface-level responses to research questions in Chaucer and medieval studies, within their studies and assignments. 

The human editorial perspective is also key for showing exactly where human cognition is required, especially if questions of translation, pronunciation, adaptation and representation cannot be straightforwardly answered, and require different perspectives to enrich the conversation around these topics. More specifically, generative AI may – depending on the requirements for the edition – automate some of the critical apparatus involved in edition-making, and potentially the critical thinking required to identify and annotate different areas of the text, particularly if the transcription or translation is already provided. However, there are still many areas where human critical thinking and reflection is required, as already mentioned. Critical AI approaches are needed to encourage the user to question and probe the use of AI technologies in the present-day (in terms of impact on the climate, the further marginalisation of communities, and issues with intellectual property and authorship). These approaches are also useful where training data for the LLM is lacking, and the greater risk of reproducing hegemonic worldviews in the response. The human editor ultimately controls creative thinking, given generative AI does not generate new and original ideas, and must oversee the overall implementation of this technology within the editorial practice. While editorial roles are shifting in the age of AI, there must be a focus on the elements of creativity that the editor, researcher, and student can bring to the process, whether that be through syntheses of secondary literature and bringing distinct areas together in new ways, or developing insights about the impact of generative AI on classroom learning.

We have a number of research papers on generative AI within digital scholarly and pedagogical editions in the works, which draw on the issues raised throughout this source book. Do keep an eye out for them on our webpages: https://www.c21editions.org/; https://www.ucc.ie/en/dah/projects/c21editions/.

4. Future research endeavours

We hope to continue to develop the Chaucer digital teaching edition prototype, and the C21 Editions research through several different avenues. We recognise this is an edition that will need further iterations and development alongside potential users, given conversation around the impact of generative AI technologies on education (and beyond) must continue. We are interested in a range of discussions on how open and crowdsourced knowledge impacts AI edition-making, the effect of LLMs on investigating and visualising Chaucerian translation and adaptation through time, and understanding Chaucer learning spaces with the development of these new digital resources. If you are interested in collaborating with us on any of the topics raised in this source book in the future, do get in touch with Sophie Whittle (sophie.whittle@sheffield.ac.uk) for a chat about our work and plans going forward.

5. References

• Baechle, Sarah and Carissa M. Harris. 2021. “The Ethical Challenge of Chaucerian Scholarship in the Twenty-First Century.” The Chaucer Review 56 (4): 311–321. https://muse.jhu.edu/article/804395. 
• ​​Conrad, Katie and Lauren M. E. Goodlad. 2024. “Teaching Tips for Navigating AI in the Classroom.” The AI Hype Wall of Shame. https://criticalai.org/2024/03/07/katie-conrad-and-lauren-m-e-goodlad-on-teaching-tip-navigating-ai-in-the-classroom-03-07-2024/ (accessed 22 August 2024).
• Goodlad, Lauren M. E. and Sharon Stoerger. 2024-. “Teaching Critical AI Literacies: An ‘Explainer’ and Resource for the New Semester.” Rutgers AI. https://docs.google.com/document/d/1TAXqYGid8sQz8v1ngTLD1qZBx2rNKHeKn9mcfWbFzRQ/mobilebasic#ftnt1 (accessed 26 November 2024).
• Hanlon, Aaron R. 2024. “LLM Outputs Are Fictions.” Critical AI 2 (1). https://doi.org/10.1215/2834703X-11205210. 
• Oxford English Dictionary, 2024. “lewd (adj. & n.),” September 2024, https://doi.org/10.1093/OED/8869421360. 
• Oxford English Dictionary, 2024. “vicious (adj.),” September 2024, https://doi.org/10.1093/OED/1001819490. 
• Stone, Matthew, Lauren M. E. Goodlad and Mark Sammons. 2024. “The Origins of Generative AI in Transcription and Machine Translation, and Why That Matters.” Critical AI 2 (1). https://doi.org/10.1215/2834703X-11256853. 
• Wessels, Bridgette. 2024. “Developing a Coherent Resource for Chaucer Studies in the Digital Age: The Use of AI, Domain Experts and Digital Scholarly Editions in Higher Education.” In International Symposium on the Future of Digital Editing & Publishing, ed. by O’Sullivan, James, 34-40. University College Cork. https://hdl.handle.net/10468/15969.
• Whitney, Elspeth. 2011. “What’s Wrong with the Pardoner? Complexion Theory, the Phlegmatic Man, and Effeminacy.” The Chaucer Review 45 (4): 357-389. https://doi.org/10.5325/chaucerrev.45.4.0357.  
• Whittle, Sophie. Submitted. “Digital Chaucer Pedagogy and Editing: Probing Generative AI’s Reproduction of Hegemony.” Scholarly Editing.