Interaction #3: Identifying Uncertainties in Transcribed Historical Recipes: Two Steps Approach to Co-creation and Research Software Development

How can uncertainties regarding specific historical knowledge be identified, annotated and shared among researchers and citizen scientists? To what extent can an “hybrid” methodology connecting co-creation principles and software development be made for meaningful learning and knowledge sharing processes? These questions guided the first uncertainties workshop between the ProvideDH team and the Gastrosophie project, which among other things focuses on the volunteer-based transcription of digitized corpora of recipes from different periods of times (specially handwritten manuscripts) in Austrian German language.

Introduction and Context

The workshop was preceded by a short presentation from the Gastrosophie project director, Marlene Ernst, with discussion of next steps for the overall project with other researchers and citizen scientists, including their nomination in collaboration with the exploration space team of ACDH-CH for the Austrian Citizen Science Awards. Afterwards, a short introduction on ProvideDH and the project core goal of addressing uncertainties led to a preliminary stage of “self-accreditation” by roles and skills in the context of Digital Humanities research and the workshop topic on recipes and gastrosophy. 

Part A: Annotation “co-ompetition” on paper transcriptions

Once participants formed 4 groups of 3/4 people each, distributed following a diversity criteria from the categories selected on badges, they were given 13 printed recipes per group (on A3 paper size, the same recipes for all of them). The distribution of participants in this sense, considering their selected categories for roles and skills personal badges were:

• Group 1 (3 female): participant 1: “connector, academic, questioner”; participant 2: “crossreader, copy editor, questioner”; participant 3: “transcriber, cooker, nutritionist”
• Group 2 (2 female, 1 male): participant 1: “transcriber, cooker, artist”; participant 2: “academic, questioner, experimenter”; participant 3: “nutrisgtionist, cooker, transcriber”.
• Group 3 (2 female, 1 male): participant 1: “transcriber, cooker, academic”; participant 2: “cooker, nutritionist, transcriber”; participant 3: “cooker, transcriber, experimenter”. 
• Group 4 (3 female): participant 1: “crossreader, cooker, artistic”; participant 2: “transcriber, copy editor, academic”; participant 3: “cooker, copy editor, transcriber”.

Following examples and instructions regarding the transcribed recipes to identify four types of uncertainties as defined in the ProvideDH project (ignorance, credibility, imprecision, incompleteness), based on the work from Theron et al. (2018), participants then added custom coloured stickers to identify them between lines, as well as leaving a handwritten annotation (to count as a “point” for the gamified dynamic).

This led to a total of 58 full annotations (15 on average per team) where 25 referred to “ignorance-related” uncertainties, 14 to “imprecision” uncertainties, 9 to “incompleteness” uncertainties, while only 3 to “credibility” uncertainties. Additionally, participants identified 7 uncertainties who they considered from a different category not reflected in the taxonomy. 

After the two paper annotation rounds by teams (of 10 minutes each) the researchers and citizen scientists discussed the most relevant uncertainties found, as well as examples of the ones which did not match the provided taxonomy (under the label “Other”). In this regard, the discussion centered on the one hand on the lack of examples of credibility uncertainties detected for transcribed recipes, where some could have been related to the real current application of some of them, while others to concepts difficult to consider (like “roasted butter”) or the perceptions and values embedded in some known recipes (for example regarding general considerations of unhygienic or pejorative perceptions from original authors in other cultural contexts). Regarding other types of uncertainties, not reflected by the taxonomy, on the other hand, they mainly referred to dialect-related issues that could be annotated in advance by experts in ancient Austrian German, as well as redundancy in the style of writing for some recipes. In relation to incompleteness-related uncertainties found in the recipes, the discussion also pointed to some recipes being very short in terms of content (even some of them with only the title or a single sentence).

Part B: Annotations on ProvideDH prototype

A second part of the workshop, after concluding the “co-ompetition” mode to make participants engaged in the process, started once each team was more familiar with the annotations on the ProvideDH tool taxonomy. At this stage participants were given precise instructions on how to access the beta version of our prototype for research annotations, accessing specific project dashboard with digitised recipe transcriptions in specific folders. After accessing specific recipes from their respective dashboards, teams had to highlight specific words on recipes, and afterwards activate specific categories and properties to each of them. 

The new areas to information regarding this second annotation process were related to the category and level of uncertainty, its asserted value and description field for sharing comments, which represented no significant levels of misunderstanding or usability problems. 

Allowing for participant observation from the ACDH-CH exploration space facilitators, also considering factors of timing, motivation and level of digital expertise of participants, the session finished with the aim of a following one, where both parts of this novel co-creation approach for citizen science could be combined for wider annotations saved and shared on the ProvideDH prototype tool.