Two limitations arose in the context of our experimental design. First, we have used an edited list of theory frameworks and representatives, which we then systematically searched in our corpus. We decided to use a manually edited list of theory frameworks, as we encountered various problems when using only high-level concepts (such as structuralism or postcolonialism), because many of the articles instead mention typical representatives of the specific theoretical currents—for example, one tends to find Eichenbaum rather than formalism. Second, our corpus of DH journals is certainly not meant to be representative of all DH scholarly communication. Since it is impossible to survey the entirety of theory frameworks in the humanities, we have chosen literary and cultural theories as a representative subfield.

To investigate this subfield we created dictionaries, which are based on three widely used introductory works to literary theory: Selden et al., Rivkin and Ryan, and Castle. Selection criteria included an assumed degree of familiarity and dissemination of the introductory works (such as the number of editions and citations). Our selection focused less on a diversification of the theoretical canon and thus shows a rather strong gender and diversity imbalance (Risam 17). The dictionaries are structured as follows: each dictionary contains an umbrella term for the theoretical approach, typical representatives (“name and surname” as well as “surname” only), and common multi-word combinations.4

Our dictionaries provide heuristic tools that allow us to address potential discursive intersections of theory. Please note that we do not intend to accurately reproduce individual branches of theory by means of representative authors, but rather to generate a generic inventory of relevant terms that are suitable to represent the typical use of theory in the humanities. Furthermore, we do not claim that these dictionaries are exhaustive or representative of all theory frameworks that might be used in DH. Our compilation of traditional textbooks is rather a first attempt to address the question of what DH scholars probably mean when they talk about theory. We believe that our dictionaries provide first insights into the nature of theory in DH research, because they serve as syntheses of high-level concepts and representatives for literary and cultural theories.

In total, 13 unordered dictionaries were aggregated on the basis of their tables of contents, including names of theoretical approaches and schools as well as some of their representatives. Furthermore, we added hermeneutics, which was not explicitly part of the three mentioned introductory books, but which we frequently observed in our corpus during the first experiments. The final list of 14 theory frameworks, along with one representative given as an exemplar of each, runs as follows:

Finally, ambiguity was addressed in several ways. First, although in Selden et al.’s Reader theoretical constructs or other umbrella terms, such as metaphor or sexual politics, appear alongside names of theories and representatives, we elected not to include these constructs in our dictionaries because of their semantic ambiguity. Second, we also removed some names that were highly ambiguous; for instance, (Walter) Benjamin, whose last name is also a first name that appears in multiple articles. Third, classifications of representatives are not always clear-cut as they may appear in two or more dictionaries at the same time (e.g., Michel Foucault). In such cases, we manually selected what we assumed to be the most representative dictionary. Lastly, we split Rivkin’s “Feminism” and “Gender studies” and Selden et al.’s “Feminst theories” and “Gay, lesbian, queer theories” categories into two dictionaries called “Gender and LGBTQ+ Studies” and “Feminist Theory.”5

Searching for the items from our dictionaries, we found that at least one of the specific cultural and literary theory terms appears in 793 articles of a total of 3,737 articles in the corpus. We found additional verbatim occurrences for theory and theories (1,037) that were not part of our dictionaries (see Figure 2). In order to get a rough overview of the share of further theory frameworks, we searched for all instances of “theory + of,” “noun + theory/ies,” and “adjective + theory/ies,” which, for the most part, brings to the fore further theory frameworks that go far beyond the scope of our dictionaries for cultural and literary theory. Not only did we find numerous other humanistic theories (e.g., “theory of meaning,” “theory of textuality,” “theory of genres,” “theory of metaphor,” “theory of lexical diffusion”), we also found theories from other domains and disciplines (e.g., “information theory,” “graph theory,” “evolution theory,” “game theory,” “chaos theory”). In follow-up studies, we will systematically extract other theory frameworks and produce further dictionaries and augment them with representative theoreticians from Wikipedia and Wikidata (Gutiérrez de la Torre et al.).

These simple figures are by themselves already telling, as it shows that almost half the articles (48.97%) mention theory in one way or another, thereby already challenging the popular narrative that DH may lack theory (Cecire). That said, it is obvious that the mere mention of theory does not automatically entail an actual application or development of a theory. This is why we wanted to take a closer look at the use of specific theory frameworks from traditional humanities disciplines.

After some corpus-wide frequency analysis, we took a closer look at the frequency of theory references in single documents. More concretely, we were interested in how often a dictionary item is used within one article. As a measure, we adopted the type-token-ratio (TTR), which is popular in quantitative linguistics to analyze the complexity of language by means of its vocabulary performance (Hess et al.). TTR distinguishes types, which are the number of unique words in a text, and tokens, which are the actual realizations of one type in a text. The TTR is calculated by dividing the number of types by the number of tokens. We obtained results in the range of 0 to 1, where values toward 1 can be interpreted as having high lexical variety. An actual score of 1 would mean that every type of a text is realized by exactly one token; that is, every word used in the text is unique.

For our case of investigating the frequency of theory references within different documents, we calculated TTR exclusively for the theory items of our dictionary, not for the whole document texts. A TTR of exactly 1 here would mean that each theory type is realized by exactly one token, which could be interpreted as a rather shallow reference to the theory framework, as one would assume that an article that heavily relies on theoretical references to, for instance “Roland Barthes,” would mention him more than just once in the paper. Interestingly, this happens to be the case for 34.8% of articles that have at least one specific theory reference. This shows that most of the documents with a TTR=1 reference exactly one specific theory item, one single time (222). In a few cases, we found two (40), three (11), or four (2) theory references being mentioned exactly one time each. As only a comparatively small number of articles (222) picks up one specific theory reference from our dictionaries only once within the whole article, this might lead to the conclusion that most DH articles do indeed address the topic of theory in more than just a cursory way. This assumption is also reflected by the document frequencies (type count) and total occurrences (token count) of the most frequent theory items in our corpus (full list available online),6 which indicate that these items are heavily referenced in many different articles and also with a rather high density within individual articles.

As the ranking of the top-ten document frequencies (df) by type count (see Table 2) shows, hermeneutic references (df=126) are particularly dominant, which is probably due to the wide semantic range of hermeneutics. According to Joris van Zundert, hermeneutics “turned from a theory of the interpretation of text into an ontological theory of understanding. It can now be understood broadly as the theory of the processes that turn information into knowledge” (333). As humanities are implicitly indicated as being hermeneutic, DH is also often located within a hermeneutical tradition. Moreover, DH is certainly deeply rooted in textual scholarship and philology. This could also explain the dominance of linguistic theory (df=87).

It is also noteworthy that close reading is one of the core methods of “new criticism” (Ransom), which was “an early to mid-twentieth-century literary movement that subordinated the historical […] concerns of previous scholarship to the text itself” (Bode 92). Therefore, the high document frequency of new critics (df=66) might point to theoretical endeavors framing and conceptualizing close and distant reading. Alan Liu even argues that DH—the catch-all term distant reading in particular—has disturbed the truce between new criticism and cultural-critical readings post 1968: “An unspoken demilitarized zone thus intervened between close and cultural-critical reading. The digital humanities break this détente.”

Against this background, the document frequency of critical theory (df=62) can be further commented on. Our dictionary of cultural studies contains critical theory as one type. Critical theory includes, in a broader sense, many theoretical approaches, which “have emerged in connection with the many social movements that identify varied dimensions of the domination of human beings in modern societies” (Bohman). In a narrower sense, critical theory designates the Frankfurt School. Within the theory debate, David M. Berry and other scholars invoke critical theories to strengthen the role of criticism within DH research (Berry 140; Burdick et al. 76).

With regard to Table 2, it is also noteworthy that while poststructuralism is not mentioned explicitly as a theoretical framework, three of its most popular representatives—Roland Barthes (df=78), Michel Foucault (df=59), and Jacques Derrida (df=54)—have surprisingly high document frequencies. We will discuss the role of poststructuralism in more detail in the section on cooccurrence analysis, where we will encounter the names of these three French philosophers once again.

As opposed to the single mentions of theory items that were discussed previously, there are also terms that are referenced extensively within single articles. In our corpus, the terms with the highest average token count in one document are not umbrella terms, such as hermeneutics or poststructuralism, but rather specific theorists (full list available online).7 Jacques Lacan (dict_psychoanalysis) has an average token count per document of 11.6 and appears in 16 different papers. Lacan is closely followed by Vladimir Propp (dict_structuralism) with an average of 10.2 tokens per document and a document frequency of 20. Frank Raymond Leavis (dict_formalism_new criticism) is another example, as he is mentioned in a total of 8 papers with an average token count of 6.6.

Approaching these single articles via close reading reveals for Lacan that his theoretical constructs are discussed under a computational paradigm. More concretely, Terry Harpold deals with Lacan’s four discourses, while Tamise van Pelt’s article refers to a Lacanian notion of subjectivity. The case for Propp is a bit different. His formalistic approach seems highly adaptable to the demand for discrete categories that are readable by the computer. Journal articles referencing Propp are then concerned, for example, with the reproducibility of text annotations as well as machine learning (Fisseni et al.; Finlayson).

Leavis’s references mostly appear in the context of Charles P. Snow’s “two cultures” dichotomy. It is worth highlighting that the Leavis references seem to be intertwined with the narrative “bridging the gap” (Porsdam), which plays an important role in defining an epistemic culture of DH. Leavis (and Snow) are thus representatives of a larger humanities discourse, which is also echoed in DH.

Having discussed the frequencies of single theory items, we will now outline in more detail the results of our investigation into the relations between those items by means of cooccurrence analysis of theory items (full list of cooccurrences available online).8 These results (see Table 3) largely aligned with the results of the previous frequency analysis (see Table 2), with unexpected theory cooccurrences being rather rare. This might well be an effect of our dictionaries, which are limited to cultural and literary theories in a broader sense. We also did not use any significance weights like Pointwise Mutual Information at this point. Rather, this evaluation is intended to explore frequent patterns in the texts and to serve as a first plausibility check.

The highest ranks are again taken by renowned poststructuralists, namely Foucault, Barthes, and Derrida. As might be expected, cultural studies and hermeneutics frequently cooccur with these poststructuralist representatives. The pairing of Barthes and Foucault appears in a total of 18 journal articles, 9 of which are dedicated to the broader topic of authorship theories. Another explanation may be that cooccurrences of Foucault and Derrida (as well as Barthes) could be traced back to text encoding initiatives, which shaped, in particular, early DH projects. Schreibman points out that critics

In this regard, “hypertext” also appears in over half of the mentions of Derrida and Foucault (10 of 18 cooccurrences), while George P. Landow, who adapted Barthes’ ideas for his hypertext theory, is mentioned in these contexts, too.

A close reading of these passages suggests that poststructuralist approaches in DH are mostly being used to underpin aspects of modeling and textual representation. This anecdotal close read demonstrates how the shared uses of our previously defined dictionary items can be utilized to identify possible patterns and qualitatively evaluate them in more detail. The example shown here only highlights the theoretical embeddedness of a particular topic; however, with expanded term lists, more such forays would be possible to explain theory usage in DH publications.

Our semasiological investigations have revealed how usage, functions, and semantics of theory are interfering in DH research. The ranking of the document frequencies by type count provides a rather expected result of theory within DH. Hermeneutics, cultural studies (critical theory), and new criticism are frequently brought up as theoretical frameworks considering our dictionaries. What our case study indicates is that the empirical basis of the “end of theory”-narrative in DH is weak, insofar as there are diverse significant references to canonical authors in the tradition of theory in the humanities. This continuity of theoretical reflection might be worth telling, because it is contrary to the idea of a disruptive break in DH’s knowledge production. Jean Bauer has already stated in 2010 that “I am sick and tired of people saying that my friends, my colleagues, and I do not understand or care about theory. Every digital humanities project I have ever worked on or heard about is steeped in theoretical implications AND THEIR CREATORS KNOW IT” (Bauer’s emphasis). Moreover, it is noteworthy that our case study hints at an uneventfulness of theory in DH. This leads us to suspect that the narrative of “theorylessness” must play a different role for DH’s epistemic cultures. M. Beatrice Fazi explains that “the prospect of the end of theory is also reflected in popular concerns about the end of cognitive work due to algorithmic automation, and in related worries about the shrinking of human intellectual faculties in a society where rational decision is increasingly delegated to machines” (107).

The single mentions of theory items as well as its cooccurrences give further insights into the different ways theory is used within DH research. We also conducted some scalable readings by oscillating between quantitative explorations and context-sensitive references in the respective articles. While referencing Lacan, for example, seems to go hand in hand with reflecting on philosophical ideas such as subjectivity for DH’s specific issues, Leavis and Snow (“two culture”-dichotomy) show that DH’s theories are located within a larger story of theory in sciences. Propp’s formalistic approach, however, serves more as a demonstrative example of “theory in practice.” Cooccurrences can be interpreted in a similar vein. Theoretical references are made here in order to weave them into practices, tools, and (digital) representations. Although this operationalization of theory has been recently subjected to criticism (Alvarado), it is nevertheless reasonable to assume that theory references are thus linked to a certain expectation; namely, to address humanities claims under a computational paradigm. Future consideration could be given to systematically expanding the dictionaries. In particular, theories from media studies, linguistics, sociology, and computer science should be included.

One conceptual foundation of this case study can be found in the idea of distributional semantics. The distributional hypothesis suggests that words with similar distributions of context—that is, similar surrounding words—have similar meanings (Harris). For example, lion and tiger have other context words (e.g., teeth and claws) in common than car and bus (e.g., wheels and street). Word embeddings model such cooccurrences of words as vector representations in a multidimensional space, which then can be compared to each other using similarity metrics, such as the cosine distance (Mikolov et al.). We followed existing approaches which assume that semantic representations and their stability can be represented by means of word embeddings and that these embeddings can be compared between diachronic time periods in corpora (Martinc et al.; Giulianelli et al.; Kahmann et al.; Hamilton et al.; Jatowt and Duh).

As a first step, we fine-tuned a pre-trained BERT language model for domain adaptation on our DH journal corpus. Following the approach outlined by Matej Martinc et al., we did not conduct any diachronic fine-tuning. Since embeddings in the BERT language model are contextual, which means they are dependent on the time-specific context, we used this as our input to access the diachronic semantic stability. We used the English BERT-base-uncased model with 12 attention layers and a hidden layer size of 768. Much like Mario Giulianelli et al. and Martinc et al., we created sequences of byte-pair encoding tokens.

For each of these sequences, we generated a sequence embedding by summing the last four encoder output layers. The resulting sequence embedding represents a concatenation of contextual embeddings for the tokens in the input sequence. We sliced those concatenations and acquired a representation (i.e., a contextual token embedding) for each word usage in our journal articles. These representations differ depending on the context in which the token is embedded. As a consequence, the same word has a different representation in each context. Finally, we could aggregate the embeddings on the token level and were able to compare different time spans in the corpus with regard to the semantic representation of the theory-related vocabulary in our study.

In a first experiment, we extracted different word usage contexts of theory via its local embeddings, hoping to reveal different senses of theory. A k-means clustering approach was chosen to explore different embeddings in the texts (see Figure 3). We used a silhouette analysis to identify the optimal number of clusters (k), which in this case appears to be 6 clusters. Interestingly, the clusters mostly reflect different forms of syntactic embeddings of the word form theory, for instance “in theory,” “theory of,” “noun + theory,” and the plural form “theories,” as well as a cluster with concrete instances of theories and a cluster with rather conceptual aspects of theory.

The mere clustering of the local embeddings of the word theory is obviously limited in its expressive power regarding the role of theory in DH, however, as this simple approach does not reveal distinct senses and usage patterns of theory as a concept. We therefore also took a diachronic perspective on theory, aggregating the concept’s local embeddings for different time slices. To gain better insights into different senses of theory, we also chose to analyze semantically related concepts, to see how those relate to theory in the course of time.

Because we sought to examine relations of theory to contextual word embeddings of further concepts that play a role in DH’s epistemic cultures, we took a closer look at model, method, experiment, and tool, as they seem essential for characterizing DH research and are frequently encountered in corresponding discourses. Above all, we were interested not only in semantic ambiguity but also in questions of controversiality and indispensability of theory. Modeling, for instance, is described as a core DH activity (Flanders and Jannidis, Knowledge Organization and Data Modeling in the Humanities). At the same time, DH research is considered to expand the methodological repertoire of the humanities by using tools and other infrastructural settings. Finally, the term experiment will be discussed against the background of the emerging DH laboratories (Pawlicka-Deger).

To provide an overview of the basic contexts of the epistemic concepts theory, model, method, tool, and experiment, Table 4 shows the top-10 terms that have the most similar embedding vectors. We will revisit Table 4 in the following discussion on the relationship between these five concepts. Figure 4 provides an overview of how the concepts tool, model, experiment, and method evolve with regard to their semantic similarity to theory. Following the suggestions of Shen Dinghan et al. and Vitalii Zhelezniak et al., we used max pooling of the contextualized embeddings of each term within a 3-year slice, as it typically takes out the influence of syntactical embedding information.

Interestingly, theory is almost a straight line, as its self-similarity is rather high, even from the beginning of the period surveyed. As time goes by, some more nuanced contexts are added to the theory vector. The vectors of the other concepts are stacked for every 3-year-slice, which means as time goes by, more contextual meaning is also added to the other vectors. The graph also shows that the ranking of similar concepts is stable over time, meaning model is always the concept that is most similar to theory, then comes method, experiment, and, finally, tool, which has fairly low numbers in the beginning.

As Figure 4 shows, we initially observe a gradual increase of contextual meaning of theory, while from the 1990s on, contextual meanings of theory became established. As with model, between the 1966 and 1990, an increase of contextual meanings can be observed, but this quickly levels off. According to our experimental design, it seems worth mentioning that theory and model were neither used synonymously nor did they gradually diverge. Rather, both concepts converged quite soon to their final contextual meaning. Fotis Jannidis and Julia Flanders mention that within DH research views on formal modeling among others are advocated which “tend […] to focus on how the concept of the ‘model’ is itself embedded in more general concepts like theory and how they, theories and models determine or at least interact with the formal model” (28). Comparing both context vectors may broaden the perspective of how model and theory are related.

As Table 4 shows, different semantic fields of theory and model can be described over the entire period covered by our corpus. The terms principle, hypothesis, doctrine, and paradigm indicate that theory rather covers contexts of regularity as well as (axiomatic) programmatic and forms of intersubjective knowledge, while model has more contextual similarity with terms such as representation, prototype, and simulation, which focus on different kinds of mapping as well as formal systematizations. This could be one reason why theory and model have the same contextual distance to each other (see Figure 4): they consistently seem to cover different semantic fields. It could be assumed that these two semantic fields might complement each other. Thus, in the top-10 ranking of terms with the most similar context words as model, theory takes the third highest rank. This is not so with model, which does not appear in the top-10 ranking of theory at all.

These semantic frictions between theory and model seem to represent the ambiguity of these concepts within DH research. Neither the concepts nor their semantic fields can be transferred—rather, they intertwine. In comparison, the relation between theory and method seems to be more explicit. Not only do they cover distinguishable semantic fields, as Table 4 clearly shows, but the top terms of methods include technique, procedure, approach, and strategy, while the distance of the context-vectors remains constant (see Figure 4).

A slightly different picture emerged when we compared the semantic contexts of theory and experiment (see Figure 4). While the vector for experiment seems to have been established in the first six years, it then slowly converges to theory, adding further meanings. The top-10 highest-ranked terms for experiment, however, point to an (oppositional) semantic field; namely, the more practical side of DH research. Moreover, the highest ranking terms for experiment seem to break into two semantic fields: 1) terms such as trial, test, laboratory, investigation, and observation suggest contexts that could be associated with empirical science research settings. The fact that the first place is occupied by trial is particularly noteworthy, as it introduces notions of failure within DH research. 2) The terms research, study, and project seem to bring into sharper relief ongoing transformations in DH in general. Strictly speaking, the most similar context to research and project has the term experiment with regard to our five selected terms. According to Ian Hacking, experiments are not subordinate to theory; rather, “experimentation has a life of its own” (Hacking 150). With regard to the decoupling of experiment and theory building, Willard McCarty concludes that for DH,

Interestingly, however, the semantic similarity between theory and experiment does not change within our corpus (see Figure 4). Thus, no meanings are added that would further distance the two concepts. This situation is somewhat different for the concept tool. Between 1966 and 1985 the tool vector fluctuated in its movement as it drew closer and moved further away from the theory vector. From the mid-1980s on, there is a steady approximation. It could be hypothesized that one reason for this constant convergence of these two vectors might be the emerging idea of a theory-driven development of tools.

Next, we took a closer look at the theory vector and how it changed its contexts over time. We created a matrix that contains all the ranks for all the other terms for each of the 3-year-slices (full list of ranks over time available online).9 For the purpose of a more consistent comparison that is not distorted by grammatical or syntactic effects, we compared theory only to other nouns.

Figure 5 provides an overview of the top-25 terms that are best ranked by means of their cosine similarity to the concept theory throughout the entire corpus. As some of these concepts have quite some fluctuations in their ranks through time, it is impossible to visualize the whole spectrum of ranks for multiple terms in one plot. Therefore, we decided to only visualize the rank movement within the first 25 ranks. Theorist, for example, steadily increases its ranks (1969–1972: rank 1,030 → 1981–1983: rank 472 → 2002–2004: rank 49), but it is only in the time slice of 2011 to 2013 that it hits the top 25 ranks and thus appears in the graph.

Strikingly, the terms principle, idea, and concept have the most similar context vectors to theory, indicating a rather fundamental and conceptual use of theory in the context of DH research articles. These terms are closely followed by the term model, reaffirming the observations about the rather stable and semantically distinct relation between theory and model that were already described for Figure 4. Interestingly, technology and methodology only appear in the top-20 most-similar terms in the early 1990s. It could be argued that technology’s appearance in the top 20 is related to the dissemination of large and searchable (textual) databases, and more generally to the advent of the World Wide Web and its manifold technological inventions and implications. This seems to be somehow related to the need for a methodology, in other words a more abstract reflection, systematization, and theorization of methods.

While methodology seems to co-evolve with technology, method on the other hand remains rather stable and rather close to theory throughout time. Surprisingly, Underwood characterizes the 1990s as the period where theory building was missed (70). At the same time, the 1990s seem to play a crucial role in DH’s epistemic culture, as the periodization approach introduced by Todd Presner and Jeffrey Schnapp, who suggest two main waves of DH, also starts precisely here. The first wave, from the mid-1990s to the early 2000s, focuses “on large-scale digitization projects and the establishment of technological infrastructure.” The second wave, which entails the period from the early 2000s until 2010, is “deeply generative, creating the environments and tools for producing, curating, and interacting with knowledge that is ‘born digital’ and lives in various digital contexts.” While this periodization has been criticized by some, we can definitely add to this debate with our observation that significant semantic shifts can be observed during this period (Berry 4). As technology and methodology gain new meanings, they also become more similar to the contextual embeddings of theory.

Our second case study presents an onomasiological investigation of theory within DH research. A key insight from the comparison of theory and related epistemic concepts shows that most acquired their full contextual meaning early, in the first five years of our corpus. Afterwards, the concepts remained rather stable, in terms of their self-similarity on the one hand, and their similarity to the concept of theory on the other. A further investigation of the epistemological role of tools in DH is very much needed.

Another interesting insight comes from the comparison of the two contextual embeddings of theory and model. It became apparent that model comes closer to contexts of representation. Moreover, the relation of (opposite) terms to theory such as experiment and tool was found to be rather stable and expected. To put it simply, our contextual embeddings stories are more predictable and less eventful than the often polarized (research) discourse might convey. Looking at the time component of theory references, it is striking that the mid-1990s was a point when new (conceptual) stories began to contextually interfere with each other or were forgotten (Underwood). Further research approaches could therefore focus more on periodization issues.