Like Two Pis in a Pod: Author Similarity Across Time in the Ancient Greek Corpus

One commonly recognized feature of the Ancient Greek corpus is that later texts frequently imitate and allude to model texts from earlier time periods, but analysis of this phenomenon is mostly done for specific author pairs based on close reading and highly visible instances of imitation. In this work, we use computational techniques to examine the similarity of a wide range of Ancient Greek authors, with a focus on similarity between authors writing many centuries apart. We represent texts and authors based on their usage of high-frequency words to capture author signatures rather than document topics and measure similarity using JensenShannon Divergence. We then analyze author similarity across centuries, finding high similarity between specific authors and across the corpus that is not common to all languages. Traditional analyses often assert that Ancient Greek authors imitate models set by previous authors. Arrian, writing in the second century CE, is said to have based his Indica on the model of Herodotus,1 Aelius Aristides is said to have written in the style of Demosthenes and other rhetoricians,2 and Apollonius Rhodius is said to have crafted his Argonautica in the style of Homer. As Antonios Rengakos puts it, the Argonautica is full of “imitations of Homeric phrases, verses, motifs or scenes and reproduces lexical, morphological, syntactical and metrical peculiarities of the old epic.”3 Most prior analyses focus on a few authors and examine highly visible, marked imitation, like the use of words that appear once in the entirety of the Homeric corpus (hapax legomena)4 or the reuse of nearly identical phrase structures from earlier speeches.5 While previous studies have focused more specifically on imitation by individual authors, many Ancient Greek texts drew on prior works, with authors in genres like history, rhetoric, and poetry often writing in a specific literary dialect to match earlier works.6 This means we might expect to see imitation of earlier models not just by a L I K E T W O P I S I N A P O D : A U T H O R S I M I L A R I T Y A C R O S S T I M E I N T H E A N C I E N T G R E E K C O R P U S 2 few authors, but by many authors throughout the corpus. This hypothesis is difficult to test, as the idea of imitation involves intention to be similar, and intention is difficult to verify on a case-by-case basis, let alone across a corpus. In this work, we will instead focus only on similarity between authors, and leave determining author intention to other works. More specifically, we will evaluate the hypothesis that authors across the extant Ancient Greek corpus, even those writing centuries apart, are very similar to each other when compared to authors from other corpora. To test this hypothesis, we must have a way to measure similarity between two authors or texts and baseline corpora for comparison with the Ancient Greek corpus. The similarity measurement will enable us to discuss author similarity across the corpus, and the baseline corpora will allow us to determine whether the similarity between authors in the Greek corpus is in line with patterns in any language corpus or whether it shows unusually high (or low) similarity. Computational techniques allow us to analyze similarity in ways that go beyond focusing on a few specific instances. Recent work has begun to use computational methods to analyze classical texts, including authorship and allusion in Latin texts7 and the syntactic style of Attic prose.8 Our goal in this work is to analyze similarities in the writing style of Ancient Greek authors. The definition of “style” is a thorny problem that we cannot fully address within the context of this work, but at a high level we might expect the “style” of a section of text to be informed by some combination of its genre (e.g., fantasy, biography, epic poem, military history, philosophical treatise, etc.), dialect (e.g., American vs British English), time period (English as a language changes from Shakespeare to Charles Dickens to J.K. Rowling), register (simple or artistically stylized), and other internal tendencies of the author. Parts of this “style” might vary across different works from the same author, or even possibly within a text in the case of a work with multiple styles like Faulkner’s As I Lay Dying. In this paper we focus on authors’ usage of very common words, which has been used successfully in non-classics work on stylometry to capture information about an author’s writing style.9 There are a variety of potential caveats and issues which we will discuss as they come up, but the idea of an author signature provides us with a more concrete starting point for comparing different authors. J O U R N A L O F C U L T U R A L A N A L Y T I C S 3 While prior works focus on using author signatures of this sort to resolve authorship questions about a given text, our work is focused on comparing word usage of known authors writing in different styles and time periods. There has been past work on examining imitation with regards to content words, dialect forms, or salient phrases10 as well as analyses of allusion and intertextuality based on phrases or a small number of words.11 However, to the best of our knowledge previous study has focused on comparing relatively short phrases between a few works, rather than considering “style” or author signature in entire texts across a larger corpus. For our baseline comparison corpora, we choose texts from English and Icelandic. These corpora, like the Greek corpus, have a mix of genres spanning around six to ten centuries of time with texts from centuries across that span. We chose English because we expect authors to show significant differentiation over the past centuries as the language changed from “Middle” to “Modern” English.12 Conversely, we chose Icelandic because we expect authors to show more similarity across the centuries, as linguists consider Icelandic to be a relatively conservative language in terms of change over time.13 It is also an apt comparison for Greek because we have versions of texts from a wide range of time periods with standardized spelling and morphology. Between these two corpora – one quickly changing, one more conservative – we hope to contextualize the similarity over time in our Greek corpus. Our goal in choosing these additional corpora is to determine whether the stability we observe in Greek is universal among languages with long-term written traditions. We make no claim that the pattern we observe in Greek is unique to that language, and in fact we suspect that similar patterns may exist in other ancient languages such as Akkadian, Latin, and Sanskrit. Using the top words as a measure of author signature and English and Icelandic as baseline corpora, we make three major contributions in this work. First, we show that a feature set based on top words captures information about authors that matches conclusions from prior non-computational scholarship in classics. Second, we explore various metrics for measuring similarity between authors and find that, for the task of predicting the work and author of text segments, a similarity metric based on Jensen-Shannon Divergence performs very well, showing 2.5% improvement over Burrow’s Delta and 6% improvement over Cosine Similarity. Finally, we analyze the relationship between author similarity and relative composition date, and show that, when compared to authors in English and Icelandic, Ancient Greek L I K E T W O P I S I N A P O D : A U T H O R S I M I L A R I T Y A C R O S S T I M E I N T H E A N C I E N T G R E E K C O R P U S 4 authors are far more similar across longer time periods. We find similarity between authors both at the individual author level and across the whole corpus that is significantly greater than the collections of similar chronological breadth in English and Icelandic.

Traditional analyses often assert that Ancient Greek authors imitate models set by previous authors. Arrian, writing in the second century CE, is said to have based his Indica on the model of Herodotus, 1 Aelius Aristides is said to have written in the style of Demosthenes and other rhetoricians, 2 and Apollonius Rhodius is said to have crafted his Argonautica in the style of Homer. As Antonios Rengakos puts it, the Argonautica is full of "imitations of Homeric phrases, verses, motifs or scenes and reproduces lexical, morphological, syntactical and metrical peculiarities of the old epic." 3 Most prior analyses focus on a few authors and examine highly visible, marked imitation, like the use of words that appear once in the entirety of the Homeric corpus (hapax legomena) 4 or the reuse of nearly identical phrase structures from earlier speeches. 5 While previous studies have focused more specifically on imitation by individual authors, many Ancient Greek texts drew on prior works, with authors in genres like history, rhetoric, and poetry often writing in a specific literary dialect to match earlier works. 6 This means we might expect to see imitation of earlier models not just by a few authors, but by many authors throughout the corpus. This hypothesis is difficult to test, as the idea of imitation involves intention to be similar, and intention is difficult to verify on a case-by-case basis, let alone across a corpus. In this work, we will instead focus only on similarity between authors, and leave determining author intention to other works. More specifically, we will evaluate the hypothesis that authors across the extant Ancient Greek corpus, even those writing centuries apart, are very similar to each other when compared to authors from other corpora.
To test this hypothesis, we must have a way to measure similarity between two authors or texts and baseline corpora for comparison with the Ancient Greek corpus. The similarity measurement will enable us to discuss author similarity across the corpus, and the baseline corpora will allow us to determine whether the similarity between authors in the Greek corpus is in line with patterns in any language corpus or whether it shows unusually high (or low) similarity.
Computational techniques allow us to analyze similarity in ways that go beyond focusing on a few specific instances. Recent work has begun to use computational methods to analyze classical texts, including authorship and allusion in Latin texts 7 and the syntactic style of Attic prose. 8 Our goal in this work is to analyze similarities in the writing style of Ancient Greek authors. The definition of "style" is a thorny problem that we cannot fully address within the context of this work, but at a high level we might expect the "style" of a section of text to be informed by some combination of its genre (e.g., fantasy, biography, epic poem, military history, philosophical treatise, etc.), dialect (e.g., American vs British English), time period (English as a language changes from Shakespeare to Charles Dickens to J.K. Rowling), register (simple or artistically stylized), and other internal tendencies of the author. Parts of this "style" might vary across different works from the same author, or even possibly within a text in the case of a work with multiple styles like Faulkner's As I Lay Dying. In this paper we focus on authors' usage of very common words, which has been used successfully in non-classics work on stylometry to capture information about an author's writing style. 9 There are a variety of potential caveats and issues which we will discuss as they come up, but the idea of an author signature provides us with a more concrete starting point for comparing different authors.
While prior works focus on using author signatures of this sort to resolve authorship questions about a given text, our work is focused on comparing word usage of known authors writing in different styles and time periods. There has been past work on examining imitation with regards to content words, dialect forms, or salient phrases 10 as well as analyses of allusion and intertextuality based on phrases or a small number of words. 11 However, to the best of our knowledge previous study has focused on comparing relatively short phrases between a few works, rather than considering "style" or author signature in entire texts across a larger corpus.
For our baseline comparison corpora, we choose texts from English and Icelandic. These corpora, like the Greek corpus, have a mix of genres spanning around six to ten centuries of time with texts from centuries across that span. We chose English because we expect authors to show significant differentiation over the past centuries as the language changed from "Middle" to "Modern" English. 12 Conversely, we chose Icelandic because we expect authors to show more similarity across the centuries, as linguists consider Icelandic to be a relatively conservative language in terms of change over time. 13 It is also an apt comparison for Greek because we have versions of texts from a wide range of time periods with standardized spelling and morphology. Between these two corpora -one quickly changing, one more conservative -we hope to contextualize the similarity over time in our Greek corpus. Our goal in choosing these additional corpora is to determine whether the stability we observe in Greek is universal among languages with long-term written traditions. We make no claim that the pattern we observe in Greek is unique to that language, and in fact we suspect that similar patterns may exist in other ancient languages such as Akkadian, Latin, and Sanskrit.
Using the top words as a measure of author signature and English and Icelandic as baseline corpora, we make three major contributions in this work. First, we show that a feature set based on top words captures information about authors that matches conclusions from prior non-computational scholarship in classics. Second, we explore various metrics for measuring similarity between authors and find that, for the task of predicting the work and author of text segments, a similarity metric based on Jensen-Shannon Divergence performs very well, showing 2.5% improvement over Burrow's Delta and 6% improvement over Cosine Similarity. Finally, we analyze the relationship between author similarity and relative composition date, and show that, when compared to authors in English and Icelandic, Ancient Greek authors are far more similar across longer time periods. We find similarity between authors both at the individual author level and across the whole corpus that is significantly greater than the collections of similar chronological breadth in English and Icelandic.

Data and Corpora
Our analyses focus on texts available through the Perseus Digital Library's Greek Collection. 14 This corpus includes 464 works from 92 authors spanning from circa the 8th Century BCE to the 6th Century CE. Where possible, we divide each work into smaller segments: for example, we divide Herodotus' Histories (a single "work") into its nine books (each of which is a "segment"). When a work cannot be broken up naturally, it is considered a single segment, so Euripides' Medea is both one work and one segment. This partition leads to 1,337 total segments. Of these segments, 1,139 (from 65 authors) are prose and 198 (from 27 authors) are poetry. We remove non-Greek characters and punctuation and, where possible, restore elided tokens, so παρ' is restored to παρά. The full dataset has 9,707,987 tokens (total words) and 486,326 types (unique words) after preprocessing. See Table 1 for a top-level breakdown of the size of works and segments.
We examine the dataset in two forms. First, we group works by author, including all authors, no matter how little text there is. Each author has at least 2,000 tokens except for Bion of Phlossa (1,803 tokens) and the anonymous author of the fragmentary Hymn to Dionysus (just 144 tokens). The small sample size does not seem to adversely affect analysis of Bion. The short length of the Hymn to Dionysus does have an impact on some analyses, which we discuss below. We also analyze the texts divided into individual segments, only considering segments with at least 1,000 tokens. This gives 1,204 segments to analyze (out of the total of 1,337).
We expect that the language of Ancient Greek prose and poetry will show clear distinctions. Ancient Greek poetry, in addition to constraints of grammar, topic, and style, also had to conform to a poetic meter. There were a variety of meters used in different contexts (e.g., Dactylic Hexameter was the meter of epic poetry) but all of them mandated some pattern of long and short syllables in each line of poem. This means that, unlike prose, all Ancient Greek poems are naturally structured around lines and have extra constraints on how sentences can be constructed.
Stylometric analysis is more uncertain for ancient documents than modern documents because of their complex chains of transmission. Unlike most work on modern authors, it is not safe to assume that the published version accurately represents the author's original work and, by extension, their author signature. Because the texts were written so long ago, our modern editions are an editor's reading of a set of medieval manuscripts, which may have long histories themselves. For example, after Euripides wrote the text of Medea it was passed down by actors in Athens, then standardized by scholars in Alexandria, then transmitted in a variety of manuscripts through the medieval period to the modern day, where the manuscripts were combined by an editor and then digitized to create the single version of the text which we use. Even if we find patterns in word use in modern digital texts, there is no guarantee that these patterns existed in the text as first written by Euripides. Previous work on medieval Dutch texts has even shown that copying scribes can introduce their own signature to texts. 15 Accounting for the interaction between the editorial and scribal artifacts of the manuscript tradition and the output of our method is beyond the scope of this study but is important to acknowledge.
The English corpus has 204,366,114 tokens and 701,562 types (see Table 1 for a top-level breakdown of the size of works and segments). It is a combination of the following corpora, resulting in 166 authors with 2,759 unique works and 2,960 segments: • Modern English texts from the Gutenberg Dataset, 16 with a few duplicate texts and texts including a mixture of prose and poetry removed.
• The Icelandic corpus has 7,587,999 tokens and 290,924 types (see Table 1 for a toplevel breakdown of the size of works and segments). It is a combination of the following corpora, resulting in 196 authors with 213 unique works, each of which is one segment: • Icelandic Sagas from the Saga Corpus, with duplicate manuscripts removed. 21 • The Icelandic Parsed Historical Corpus (IcePaHC), a collection of texts from 1150-2008 CE. 22 Duplicate manuscripts and translations were removed.
• 21st Century Icelandic texts from the Tagged Icelandic Corpus (MÍM). 23 We only used the texts labeled books, with articles by Baldur Jónsson and recipe books removed.
For both languages our goal was to create corpora with texts from a wide range of centuries by tying together multiple existing corpora from more localized time periods. This allows us to create baseline corpora that are comparable to Greek.  We would also like to confirm it is fair to compare usage of top words in these corpora. If the Ancient Greek vocabulary was consistent across time while the English and Icelandic vocabulary changed drastically this would provide a simple explanation for greater stability over time in Greek without the need for further exploration. In order to get a quick sanity check on change in word usage over time in our corpora, we calculate the 100 most frequent words for each century, then examine the amount of overlap in these words for time periods six centuries apart. For the English corpus, the mean and median are 48 words of overlap. For the Icelandic corpus, the mean is 68.25 and the median is 68. For the Greek corpus, the mean is 68.14 and the median is 71. These results show that the Icelandic corpora is a good comparison for Greek due to their similar stability of vocabulary over time.
The English corpora shows less stability but seems reasonable for a comparison corpus chosen to show more change over time and fits with the claims about the relative stability of English and Icelandic.

Document Representations through High-Frequency Words
As discussed above, in order to analyze author similarity in the Ancient Greek corpus, we must have a way to measure it. To measure similarity, we first extract each author's usage of some top words as a feature vector and then use a metric to measure the similarity between these feature vectors. We begin with the process for extracting our feature vector and show that it captures useful information about authors and agrees with conclusions from prior works in classics.
In this work, our feature set for measuring author signatures is the frequency of the most frequently used tokens across the corpus. Usage of the most frequent words has shown promising results for identifying author signatures in the past, in particular because content words are more dependent on genre and topic matter. 24 Some work has found that frequent words alone can do better than part of speech information or a combination of the two. 25 The intuition behind analyzing top words is that an author's usage of these words (say, the ratio between their usage of "but" and "and") is mostly unconscious and provides a signature (or fingerprint) for that author, while their usage of content words (say, "tropical" or "lodestar") reflects the topic of their work or conscious decisions about their style rather than a fundamental characteristic of the author.

Ancient
Greek is highly inflected, so a common first step when working with these texts is to replace surface forms with lemmata (e.g. "trees" to "tree" and "held" to "hold"). While choosing our top words we consider only surface forms, not lemmata, for two reasons. First, there is valuable information in the inflection of highfrequency words: for example, an author's usage of τῆς as compared to τόν is an interesting and potentially relevant distinction. Second, lemmatization is not always reliable: many tokens are ambiguous, and we are not able to resolve these across 9.7 million tokens.
When choosing our list of top words for each language, we select the 250 most frequent words across all texts that also occur in more than 50% of authors. This cutoff prevents inclusion of words, mostly names, that appear frequently in only a few authors. In Greek the excluded words are Ῥωμαίων, "Romans'" (27% of authors), Σωκράτης, "Socrates" (29%) and Ἀθηναῖοι, "Athenians" (50%). In Icelandic the excluded words are Ned (15% of authors), konungur, "king" (44%) and Jón (45%). There are no words excluded in the English corpus. Raising the cutoff to 60% would remove a variety of non-name words including Greek ὑμᾶς, "you (pl.)," Icelandic kannski, "maybe," and English "should." The full list of tokens used for each language can be found in Appendix 2.
In Measuring Similarity Between Authors and Segments below we also examine a list of words including not only the top 250 words across all texts combined but also the list of the top 100 words in only the poetic texts. Due to overlap between these two sets, this yields a list of 264 total words. Common words from poetry were considered because the word usage in Ancient Greek poetry and prose has key distinctions, as we will see below. Since the corpus is dominated by prose, including more words that are specifically relevant to poetry (including more poetry-specific words like κεν and words that appear more often in poetry than prose, like Ζεύς) may help better capture the signatures of poetic authors. The full list of added words is τοι, μιν, ἦ, ἀμφί, αὐτάρ, Διός, σ᾽, ἐνί, περ, οὔ, ἔνθα, κεν, Ζεύς, πατρός.
In the following, our feature set is the frequency of the top words within each author or segment. We therefore represent each author and segment with a vector of 250 or 264 features, where ! corresponds to the frequency of word within the author or segment. ! = (#of occurrences of word in text ) (total #of words in text ) Note that the total number of words includes all words, not just the top 250/264.

Detecting Author Characteristics
Before using the set of most frequent words to analyze author similarity, we show that this feature set captures information about the texts by analyzing the Ancient Greek data to see if there are any interesting properties that can be detected from the word representation vectors.
We begin with a visualization of similarity between texts based on the top 250 words.
Since the position of points in 250-dimensional space is difficult to present, we use a two-dimensional tSNE projection 26 in Figure 1. The tSNE visualization attempts to preserve relative distances, so points that have similar word usage appear closer together. The most salient feature of Figure 1 is the large gap between authors writing prose (red) and poetry (blue), suggesting that 250 words are sufficient to distinguish these genres. In fact, the difference is so clear that the Hymn to Dionysus fragment is clearly recognized as poetry based on a tiny text sample (144 total tokens). The clear distinction between prose and poetry texts gives some hope that other characteristics of these authors might be distinguishable as well. In Figure 2  The unsupervised tSNE Projections do not show a clear distinction between categories beyond poetry/prose, but we may still be able to detect these categories using machine learning. To test this hypothesis, we run three classifiers: A Majority Class baseline, K Nearest Neighbors (with K=2 chosen from the set {1, 2, 3, 5, 10, 20} based on best performance on the training folds) and a Multinomial Naive Bayes. For each classifier, we divide the data into nine folds. Since there are 92 authors, this provides around 10 authors to test on for each fold. The folds are chosen by randomly dividing the given author or segments into groups of roughly equal size. In addition, the segments are divided so that segments from the same work are in the training or test fold, but not both, except when predicting authors (marked with a *) to account for the fact that many authors in the dataset have only a single work and would therefore be impossible to accurately classify without relaxing this constraint. For each fold, we evaluate a model trained on the other eight folds.  Both the KNN and Naive Bayes classifiers do extremely well at predicting genre (poetry vs prose) of authors and segments, achieving >98% accuracy in all cases, which is in line with prior state of the art techniques. 28 Dialect and time period prediction for authors are slightly worse, but still far better than the majority class baseline. When considering individual segments, K Nearest Neighbors performs better than the majority class at predicting dialect and time period. Because the test set does not include segments from books in the training set, this accuracy is not based on detecting segments from the same book. Allowing segments from the same book in both folds increases the accuracy of KNN to 96% and 95% for dialect and time period prediction respectively.
The final row of Table 2 also shows the accuracy of these classifiers at predicting the author of a segment. Naive Bayes accurately classifies 88% of segments, with KNN performing almost as well. This performance is not perfect and could certainly be improved, but it does show that the features are at least partially predictive of authorship, even with relatively simple techniques.
When considering the authors together, the data can be used to predict dialect and time period much better than a simple baseline, but not well enough to confidently classify every text. With the larger amount of data present in the segment-bysegment analysis, we can predict dialect, time period, and to a lesser extent author with high accuracy. At the author level, dialect and time period are perhaps hard to predict with the small amount of data, but at the segment level they are reasonably predictive, even if they do not show clearly distinct clusters.
These results could perhaps be improved by using more complex classifiers with greater hyperparameter tuning, but that task is beyond the scope of this paper. Even these results show that the feature set we have chosen -examining the frequency of the top words within an individual segment or an author's work as a wholecaptures information about these texts including genre (poetry vs prose), and, when there are many samples in the segment case, dialect, time period, and authorship.

Analyzing Author Segments
We have seen that our feature set captures relevant information about our texts; we now show it provides results that agree with prior work in classics. One frequent area of discussion in classics is the authorship of specific texts. When considering works or segments attributed to a specific author, we expect that segments which are outliers based on our feature set were either written by a different author or written in a very different style. To visualize this, we create a 2-dimensional tSNE projection of the many different segments, as we did for the authors, and analyze segments that are outliers compared to the author's other segments. For the most part, segments by similar authors cluster together nicely, but there are some exceptions. Each panel in Figure 3 highlights a single author's segments in blue, showing one clear outlier. In the upper left, we see speeches attributed to Demosthenes, with speech 59 (Against Neaera) distinct from the rest. Critics since Dionysius of Halicarnassus have considered this speech to be by an author other than Demosthenes, 29 with some modern scholars attributing the text to Apollodorus. 30 On the other hand, the upper right shows that Isocrates' Speech 21, Against Euthynus, is also a clear outlier, but the attribution of this speech to Isocrates is quite secure. 31 While the text is almost certainly the work of Isocrates, it is recognized as being in a markedly different style.
The other two cases are more complex. Xenophon's authorship of the Cynegeticus (bottom left) has long been both challenged and defended, but it is unquestionably written in a very different style from the rest of Xenophon's work. 32 Our analysis (bottom right) supports a recent study by Thomas Koentges that found Plato's Menexenus to be the most unusual of his works based on a few computational analyses. 33 This text has been suspected of being non-Platonic in the past, but not recently. 34 Although our analysis highlights the existence of differences, further work would be required to distinguish the "different author" and "same author, unusual style" possibilities.
Across a few different data analyses, we have shown that high-frequency words capture valuable information about authors and texts, including genre, time period, dialect, authorship, and style. Based on the performance of our simple classifier with respect to various author characteristics and the agreement with prior work concerning outlier segments from certain authors, we can be confident that these features are capturing characteristics of the authors and will be a good basis for comparing varied writing signatures.

Measuring Similarity Between Authors and Segments
Given our feature set, the second part of calculating similarity is a using consistent metric for comparing top word use from different authors. We use the Jensen-Shannon Divergence metric, finding that it slightly outperforms other similarity metrics at predicting the author and work of individual segments.
A variety of methods have been used in authorship attribution, including comparing raw frequencies, 35 using Euclidean distance, 36 and bootstrap consensus trees, 37 among others. One of the most commonly used metrics is Burrows' Delta, which compares normalized relative usage of top words. 38 To get a similarity metric rather than a divergence, we calculate

JS Similarity( , ) = 1 − JS Divergence( , )
Since this metric compares two probability distributions, we include the total number of non-top words in addition to each of the top words. We calculate probabilities using the frequency of words in each author, work, or segment, and to prevent probabilities of zero we add 1 to the count of each word before computing this frequency. The metric is symmetric and decomposes over individual words, meaning that we can tell how much each individual word (καί, δέ, etc.) contributed to the similarity between two authors. Cosine Similarity, which is commonly used for comparison, does not have this feature.

Metric Evaluation
To evaluate the performance of Jensen-Shannon Similarity, we compare its performance to four other common similarity metrics: Burrows' Delta, discussed above, a popular metric for authorship analysis, 40 Manhattan Distance (which focuses on absolute differences in word usage), Canberra Distance (which focuses on relative differences in word usage), and Cosine Distance.
We expect that for a given segment, the most similar segment should be from the same work and by the same author. To test this, we first examine all segments that have another segment from the same work (e.g., Iliad book 1 has all the other books of the Iliad in the same work) and determine whether the most similar segment is from the same work using a leave-one-out validation method. This validation is equivalent to a K Nearest Neighbors classifier with 1 neighbor and distance determined by the given metric. We next compare all segments that have another segment by the same author and determine whether the most similar segment is by the same author, again using leave-one-out.
For each metric, we also examine its performance when using the top 250 overall words and the top 250 overall words + top 100 poetry words (for a total of 264 words). As a comparison for the significance of our metrics, we consider two baselines: Cosine Similarity, which is commonly used for comparing distributions, and Burrows' Delta, which is used for analyzing authorship based on top words. For both metrics, we choose the top 250 overall words as our baseline to determine whether using extra poetry words shows improvement.  When comparing segments from the same work ( In both cases, Jensen-Shannon Similarity performs the best of the metrics with high significance (p < 0.01). This advantage is not due to plus-one smoothing or the number of unrecorded non-top words: adding this information does not significantly improve the other methods. We also recognize that because Jensen-Shannon Similarity relies on probability estimates, it may suffer from bias when working with smaller texts. Even with this bias, it shows strong performance on these prediction tasks, including analysis of all of the shortest Ancient Greek segments. Considering this, we proceed with the similarity metric while keeping an eye out for patterns that appear to result from biased probability estimates from small segments rather than actual signal in the text.
It is also worth noting that adding 14 extra poetry words does not always improve accuracy, and for Jensen-Shannon it actually decreases it. Given this result, we will use the top 250 words without poetry words for further analysis, which allows us to avoid potential bias from using a metric constructed with some knowledge of the corpus (i.e. poetry vs prose authors).
These results show that Jensen-Shannon Similarity with the top 250 words is the best metric at identifying work or author, and we can be confident that high similarities according to this metric are based on actual similarities of the texts. We also note that Gerlach and Font-Clos independently found Jensen-Shannon Divergence to be a useful metric for comparing texts of different genres. 41 It is possible that there may be better metrics available, or performing preprocessing such as document normalization may improve the performance of some or all of our metrics. Our goal, however, is to find a metric that performs well enough at detecting similarity to be useful for further analyses of similarity, not to find the absolute best possible metric, and we believe Jensen-Shannon Similarity meets this standard.

Analyzing High-Similarity Authors
Now that we have established that our feature set and metric form a good measurement of similarity between different segments, we can begin evaluating whether authors from our Ancient Greek corpus are more similar across long time periods than authors from our English and Icelandic corpora. We begin by taking the top 100 closest author pairs in Ancient Greek and looking at the pairs of authors writing at least four centuries apart. In the Ancient Greek corpus, there are 23 such pairs, which is far more than we find in the baseline corpora.
Of these top 100 closest author pairs in Ancient Greek, 23 pairs wrote at least four centuries apart. These pairs fall into a few clear categories: , who wrote about a century after Dionysius with a similar signature (these two are the 8th most similar pair in our set). The result indicates that Eusebius writes like Flavius Josephus, who in turn writes like Dionysius of Halicarnassus. This is similar to the Attic orators above: Aelius Aristides did not attempt to imitate every one of these authors at once, but wrote in an Attic style based at times on Demosthenes, Isocrates, Plato, and Xenophon. 42 Aelius therefore appears similar to, say, Aeschines due to his similarity to Demosthenes and Isaeus, who are in turn very similar to Aeschines.
These authors are more similar than authors writing at the same time in similar genres, such as Isocrates and Lysias (members of the "Ten Attic Orators"), Plato and Xenophon, or Aratus and Callimachus, and they are far more similar than the historians Herodotus and Thucydides (697th most similar). So there does seem to be evidence of high similarity between authors writing in similar genres, even centuries apart.
Authors writing at least four centuries apart but still showing high levels of similarity also appear more frequently in the Ancient Greek corpus than the baselines. Of the top 100 (2.4%) pairs of Greek authors, 23 (23%) wrote at least four centuries apart.
Of the top 2.4% pairs of Icelandic authors, 2.6% (12/457) are writing at least four centuries apart, and of the top 2.4% pairs of English authors, 0% (0/328) are writing at least four centuries apart; in fact, none of these top pairs are writing more than one century apart. So Ancient Greek shows a much higher occurrence of temporally distant authors in the most similar pairs. There is no plausible case that the similarity between these author pairs is due simply to random variance present in all languages; as the English corpus makes clear, it is not even a necessary feature of a corpus.
These similar authors are also reasonably consistent across the other metrics considered, so when compared using other metrics in Table 5 the pairs usually appear in the top 100 pairs and in all but six cases appear in the top 10% (418) of pairs. The six pairs not in the top 10% according to Canberra distance are later epic poets compared to earlier ones, which is likely due to Canberra distance focusing too much on the relative frequencies of less frequent words. This shows that the similarity seen is not just an artifact of our Jensen-Shannon similarity metric but reflects underlying similarity between these authors picked up by other metrics like Burrow's Delta, Manhattan Similarity, and Cosine Similarity.   (-1096) 27 (43) Basil/John of Damascus 71 55 (16) 77 (-6) 37 (34) 376 (-305) Appian/Thucydides 74 46 (28) 50 (

Composition Date and Author Similarity
The previous section focused on a few key similar authors, but we now turn to answering our original question for the whole corpus: is there evidence that authors in the Ancient Greek corpus are more similar across time than authors from the English and Icelandic corpora? To evaluate this for a given corpus we plot each pair of authors as a single point, with the centuries between those authors on the x-axis and the similarity between those authors on the y-axis. By running a linear regression on this dataset, we can get a numerical value for the correlation between distance in time and similarity, i.e. a measurement of how similar authors are across time. We can then examine the measurements for each corpus and compare them. We find that temporal distance between authors explains only 3% of the variance in the Greek dataset but 58% and 39% of the variance in the English and Icelandic corpora, respectively -that is, Greek authors writing far apart in time are far more similar than authors from the English and Icelandic baselines.
When analyzing the Ancient Greek corpus, we make a few adjustments. First, we remove the Hymn to Dionysus and Euclid -the Hymn to Dionysus has only 144 tokens, and the texts of Euclid are full of geometric figures, so between the two of them they account for nearly every low-similarity outlier. Second, we remove author pairs more than nine centuries apart. Third, we color each of the points based on whether the authors are writing in the same genre or different genres; as seen in Figure 4, this helps explain a bi-modal feature of the data where authors writing in the same genre are generally more similar than authors writing in different genres.
The Greek results are visible in Figure 4. There is a downward slope in similarity as authors write further apart in time, but this is mostly the result of relative frequency of prose and poetry authors over time, and only explains 3% of the variance (R2=0.03474, F(3842)=138. 3 Figure 6 shows Icelandic author pairs. Like English, as the temporal distance between authors increases there is a clear decrease in similarity, though it accounts for only 39% of the variance rather than 58% (R2=0.3883, F(19108)= 12127.527, p<2.0E-307). There are a few pairs four to seven centuries apart that are more similar than the bulk of authors for that time, because three 19 th and 20 th century authors wrote sagas that are similar to older models, but there is still a clear downward trend over the corpus as a whole. The English and Icelandic data supports the hypothesis that English has undergone more change than Icelandic in the recent past. 43 However, the Greek texts show even less change than Icelandic over time, and far more instances of high similarity across a long time period. While we saw that century difference explains only 3% of the variance in the Ancient Greek Corpus, it explains 39-58% for English and Icelandic. The Ancient Greek Corpus shows remarkable stability even when compared to a corpus from Icelandic, a language that is culturally conservative and morphologically and orthographically standardized across time; with this context it is clear that the observed similarity between authors writing many centuries apart is a significant feature of Ancient Greek and not a given for any language. These results provide strong support for our hypothesis: it does appear that authors in the extant Ancient Greek corpus are more similar across time than authors in our baseline comparison corpora.

Conclusion
We now consider why Ancient Greek author similarity is so remarkably strong across time, in stark contrast to English and Icelandic where distance in time correlates more closely with difference between texts. The similarity between authors across long time periods was not due to a similarity in the spoken language across those time periods. Many of the later authors would have spoken the Koine (common) dialect of Greek rather than the Attic, Ionic, or Homeric dialects of the texts they imitated. When Callimachus wrote in the Homeric dialect, which combines features of Ionic and Aeolic, both dialects had been mostly replaced by the Attic-based Koine, and by the time of Aelius Aristides and his contemporaries "those who wanted to write the best Attic... clearly needed help... no one had spoken the prescribed model Attic for centuries." 44 Genre has a significant influence on the similarity of Ancient Greek texts: prose works from six centuries apart are usually more similar than a prose-poetry pair written in the same century. Because works of poetry had additional constraints compared to prose, they also had different word usage: for example, the constraints of meter made it more difficult to use the article with every noun. The observed similarity across time is likely due in part to the strength and consistency of these poetic constraints.
Another likely source of similarity is the connection between genre and dialect. In the classical period different genres were generally associated with specific dialects, so authors wrote lyric poetry in the associated dialect, literary Doric, regardless of their own native dialect. 45 Later authors, especially orators and epic poets, seem to have followed this model and intentionally written in older dialects. It is even possible that this similarity is caused by the fact that the later authors were writing in a non-native dialect of the language. If Aelius Aristides spoke Koine Greek but wanted his speeches to sound like Demosthenes, he had to learn Attic Greek from earlier authors like Demosthenes; there were no native Attic speakers to teach the language. Since he would therefore be learning Demosthenian specifically rather than Attic, it makes sense that his speeches may look more similar to Demosthenes than a generic Attic text. There is in fact evidence from ancient authors that memorizing entire texts by heart and imitating prior works were seen as ways of developing and improving one's own style. 46 It seems reasonable that memorization and imitation of an earlier author may lead to a writing style similar to that author.
This similarity may also have been driven by culture. Similarity to older models was the culturally correct way to write epic poems, speeches, high-register histories, and so on. Authors would have had a cultural incentive to do a good job at this, since a work that looked like the older models would be more well-regarded. There could also be survivor bias in the corpus: the works that survive to today are, for the most part, the best works by the best authors, according to cultural standards that viewed writing in certain styles as "the best." If the only Greek speeches from the Roman Empire preserved to the modern day are the ones that are "the best," where the best is defined as the most like speeches from Classical Athens, this could provide an explanation for the strength of this effect across the corpus.
These similarity results provide support for many existing claims of imitation, including the cases of Arrian, Aelius Aristides, and Apollonius discussed in the introduction. They may also provide a starting point for examinations of imitation by other less commonly treated authors in the Greek corpus, and with comparison to further corpora could be used to evaluate claims about the presence of imitation throughout the corpus. Similarity and potential imitation by Greek authors many centuries apart could also provide an interesting case study for ideas about imitation and language from cultural evolution, including language change and the value of multiple teachers. 47 Imitation is discussed as a tool for improving writing and style in the modern day, so an examination of the practice and impact of this phenomenon in more ancient texts would provide an interesting comparison. 48 Though we see these similarities to an unusual degree in the Ancient Greek corpus, this is likely not a capability that only Ancient Greek-speaking humans had. Modern research has shown that author signatures are not immutable: there are examples of authors varying their own signature in different works or even within a single book. 49 However, there is less modern work on how an author might adjust their signature to be more like a specific model: the ancient sources suggest copying and memorization would help, but this hypothesis has not been proven. 50 One future direction of exploration is examining mechanisms for achieving this similarity: we find no evidence that authors are copying segments of text, but we could not establish whether they are copying sentence templates. Another interesting question would be whether modern individuals could achieve this sort of similarity as well. For example, would actors who have memorized plays of Shakespeare be able to mimic the word frequencies of Shakespeare more accurately than a control group? In addition, comparison to further languages may yield additional interesting results. There are many more languages with multi-century literary traditions and the potential for imitation that could be compared to the Ancient Greek system and help clarify how much of an outlier this tradition is.
While this work suggests a variety of further areas of exploration, it shows that Ancient Greek authors of the Hellenistic and Roman periods wrote in a remarkably similar fashion to their predecessors in the classical period, at least based on their usage of common words. Further, comparisons to English and Icelandic show that this is not a natural feature of every language. Even though we approached this problem from a different direction than the usual Classics approach, we hope this, too, is instructive. As is increasingly being recognized, computational techniques can supplement and work alongside more traditional methods in Classics scholarship, providing useful context, answering questions in different ways, and opening new doors for further study of classical texts.