In their classic introduction into literary evaluation, Von Heydebrand and Winko write disapprovingly of ordinary readers who ‘usually bring their own needs and values to or into the work regardless of the text’s intention’ (186), and summarily write about a reader’s ‘heteronomous and therefore inadequate reading’ (213). Following this traditional suspicion of lay readings harboured by literary scholarship the prejudice against lay reviews (for examples see Stein; Ernst) has been widely shared. According to Bachmann-Stein, ‘Lay reviews do not orient themselves towards the standards of literary criticism. Rather, the reviewing is guided by psychological criteria referring to the impact of the individual reading experience’ (89). This distrust of online book discussion went into overdrive in Ronán McDonald’s pamphlet on The death of the critic. He saw it as a form of ‘people power’ that ‘decks out banality and uniformity in the guise of democracy and improvement’ (17).

To check whether this widely held assumption that reader reviews talk mostly about the reading experience is true, we will look at a word sketch of ‘plot’ (but might as well have discussed the writing, or the dialogues, or other issues – see below). Story or plot is a central concern in literary studies and a well-crafted plot is important for readers of all types of narrative (Boyd; Tobin). In online book reviews, the plot is one of the most frequently discussed aspects of the book (Kutzner). This is true for literary as well as for popular books (Daniels). What does a word sketch of ‘plot’ show us about how it is discussed? If the reviews are mostly about the readers’ experiences, we would expect the collocates of ‘plot’ to refer more often to the readers’ impressions, and in general to avoid technical discussion of books’ merits.

As explained above, a word sketch displays the collocates of ‘plot’ in several contexts (‘grammatical relations’, in Sketch Engine terminology). Figure 1 shows how Sketch Engine presents the adjectives most frequently associated with the word ‘plot’ in the Goodreads corpus,6 followed by the absolute frequency of the collocation and its (LogDice) typicality.7 These are words such as ‘predictable’, ‘interesting’ or ‘intriguing’ that we would indeed expect to be used with ‘plot’. Table 1 gives the full list of these adjectives. Some are mostly evaluative (‘excellent’, ‘awesome’, ‘bad’), some are mostly descriptive (‘twisty’, ‘straightforward’), others are evaluative as well as descriptive (‘predictable’, ‘compelling’, ‘confusing’). Many of the words do indeed imply a reading experience (‘intriguing’, ‘engaging’, ‘compelling’), and are necessarily subjective. But taken together, they provide a rich and balanced vocabulary for qualifying plots.

Table 2 presents nouns modified by ‘plot’ in the reviews. Some are compounds denoting aspects of plots in the books (‘plot twists’, ‘plot devices’, ‘plot direction’), some are about the discussion of plot (‘plot summary’). The algorithm is clearly imperfect: ‘plot nothing’ does not seem to make sense. Yet it is impressive that simple statistics and a few language rules can lift these patterns out of a few hundred thousand reviews. It is relevant because it shows that, at least collectively, online reviewers have a subtle and extensive vocabulary to talk about plot, even if individually they don’t use all these collocations. If they were only interested in ‘their own needs and values’ there would be no need for this extensive vocabulary. This does not imply that the majority of reviews should discuss the plot – for one thing, most reviews are too short for that.

Plot is certainly not the only aspect of books that reviewers discuss. This is shown by another entry in the ‘plot’ word sketch, the ‘Plot and/or ….’ list. Table 3 shows an edited version (there were too many irrelevant words here). These words are used by reviewers in a conjunction with ‘plot’, so they write ‘plot and writing’, ‘plot or theme’, etc. The words in this list show that online reviewers are not merely interested in plot. We could look at the Work Sketch for each of these terms and find a similarly nuanced vocabulary for discussing that aspect. For reasons of space, we can’t go into these words’ collocations, but taken together, these examples suggest that online book reviewing practices are certainly not simplistic – they are not merely based on feeling and identification – but display solid discussion on a variety of aspects related to the merits of books.8 The corpus analysis tool allows us to perceive that by zooming out and looking at the patterns that emerge when we look at thousands of reviews collectively.

Let us move from the noun ‘plot’ to an adjective, ‘beautiful’. According to an empirical study by Knoop et al., ‘beautiful’ (or rather it’s German translation ‘schön’) is the adjective that is most frequently suggested to describe an aesthetic literary experience. What does the way it is used suggest about that experience? Table 4 shows the nouns modified by ‘beautiful’ in the Goodreads corpus.

As we see, the list contains many of the words that in a book review would be expected to be qualified as beautiful (‘writing’, ‘story’, ‘cover’). But the list also contains many words that do not refer to the book as a reviewed object, but to the story world (‘woman’, ‘city’, ‘gown’).9 Some could be used in both ways (‘gift’, ‘drawing’). This presents us with a problem of ‘opinion target identification’ (Catharin and Feltrim).

Within the Sketch Engine context, there does not seem to be a way to differentiate the two domains. One option might be running a word sketch difference of the word ‘beautiful’ in subcorpora of positive and negative reviews. We could hypothesize that beautiful women occur in both high- and low-rated books, while beautiful writing probably occurs more in high-rated books. Then, in a word sketch difference of the word ‘beautiful’ between the high- and the low-rated reviews, the book domain collocates of ‘beautiful’ should appear on one end of the spectrum, the story-word collocates should be unmarked. But experiments show this doesn’t work. Beautiful scenery, beautiful cities and beautiful women occur more in positive reviews than in negative ones. Because it seems unlikely that books about beautiful scenery (or cities, or women) are rated more highly, the findings imply that in negative reviews, reviewers may see no reason to mention these aspects of the story world in their reviews. We will come back to this when we use a key word analysis to investigate differences between positive and negative reviews, below.

Of the collocates that do refer to reviewed aspects of the book, some refer to the linguistic aspects of the book, to the writing (‘writing’, ‘prose’, ‘language’), others refer to the story (‘story’, ‘ending’, ‘narrative’). If we assume with Jakobson that the poetic function is the essence of literary art, and distinguish distanced readers (who focus on form) from identifying readers (who focus on character and plot), it seems likely that the form-oriented words appear mostly with literary books, the story-oriented words with suspense-driven books. We can test that hypothesis by creating a word sketch difference by genre (Heydebrand and Winko). For that, we move to the Dutch corpus, as there we have subcorpora based on genre.10 We create in Sketch Engine a word sketch difference for the Dutch word ‘prachtig’ (one possible translation of English ‘beautiful’) between the literary and the suspense subcorpus. Sketch Engine shows the output as displayed in Figure 2. The words in green occur typically in reviews about literary books, the words in red occur typically with suspense novels. The numbers after the words give the frequency and (LogDice) typicality in the respective subcorpora.

The strongest indications for differences in how ‘prachtig’ is used for evaluating the two genres, can be seen in the columns ‘“prachtig” and/or …’ and ‘subjects of “be prachtig”’. Table 5 gives the translations of the Dutch words in Figure 2. Except for ‘house’, all words come from the reviewing domain, not the domain of the story world. In the subjects-of column we see that what is described as beautiful in suspense reviews are character and plot11 while in the literature reviews, the first four collocates refer to style. Still, the combination of ‘beautiful’ and ‘story’ also occurs more typically in literary reviews than in suspense reviews. Moving to the and/or column, we see that close associates of ‘beautiful’ in the suspense genre are ‘horrible’ and ‘oppressive’, words that are clearly connected to the effects of the story. In the literary reviews, ‘poetic’ and to a lesser extent ‘impressive’ seem clearly related to style; the other three words are all (near-synonyms of) ‘moving’. While stylistic aspects undoubtedly are partly responsible for a text being moving,12 what moves us are the characters and what happens to them.

The modified-by column contributes further support to this discussion, ‘story’ again appearing as a collocate of ‘beautiful’ in the literary reviews, and ‘house’ as an element of the story world coming up in the suspense category, which might indicate that the setting of the story world be important to this type of reader. Summarizing, we could say that what suspense readers consider beautiful when evaluating a book are plot and character (and the cover); what literary readers consider beautiful is style and story. This seems to confirm the conclusion of Riddell and Van Dalen-Oskam that there is an overlap between distanced and identifying readers: distanced readers are identifying readers with more reading techniques at their disposal (with the corollary that all readers are identifying readers).

That reading is an embodied activity is widely accepted among researchers:13 ‘[P]eople imaginatively project themselves into text worlds via embodied simulations,’ writes Raymond Gibbs (222). If this is true, we should see a reflection of that in the review texts.

In this connection, it is interesting to look once more at the word sketch for ‘beautiful’. The top modifiers of beautiful (in LogDice order) are given in Table 6. While some of these modifiers are neutral, many have a clearly corporeal semantic component.14 While for ‘heartbreakingly’ or ‘painfully’ the reference to bodily perceptions is quite explicit, we would like to suggest an embodied dimension also for ‘surprisingly’ and its stronger variants ‘amazingly’ and ‘stunningly’, for ‘hauntingly’ and others (C. R. Miller). It appears that bodily simulation in processing and understanding a fictional text, as described by Gibbs, transpires to the evaluation of that text.

To look more systematically into the role of embodiment, we would like to search for references to body parts. While Sketch Engine does not know which words refer to body parts, we can find many examples of the body’s involvement in reading thanks to the fact that ‘you’ in book reviews usually refers to the reader. A search for ‘your’ followed by a noun brings up many relevant phrases. We can do this in Sketch Engine using the CQL option in the concordance tool, and then creating a frequency graph for the noun. The top results are shown in Figure 3. The most frequently occurring word in Figure 3 is ‘heart’. The context menu allows us to jump to a concordance of the phrase ‘your heart’. Then we find phrases from reviews such as ‘the book is going to break your heart (in the best way possible)’, ‘a great historical romance to warm your heart’, ‘deep moments, the ones that rip your heart out’, or ‘has your heart ever exploded with so much joy?’

Table 7 shows some verbs and expressions occurring with ‘your’ + body part. Some of these are literal expressions, others are metaphorical. That a book ‘brings a smile on your face’ or ‘tears to your eyes’, or that you ‘shake your head’ over a character’s action – all these are literal expressions about bodily response to reading. However, most of the expressions in Table 7 involve the body in a metaphorical sense. With Gibbs and Matlock we assume that people might ‘understand metaphors by creating an imaginative simulation of their bodies in action that mimics the events alluded to by the metaphor.’ Expressions stating that a book makes you ‘open your heart’, that it ‘blew off your head’, that it ‘hits you in the face’ or ‘gets under your skin’ evoke bodily metaphors for what reading does to the reader.

Thus, the role of the body in these expressions is in most cases distinct from the bodily involvement in comprehending and experiencing (metaphors in) fictional language, which is the focus of Gibbs. They are rather comparable to embodied metaphors used for the process of reading, such as ‘reading is a journey’ or ‘reading is eating’ (Nuttall and Harrison; Ross; Herrmann and Messerli). What we see here is another way in which the body is likely to be inextricably involved in reading, as documented by lexis referring to body parts and processes. It is interesting to note that almost all of these expressions are used to describe positive reading experiences.

Research into reading has often focussed on the good things that reading can result in: empathy, self-reflection, an interest in others (Koopman and Hakemulder). But not all reading experiences are positive and not all people like all books. We can distinguish the positive from the negative reviews on the basis of the accompanying rating (number of stars). But what are the textual differences between positive and negative reviews? It stands to reason that they can be distinguished based on positive and negatively valenced words.15 But what other differences are there? Taboada and others look at words and constructions that can be used to convey negativity and illustrate those, among others, with quotes from movie and book reviews. Among the negative constructions they mention are rhetorical questions, sarcasm, certain suffixes (‘-let’), certain words (‘actually’), direct quotations, the juxtaposition of a very positive intensifier (such as ‘hilariously’) with a negative adjective (‘bad’), and why-questions. Crossley and others use LIWC16 and their own sentiment analysis tool (SEANCE) to compare positive and negative reviews. They find that negative reviews use more negative emotions (particularly anger-related terms), negations, exclusion terms, spatial terms, understanding and certainty (as defined by SEANCE). Positive reviews use more positive emotion terms, social and certainty (as defined by LIWC) terms, as well as terms related to power and respect. They also note that certain POS-tags are useful for discriminating positive from negative reviews.

We will use a keyword analysis to investigate the characteristics of negative vs. positive reviews, complemented by a brief analysis using LIWC. Keyword (or keyness) analysis renders the statistically distinctive words of a focus corpus as compared with a reference corpus (Bondi and Scott). Rather than examining content, we will initially look at function words and POS-tags. We use our subcorpora defined by rating: a subcorpus pos, containing all reviews with rating four or five, and a subcorpus neg containing ratings one, two and three.17 The reason for including three among the negative reviews is partly practical, as the distribution of ratings is heavily skewed towards the higher ratings. In the English corpus, the ratings 1 and 2 together account for less than 10 percent of the reviews, while the ratings 4 and 5 account for 74 percent. But that also means that a 3 represents a distinctly below-average reading experience. A book with rating 3 is typically described as ‘okay’ or ‘decent’, but generally still something of a disappointment.

Figure 4 shows the top 20 keywords defined by comparing the positive versus the negative English subcorpora.18 The Sketch Engine keyword tool uses a parameter that has the effect of displaying either the rarer or the more common words in the keyword list. That parameter was set at its highest value (1,000,000) to select the most common words. Even so, we notice that, due to the nature of the corpus, some content words appear among the top twenty keywords (‘read’, ‘great’ and others). We will only look at the function words and notice that ‘you’, ‘he’ (and third-person possessive pronouns) as well as ‘a’ occur more frequently in positive reviews. Interestingly, when we reverse the comparison and look at keywords in negative versus positive reviews (results not shown here) we find ‘I’ (the other singular personal pronoun) and ‘the’ (as well as ‘not’, as predicted by Crossley et al.).

Before trying to answer why these words occur in a positive or negative context, we will step up the level of abstraction and look at key POS-tags. Figure 5 shows the key POS-tags in the positive vs. the negative corpus.19 There is much to note here, for example that third-person singular present-tense verbs occur more in positive reviews (tags VHZ (‘has’), VBZ (‘is’) and VVZ for other verbs), as do nouns (NN) and proper nouns (NP). In the reverse comparison (not shown here) we observe that past tense forms are more frequent in the negative reviews. Table 8 summarizes the observed differences.

An explanation of these differences begins with the observation that most NPs in the review texts refer to characters in the books, and only to a lesser degree to authors, places and book titles. One explanation would be that positive reviews more often than negative reviews recount part of the story. That explains the proper nouns, the use of ‘he’, the third person verbs, and also the present tense, as reviewers generally use the present tense to report on the story. A brief inspection shows that indeed most occurrences of NPs in the review texts are used to describe events from the story, with description or praise of authors as second most important category. But a systematic annotation effort would be necessary to validate this impression.

This hypothesis is also consistent with the observed difference between the usage of the determiners ‘a’ (mostly with positive reviews) vs. ‘the’. In general, ‘a’ is used to introduce new subjects into a text, while ‘the’ is used to refer to subjects already known. Table 9 shows nouns occurring after the definite or indefinite article, ordered by the fraction of occurrences after ‘a(n)’. The table is split in two halves: Words above the bolded bar (under ‘relationship’) occur disproportionately more with ‘the’, the words below co-occur significantly more often with ‘a(n)’. The words below the bar include the fixed expressions ‘a bit’ and ‘a lot’, but also general nouns that you would need to (begin to) recount a plot: ‘a man’, ‘a girl’, and many others. So, it makes sense that ‘a(n)’ should occur more in positive reviews. The words above the bar, especially the highly frequent ones, refer to objects that in the context of the review do not need to be introduced, and will therefore usually have ‘the’: ‘the book’, ‘the story’, ‘the end’, ‘the author’.

If positive reviews more often recount events from the story, is there also something that negative reviews do more? We note that in our corpus, the negative reviews are on average longer than the positive reviews (mean of 123 compared to a mean of 106 words). What do they use these words for? One possibility is that negative reviewers feel more need to explain why they didn’t like a book. An exploratory analysis using LIWC showed that negative reviews score significantly higher in all cognitive aspects (insight, cause, discrepancies, tentative language and differences), except certainty. These explanations may be regarded as a reviewer’s description of their individual cognitive and emotional engagement in the reading experience, which may explain the higher use of ‘I’ in the negative reviews. The results also suggest that explanations predominantly use the past tense, which could explain the predominance of the past tense in negative reviews.

This is only the beginning of a more extensive analysis: there are many other aspects of these keyword lists that merit further investigation. Why does, for instance, ‘and’ occur more in positive reviews? An explanation could be that positive reviews use more groups of three words to describe (especially) characters: ‘strong and determined and feisty’, ‘pretty and quirky and cool’ or ‘lost and lonely and unsure’. Another finding is that question marks are a very good predictor of negative reviews (and, confirming the findings of Taboada et al., these questions are indeed often sarcastic or rhetorical). We cannot go into these and other phenomena for reasons of space.