Contextualized rating reduces concreteness of nouns
Anita Peti-Stantić, Mateusz-Milan Stanojević, and Mirjana Tonković
Psycholinguistic studies generally collect concreteness ratings by asking participants to rate words in isolation. Although such ratings are relatively stable and comparable across languages, they are vulnerable to theoretical and methodological criticism (e.g., Löhr 2021). Possible issues include, for instance, the treatment of polysemous or homonymous words (it is unclear which sense is rated if a word appears in isolation) and the overall ecological validity of the ratings because words are never used in isolation. However, rating words in naturalistic contextual frames is challenging because how words collocate with others is precisely what makes them different (e.g., Harris 1954). In other words, it is difficult to come up with natural contextual frames that will work for a variety of words so as to ensure comparability as well as ecological validity. In this paper we explore one such frame for several groups of nouns in order to find out how ratings of words in isolation differ from ratings of words in context, and whether manipulating the concreteness of the context may lead to different concreteness ratings.
We performed a rating study with 24 Croatian nouns (six each ending with the suffixes –nik, –nje, –ost and –stvo). These groups of nouns were selected because there were significant differences between the concreteness of nouns with these endings when they were judged in isolation based on ratings available in the Croatian Psycholinguistic Database (CPD; Peti-Stantić et al. 2019), with –nik nouns being typically concrete (e.g. učenik ‘student’), and the other groups being typically more abstract (e.g. nastojanje ‘endeavor’, darovitost ‘giftedness’, jedinstvo ‘unity’). Each noun was combined into the predicative contextual frame Noun is Adjective with three concrete and three abstract adjectives (rated above and below 3.5 in the CPD, respectively). This resulted in 144 combinations such as Prolaznik je malen ‘The passer-by is small’ and Prolaznik je neovisan ‘The passer-by is independent’. The sentences, together with 24 fillers were rated by altogether 86 participants, students at the University of Zagreb, using SurveyMonkey. The participants were instructed to rate the first word in each contextual frame. Each participant rated 48 sentences (half of them were fillers). In the end, each noun was rated by at least 42 participants both in the abstract and concrete context.
The results show the main effect of the group of nouns. The pattern of results parallels the one that was obtained between these groups in isolation (nouns ending with -nik were rated as the most concrete). There was a main effect of adjective type (nouns were rated as more concrete when paired with concrete vs. abstract adjective) and no interaction of noun and adjective group was found.
Finally, there was a significant difference between the ratings of nouns obtained in isolation, combined with concrete adjectives and with abstract adjectives. Nouns in isolation were rated as the highest on concreteness, followed by contextualized combinations with concrete adjectives, and followed by combinations with abstract adjectives.
Overall, these results suggest that contextualization does influence concreteness ratings of nouns, with a more concrete adjectival context leading to higher concreteness rating. Furthermore, ratings being lower in context than in isolation suggest that judgments obtained in isolation require participants to fall back on an idealized knowledge of a word, which seems to be related to the conceptual and morphological characteristics of a word (hence the differences between groups of nouns with different endings when rated in isolation). Rating words in context does not abolish the effects of these characteristics, but does seem to require participants to relegate part of their idealized knowledge to the background, and take into account the context at hand (even when instructed to rate only the first word). This is why contextualized concreteness ratings are lower than ratings in isolation. Thus, contextualized ratings may bring distribution into play, which seems less important when ratings are done in isolation. This assumption is in line with a recent comparison of human ratings to ratings obtained by machine learning, where it was found that the machine learning model – presumably because of its use of distributional evidence – consistently underpredicted the concreteness of nouns (Peti-Stantić et al. 2021). In other words, by relying on distributional evidence, the model predicted “less idealized” ratings than humans, which seems to be the case for humans as well. This hypothesis still remains to be tested.
References
Harris, Zellig S. 1954. “Distributional Structure.” WORD 10 (2–3): 146–62. https://doi.org/10.1080/00437956.1954.11659520.
Löhr, Guido. 2021. “What Are Abstract Concepts? On Lexical Ambiguity and Concreteness Ratings.” Review of Philosophy and Psychology, March. https://doi.org/10.1007/s13164-021-00542-9.
Peti-Stantić, Anita, Mateusz-Milan Stanojević, Maja Anđel, Mirjana Tonković, Gordana Keresteš, Nikola Ljubešić, Jana Willer Gold, Jelena Tušek, Irina Masnikosa, and Vedrana Gnjidić. 2019. “Hrvatska psiholingvistička baza.” Modeliranje mentalne gramatike hrvatskoga: ograničenja informacijske strukture. https://doi.org/10.17234/megahr.2019.hpb.
Peti-Stantić, Anita, Maja Anđel, Vedrana Gnjidić, Gordana Keresteš, Nikola Ljubešić, Irina Masnikosa, Mirjana Tonković, Jelena Tušek, Jana Willer-Gold, and Mateusz-Milan Stanojević. 2021. “The Croatian Psycholinguistic Database: Estimates for 6000 Nouns, Verbs, Adjectives and Adverbs.” Behavior Research Methods, April. https://doi.org/10.3758/s13428-020-01533-x.
180 total views, 3 views today
This post is also available in: 
Hrvatski (Croatian)
