Understanding the meaning of compound words:
A compositional perspective grounded in linguistic and visual experience
Marco Marelli, University of Milano-Bicocca
Since the seminal LSA proposal (Landauer & Dumais, 1997) distributional semantics has provided efficient data-driven models of the human semantic system, representing word meaning through vectors recording lexical co-occurrences in large text corpora. However, these approaches generate static descriptions of the semantic system, falling short of capturing the highly dynamical interactions occurring at the meaning level during language processing.
In the present work, I discuss the CAOSS model (Compounding as Abstract Operations in Semantic Space), a first step in this direction that moves from distributional semantics to capture the meaning of compound words (Marelli et al., 2017).
In CAOSS, word meanings are represented as vectors encoding lexical co-occurrences in a reference corpus (e.g., the meaning of “snow” will be based on how often “snow” appears with the other words), according to the tenets of distributional semantics. A compositional procedure is induced as a weighted sum: given two vectors (constituent words) u and v, their composed representation (the compound) can be computed as c=M*u+H*v, where M and H are weight matrices estimated from corpus examples. The matrices are trained using least squares regression, having the vectors of the constituents as independent words (“car” and “wash”, “rail” and ”way”) as inputs and the vectors of example compounds (“carwash”, “railway”) as outputs, so that the similarity between M*u+H*v and c is maximized. In other words, the matrices are defined in order to recreate the compound examples as accurately as possible. Once the two weight matrices are estimated, they can be applied to any word pair in order to obtain meaning representations for untrained word combinations (e.g., “snow building”).
In a series of behavioral experiments, model predictions were tested against psycholinguistic data. CAOSS is shown to mirror evidence related to the processing of novel compounds (Marelli et al., 2017; Günther & Marelli, 2020), and in particular the impact of relational information (Gagné, 2001; Gagné & Shoben, 2007) as well as the “morpheme interference effect” (Crepaldi et al., 2010). Moreover, CAOSS also provides a central contribution to the understanding of semantic transparency in familiar compounds: CAOSS estimates are shown to best characterize the transparency impact in word processing (Günther & Marelli, 2019). Finally, I discuss how CAOSS is not to be considered a “disembodied model”, since one can easily ground it in perception by feeding it images together with text data (Günther et al., 2020).
The model simulations indicate that compositionality-related phenomena are reflected in language statistics. Human speakers are able to learn these aspects from language experience and automatically apply them to the processing of any word combination. The present model is flexible enough to emulate this procedure, predicting sensible relational similarities for novel compounds and correctly capturing the contribution to semantic transparency provided by compositional operations. The model is also shown to generalize to other kind of data, being able to capture the contribution of perceptual experience in the internal dynamics of compound-word processing. Such evidence directly links linguistic composition to conceptual combination, speaking for the possible role of general-level learning procedures at the foundations of both phenomena.
Reference list
Crepaldi, D., Rastle, K., & Davis, C. J. (2010). Morphemes in their place: Evidence for position-specific identification of suffixes. Memory & cognition, 38(3), 312-321.
Gagné, C. L. (2001). Relation and lexical priming during the interpretation of noun–noun combinations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27(1), 236.
Gagné, C. L., & Shoben, E. J. (1997). Influence of thematic relations on the comprehension of modifier–noun combinations. Journal of experimental psychology: Learning, memory, and cognition, 23(1), 71.
Günther, F., & Marelli, M. (2019). Enter sandman: Compound processing and semantic transparency in a compositional perspective. Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(10), 1872.
Günther, F., & Marelli, M. (2020). Trying to make it work: Compositional effects in the processing of compound “nonwords”. Quarterly Journal of Experimental Psychology, 73(7), 1082-1091.
Günther, F., Petilli, M. A., & Marelli, M. (2020). Semantic transparency is not invisibility: A computational model of perceptually-grounded conceptual combination in word processing. Journal of Memory and Language, 112, 104104.
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological review, 104(2), 211.
Marelli, M., Gagné, C. L., & Spalding, T. L. (2017). Compounding as Abstract Operation in Semantic Space: Investigating relational effects through a large-scale, data-driven computational model. Cognition, 166, 207-224.
234 total views, 2 views today
This post is also available in: 
Hrvatski (Croatian)
