Compositionality and Bilingualism

Last week I put up a link to an online experiment.  Here’s the results! You can still do the experiment first, if you like, here.  Source code and raw results at the bottom.

Languages evolve over time under a pressure to be learned by a new generation.  Does learning two languages at once effect this pressure? My experiment says … maybe.

These pressures include ones for learnability (compression) and expressivity (able to express a large variety of meanings, Kirby, Cornish & Smith, 2008).  Bilingualism seems like an unlikely ability since learning an extra language leaves the speaker potentially no more expressive at a cost of an increase in the amount of effort required to learn it.  There is no pressure for one language structure (e.g. English) to adapt to another language (e.g. Mandarin) so that they can become optimally learnable and expressive as a single medium.  That is, there’s no reason to assume that expressivity and learnability pressures apply across languages (which are not being used by the same people).

Nevertheless, children display an aptitude and a willingness to learn and use multiple languages simultaneously, and at a similar rate to monolingual children.  Therefore, languages do seem to have adapted to be learnable simultaneously.  Does the compatibility of languages point to a strong innate property of language?  In contrast, it might point to underlying similarity in the structure of languages, brought about by universal principles of communication.

I suggest that the fact that learning two languages is not twice as difficult as learning one is partially to do with the compositional nature of langauge (the meaning of sentences is made up of sub-units of the sentence).  The alternative is a holistic langauge with a separate word for each meaning.  Compositionality allows language to be compressible and generalisable, and so easier to learn.  Imagine a 3 x 3 x 3 meaning space, like the one from Kirby, Cornish & Smith (2008), below (each word represents a single meaning, e.g ‘renana’ is a red square moving in a straight line):

Meaning space in Kirby, Cornish & Smith (2007) with words learned by a participant in generation 10 of the iterated chain.

The meaning space has 27 meanings, but in theory the whole space can be described with 9 words:  3 words denoting shape, 3 denoting colour and 3 denoting movement.  Additionally, compositionality allows ‘codeswitching’:  You can use sub-parts from either language interchangeably and still get the message through (unless the languages use similar sub-units for different meanings).  Learning two compositional languages simultaneously, therefore, may be easier than learning two holistic languages.

Experiment

I decided to test this by getting people to learn either two compositional or two holistic languages simultaneously.  I used a 2 x 2 meaning space which meant that the number of holistic words needed to describe all meanings and the number of compositional units were equal (4).  Here’s the four meanings:

Next, I created four ‘languages’ to create two conditions where each meaning had two labels.  The compositional condition had two compositional ‘languages’ (below left) and the holistic condition had two holistic languages (below right):

          

Each word was a CVCV nonsense word.  I tried to balance the distributions of letters to be equal across the conditions, but I see now that it could have been better.

I recruited 16 people through Replicated Typo to take part in the experiment online.   Participants were told that they would be doing a langauge learning experiment where each object had two words.  Participants were exposed to each label-meaning pair in their condition (8 words).  They were then shown each meaning and asked to recall both words associated with it.  Each participant went through 5 rounds of training and testing.

Results

I used two measures of how well each participant learned the words at each stage:  Total correct responses and total (normalised) Levenshtein distance.  The Levenshtein distance was calculated as the minimum number of edits that would be required to turn the participant’s response into the correct responses (word by word, expressed as a number between 0 = all perfect and 8 = all exactly wrong).

However, the compositional languages have an extra ‘codeswitching’ edge.  Participants may have been learning the sub-units correctly, but forgetting which sub-unit belonged to which ‘language’ (e.g. responding with ‘lupu’ for a red triangle).  Therefore, I also took the maximum correct responses and minimum Levenshtein distances for all responses where responses for the same object were spliced.  For example, if the participant responded with ‘lupu’ and ‘haka’ for the red triangle (instead of the correct ‘luka’ and ‘hapu’), then this would count as correct under the ‘compositional adjustment’.

The results for correct responses (without compositional adjustment) showed a main effect of test block (participants got better over time, see table below generated automatically in R with xtable), and condition, with the holistic group doing better on average in each test block than the compositional group (mean for compositional group = 3.6 words, mean for holistic group= 4.6).  There was no significant interaction between test block and condition.  Using the compositional adjustment score did not yield qualitatively different results.

ANOVA for number of correct responses

Df Sum Sq Mean Sq F value Pr(> F)
Test Block 1 121.60 121.60 28.79 < 0.0001 *
Condition 1 22.85 22.85 5.41 0.0223 *
Block x Condition 1 2.11 2.11 0.50 0.4814
Residuals 91 384.39 4.22

 ANOVA for number of correct responses with compositional adjustment

Df Sum Sq Mean Sq F value Pr(> F)
Test Block 1 121.60 121.60 29.08  < 0.0001 *
Condition 1 18.43 18.43 4.41 0.0385 *
Block x Condition 1 2.11 2.11 0.50 0.4792
Residuals 91 380.49 4.18

>
For the Levenshtein distance, there was a main effect of test block, but not condition.  With this measure, the groups are a lot closer.

ANOVA for Levenshtein distance

Df Sum Sq Mean Sq F value Pr(> F)
Test Block 1 83.23 83.23 49.74 < 0.0001 *
Condition 1 2.29 2.29 1.37 0.2452
Block x Condition 1 0.34 0.34 0.20 0.6555
Residuals 91 152.25 1.67

ANOVA for Levenshtein distance with compositional adjustment

Df Sum Sq Mean Sq F value Pr(> F)
Test Block 1 78.98 78.98 51.48  < 0.0001 *
Condition 1 1.50 1.50 0.98 0.3247
Block x Condition 1 0.23 0.23 0.15 0.7019
Residuals 91 139.61 1.53

Discussion

Contrary to the expectation that compositional languages are always equally easy or easier to learn than holistic languages, there’s a tantalising possibility that holistic languages (at least very small ones) are easier to learn simultaneously.  This might mean that the pressure for a langauge to become compositional might be less in a bilingual community.  At the least, the differences between the groups might suggest that they took different approaches to learning the two types of language.  They did not differ on the Levenshtein score suggesting that, in a real communicative situation, there may not have been such a big difference between the groups.

Doing this study online limits some options.  I would have liked more participants and to have more training and test rounds.  But there’s only so much you can ask people to do.  Perhaps feedback during testing would be a good idea, too.  I’d like to do this experiment properly with the traditional people-in-a-lab setup, and run people on a single langauge for a control, too.

Download

You can view the html and javascript source for running the experiment here and download the results here : scores , words.

References

Kirby, S., Cornish, H., & Smith, K. (2008). Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language Proceedings of the National Academy of Sciences, 105 (31), 10681-10686 DOI: 10.1073/pnas.0707835105

  • http://theadventuresofauck.blogspot.com/ Sean

    When I first wrote this post, the effect of condition was not significant, but then some more people did the experiment and has given me a significant result! I’ve changed the details above.

  • Caravelle

    One thing that changed my view of language was a study on spelling and how people remember it. The two options were whether they learned a rule and then applied it, or used a frequentist approach where they were exposed to tons of cases and went with the most frequent regularities.

    I would have thought that a rules-based approach was a no-brainer, it’s so much more economical in memory if nothing else, but that study suggested that people use a frequentist approach.

    Which isn’t that nonsensical is one considers on the one hand that the human brain has lots of memory space but needs to be fast, or on the other hand that neuronal networks do well assigning different weights to outputs depending on the input without having to “remember” each input individually.

    I’m not sure how related this random comment is to your study (which I participated in, I was doing the holistic language apparently…), but it might suggest that what we intuitively think is “simpler” or “more economical” (like compositional structures) isn’t necessarily what our brain actually goes for.

    And I noticed this : “Bilingualism seems like an unlikely ability since learning an extra language leaves the speaker potentially no more expressive at a cost of an increase in the amount of effort required to learn it. There is no pressure for one language structure (e.g. English) to adapt to another language (e.g. Mandarin) so that they can become optimally learnable and expressive as a single medium. That is, there’s no reason to assume that expressivity and learnability pressures apply across languages (which are not being used by the same people).”

    Is that a valid assumption though ? Wouldn’t there be an advantage in individuals (which are “carriers” of a language) to be able to learn other languages, especially in cases where exchanges with other groups occurs ? I don’t know how much exchanges occurred in the past; I realize that it used to be a lot less than now, but then languages used to be a lot more diverse on smaller scales than they are now too.

    Other suggestion : I’ve learned both Japanese and German, and I remember at first being able to learn new words at a brisk pace, and then I got to the point where it got really difficult because *all new words looked the same*. (especially Japanese. Srsly, wtf ?). That might suggest a reason compositional languages didn’t do as well : instead of going “hey, all those words follow a pattern”, people’s brains are going “ALL THOSE WORDS CONTAIN THE SAME ELEMENTS, HOW AM I SUPPOSED TO TELL THEM APART ???”

  • http://theadventuresofauck.blogspot.com/ Sean

    Statement about bilingualism being an unlikely ability is a caricature of a position which I think is held (maybe unconsciously) by some researchers. There is a paper that argues that second language learning would not have been adaptive in an early human environment (Hagen, 2008), but I have a few problems with it (see here).

    With regards to your other suggestion: Yes, I think that’s what’s happening in this experiment. Although the compositional language is easier to describe, it’s also easier to get confused because strings are similar. This is the kind of point that I was making: Learning two languages simultaneously can change their ‘fitness’ (how easy they are to learn).