Top-down vs bottom-up approaches to cognition: Griffiths vs McClelland

There is a battle about to commence.  A battle in the world of cognitive modelling.  Or at least a bit of a skirmish.  Two articles to be published in Trends in Cognitive Sciences debate the merits of approaching cognition from different ends of the microscope.

Structured Probabilistic Thingamy

On the side of probabilistic modelling we have Thom Griffiths, Nick Chater, Charles Kemp, Amy Perfors and Joshua Tenenbaum.  Representing (perhaps non-symbolically) emergentist approaches are James McClelland, Matthew Botvinick, David Noelle, David Plaut, Timothy Rogers, Mark Seidenberg and Linda B. Smith.  This contest is not short of heavyweights.

However, the first battleground seems to be who can come up with the most complicated diagram.  I leave this decision to the reader (see first two images).

The central issue is which approach is the most productive for explaining phenomena in cognition.  David Marr’s levels of explanation include the ‘computational’ characterisation of the problem, an ‘algorithmic’ description of the problem and an ‘implementational’ explanation which focusses on how the task is actually implemented by real brains.  Structured probabilistic takes a ‘top-down’ approach while Emergentism takes a ‘bottom-up’ approach.

Continue reading “Top-down vs bottom-up approaches to cognition: Griffiths vs McClelland”

“Xanadu” Revisited (Culturomics?)

Google has just released an interesting dataset. Geoff Nunberg describes it at Language Log:

Culled from the Google Books collection, it contains more than 5 million books published between 1800 and 2000 — at a rough estimate, 4 percent of all the books ever published — of which two-thirds are in English and the others distributed among French, German, Spanish, Chinese, Russian, and Hebrew. (The English corpus alone contains some 360 billion words, dwarfing better structured data collections like the corpora of historical and contemporary American English at BYU, which top out at a paltry 400 million words each.)

It is, he says, “the largest corpus ever assembled for humanities and social science research.” The New York Times has reported on it and there’s an article in Science based on it.

You can also play around with it online with the Google Books Ngram Viewer. You enter individual words or phrases (up to five words long) and a Google graphs their frequency over time. I’ve spent a little time playing around with it.

In particular, I’m interested in the proper noun, “Xanadu.” As you may know, it’s the name of Kubla Khan’s summer capital and is also the second word in Coleridge’s most famous poem, “Kubla Khan.” Several years ago I did a Google search on “Xanadu” and was surprised to come up with over two-million hits. How’d that happen? I wondered.

I ended up writing a long post on The Valve, which generated an interesting discussion, and then distilling that down into a tech report. You can download the report here (One Candle, a Thousand Points of Light: The Xanadu Meme). Here’s the abstract:

I treat a single word ‘xanadu’, as a ‘meme’ and follow it from a 17th century book, to a 19th century poem (Coleridge’s “Kubla Khan”), into the 20th century where it was picked up by a classic movie (“Citizen Kane”), an ongoing software development project (Ted Nelson’s Project Xanadu), and another movie and hit song, Olivia Newton-John’s Xanadu. The aggregate result can be seen when you google the word, you get 6 million hits. What is interesting about those hits is that, while some of them are directly related to Coleridge’s poem, more seem to be related to Nelson’s software project, Olivia Newton-John’s film and song, and (indirectly) to Welles’ movie. Thus one cluster of Xanadu sites is high tech while another is about luxury and excess (and then there’s the Manchester Swingers Club Xanadu).

Continue reading ““Xanadu” Revisited (Culturomics?)”

Fun Language Experiment: Results

Two days ago I ran a pilot experiment online from Replicated Typo.  Thanks to all who took part. It’s a bit cheeky to exploit our readers, but it’s all in the name of science.  Unfortunately, the pilot was a complete failure. Suggestions and comments are welcome.

The experiment was into the role of variation in language learning.  Here’s what I was up to (plus the source code for running similar experiments):

Continue reading “Fun Language Experiment: Results”

Fungus, -i. 2nd Decl. N. Masculine – or is it?: On Gender

ResearchBlogging.orgIn an attempt to write out my thoughts for others instead of continually building them up in saved stickies, folders full of .pdfs, and hastily scribbled lecture notes, as if waiting for the spontaneous incarnation of what looks increasingly like a dissertation, I’m going to give a glimpse today of what I’ve been looking into recently. (Full disclosure: I am not a biologist, and was told specifically by my High School teacher that it would be best if I didn’t do another science class. Also, I liked Latin too much, which explains the title.)

It all started, really, with trying to get my flatmate Jamie into research blogging. His intended career path is mycology, where there are apparently fewer posts available for graduate study than in Old English syntax. As he was setting up the since-neglected Fungi Imperfecti, he pointed this article out to me: A Fungus Walks Into A Singles Bar. The post explains briefly how fungi have a very complicated sexual reproduction system.

Fungi are eukaryotes, the same as all other complex organisms with complicated cell structures. However, they are in their own kingdom, for a variety of reasons. Fungi are not the same as mushrooms, which are only the fruiting bodies of certain fungi. Their reproductive mechanisms is rather unexpectedly complex, in that the normal conventions of sex do not apply. Not all fungi reproduce sexually, and many are isogamous, meaning that their gametes look the same and differ only in certain alleles in certain areas called mating-type regions. Some fungi only have two mating types, which would give the illusion of being like animal genders. However, others, like Schizophyllum commune, have over ten thousand (although these interact in an odd way, such that they’re only productive if the mating regions are highly compatible (Uyenoyama 2005)).

Some fungi are homothallic, meaning that self-mating and reproduction is possible. This means that a spore has within it two dissimilar nuclei, ready to mate – the button mushroom apparently does this (yes, the kind you buy in a supermarket.) Heterothallic fungi, on the other hand, merely needs to find another fungi that isn’t the same mating type – which is pretty easy, if there are hundreds of options. Other types of fungi can’t reproduce together, but can vegetatively blend together to share resources, interestingly enough. Think of mind-melding, like Spock. Alternatively, think of mycelia fusing together to share resources.

In short, the system is ridiculously confusing, and not at all like the simple bipolar genders of, say, humans (if we take the canonical view of human gender, meaning only two.) I’m still trying to find adequate research on the origins of this sort of system. Understandably, it’s difficult. Mycologists agree:

“The molecular genetical studies of the past ten years have revealed a genetic fluidity in fungi that could never have been imagined. Transposons and other mobile elements can switch the mating types of fungi and cause chromosonal rearrangements.Deletions of mitochondrial genes can accumulate as either symptomless plasmids or as disruptive elements leading to cellular senescence…[in summary,] many aspects of the genetic fluidity of fungi remain to be resolved, and probably many more remain to be discovered.” (Deacon, 1997: pg. 157)

At this point you’re probably asking why I’ve posted this here. Well, perhaps understandably, I started drawing comparisons between mycologic mating types and linguistic agreement immediately. First, mating-type isn’t limited to bipolarity – neither is grammatical gender. Nearly 10% of the 257 languages noted for number of genders on WALS have more than five genders. Ngan’gityemerri seems to be winning, with 15 different genders (Reid, 1997). Gender distinctions generally have to do with a semantic core – one which need not be based on sex, either, but can cover categories like animacy. Gender can normally be diagnosed by agreement marking, which, taking out genetic analysis of the parent, could be analogic to fungi offspring. Gender can be a fluid system, susceptible to decay, mostly through attrition, but also to reformation and realignment – the same is true of mating types. (For more, see Corbett, 1991)

As with all biologic to linguistic analogues, the connections are a bit tenuous. I’ve been researching fungal replication partly for the sake of dispelling the old “that’s too complex to have evolved” argument, which is probably the most fun point to argue against creationists with. However, I’ve mostly been doing this because fungi and linguistic gender distinctions are just so damn interesting.

If anyone likes, I’ll keep you updated on mycologic evolution and the linguistic analogues I can tentatively draw. For now, though, I’ve really got to get back to studying for my examination in two days. Which means I’ve got to stop thinking about a future post involving detailing why “Prokaryotic evolution and the tree of life are two different things” (Baptiste et al., 2009)…

References:

  • Corbett, G. Gender. Cambridge University Press, Cambridge: 1991.
  • Deacon, JW. Modern Mycology. Blackwell Science, Oxford: 1997.
  • Reid, Nicholas. and Harvey, Mark David,  Nominal classification in aboriginal Australia / edited by Mark Harvey, Nicholas Reid John Benjamins Pub., Philadelphia, PA :  1997.

Uyenoyama, M. (2004). Evolution under tight linkage to mating type New Phytologist, 165 (1), 63-70 DOI: 10.1111/j.1469-8137.2004.01246.x
Bapteste E, O’Malley MA, Beiko RG, Ereshefsky M, Gogarten JP, Franklin-Hall L, Lapointe FJ, Dupré J, Dagan T, Boucher Y, & Martin W (2009). Prokaryotic evolution and the tree of life are two different things. Biology direct, 4 PMID: 19788731

Cross-species signaling

One of the conundrums of language is why an individual would give away information.  Humans appear desperate to tell people about things.  Animals, on the other hand, aren’t that interested in communicating beyond basic signalling for survival.  However, the wildlife video below points out another problem:  Even if an animal has a desire to communicate, others might not want to listen.

This video captures the difficulty of trying to make yourself understood through arbitrary signalling systems, and the relative ease of gesture.

Something I didn’t know was that Hedgehog spines, as well as being protective,  evolved so that they can be vibrated as a form of signalling.

The Bog

If you like wading through deposits of dead animal material, then you should go over and visit Richard Littauer’s new blog, The Bog. Having been exposed to his writings on both this blog, and through the Edinburgh language society website, I’m sure it will be worth a visit — for good writing, if not for your dire need to distinguish between forest swamps and shrub swamps. His first post is on Mung, the colloquial name for Pylaiella littoris, which is apparently a common seaweed. Here is his quick overview of the blog:

So, The Bog is going to be the resting place for various studies and explorations. Richard Littauer is the writer; he is working on his MA in Linguistics at Edinburgh University. He writes about evolutionary linguistics and culture at Replicated Typo, about general linguistic musings at a non-academic standard at Lang. Soc., about constructed languages on Llama, and about various philosophical things at Pitch Black Press. Since none of these blogs were a perfect fit for the scientific equivalent of a swamp-romp through subjects he doesn’t study, he set up this blog. Expect posts about ecology, biology, linguistics, anthropology, or anything in between.

The fact that it’s called the Bog has nothing to do with the British slang for ‘bathroom’. Rather, Richard (well, I) have an affinity with swamps for some unexplained reasons. Expect posts on swamps.

If that doesn’t appeal to you, then Richard is also well-known for being the world’s number one Na’vi fan.

From Natyural to Nacheruhl: Utterance Selection and Language Change

Most of us should know by now that language changes. It’s why the 14th Century prose of Geoffrey Chaucer is nearly impenetrable to modern day speakers of English. It is also why Benjamin Franklin’s phonetically transcribed pronunciation of the English word natural sounded like natyural (phonetically [nætjuɹəl]) rather than our modern variant with a ch sound (phonetically [nætʃəɹəl]). However, it is often taken for granted on this blog that language change can be understood as an evolutionary process. Many people might not see the utility of such thinking outside the realm of biology. That is, evolutionary theory is strictly the preserve of describing biological change, and is less useful as a generalisable concept. A relatively recent group of papers, however, have taken the conceptual machinery of evolutionary theory (see Hull, 2001) and applied it to language.

It's all natyural, yo!

Broadly speaking, these utterance selection models highlight that language change occurs across two steps, each corresponding to an evolutionary process: (1) the production of an utterance, and (2) the propagation of linguistic variants within a speech community. The first of these, the production of an utterance, takes place across an extremely short timescale: we will replicate particular sounds, words, and constructions millions of times across our production lifetime. It is as this step where variation is generated: phonetic variation, for instance, is not only generated through different speakers having different phonetic values for a single phoneme — the same speaker will produce different phonetic values for a single phoneme based on the context. Through variation comes the possibility of selection within a speech community. This leads us to our second timescale, which sees the selection and propagation of these variants — a process that may “take many generations of the replication of the word, which may–or may not–extend beyond the lifetime of an individual speaker.” (Croft, in press).

Recent mathematical work in this area has highlighted four selection mechanisms: replicator selection, neutral evolution, neutral interactor selection, and weighted interactor selection. I’ll now provide a brief overview of each of these mechanisms in relation to language change.

Continue reading “From Natyural to Nacheruhl: Utterance Selection and Language Change”

Mutual Exclusivity in the Naming Game

The Categorisation Game or Naming Game looks at how agents in a population converge on a shared system for referring to continuous stimuli (Steels, 2005; Nowak & Krakauer, 1999). Agents play games with each other, one referring to an object with a word and the other trying to guess what object the first agent was referring to. Through experience with the world and feedback from other agents, agents update their words. Eventually, agents are able to communicate effectively.  The model is usually couched in terms of agents trying to agree on labels for colours (a continuous meaning space).  In this post I’ll show that the algorithms used have implicit mutual exclusivity biases, which favour monolingual viewpoints.  I’ll also show that this bias is not necessary and obscures some interesting insights into evolutionary dynamics of langauge.

Continue reading “Mutual Exclusivity in the Naming Game”

Mathematical Modelling 101 – The Price Equation

So in this post I’m going to assume you know absolutely nothing about anything. If you know something about something this probably isn’t what you’re looking for. If you’re looking for something which will go into depth on how the price equation is derived this probably isn’t what you’re looking for either. If you simply want to know what the price equation does and how to use it at face value then welcome! You’ve found the right place.

The price equation is used to calculate how the average value of any variant can change within a population from generation to generation.

Here I will cover everything you need to know to understand the equation and slot in the right values:

Continue reading “Mathematical Modelling 101 – The Price Equation”