Cognitivism and the Critic 2: Symbol Processing

It has long been obvious to me that the so-called cognitive revolution is what happened when computation – both the idea and the digital technology – hit the human sciences. But I’ve seen little reflection of that in the literary cognitivism of the last decade and a half. And that, I fear, is a mistake.

Thus, when I set out to write a long programmatic essay, Literary Morphology: Nine Propositions in a Naturalist Theory of Form, I argued that we think of literary text as a computational form. I submitted the essay and found that both reviewers were puzzled about what I meant by computation. While publication was not conditioned on providing such satisfaction, I did make some efforts to satisfy them, though I’d be surprised if they were completely satisfied by those efforts.

That was a few years ago.

Ever since then I pondered the issue: how do I talk about computation to a literary audience? You see, some of my graduate training was in computational linguistics, so I find it natural to think about language processing as entailing computation. As literature is constituted by language it too must involve computation. But without some background in computational linguistics or artificial intelligence, I’m not sure the notion is much more than a buzzword that’s been trendy for the last few decades – and that’s an awful long time for being trendy.

I’ve already written one post specifically on this issue: Cognitivism for the Critic, in Four & a Parable, where I write abstracts of four texts which, taken together, give a good feel for the computational side of cognitive science. Here’s another crack at it, from a different angle: symbol processing.

Operations on Symbols

I take it that ordinary arithmetic is most people’s ‘default’ case for what computation is. Not only have we all learned it, it’s fundamental to our knowledge, like reading and writing. Whatever we know, think, or intuit about computation is built on our practical knowledge of arithmetic.

As far as I can tell, we think of arithmetic as being about numbers. Numbers are different from words. And they’re different from literary texts. And not merely different. Some of us – many of whom study literature professionally – have learned that numbers and literature are deeply and utterly different to the point of being fundamentally in opposition to one another. From that point of view the notion that literary texts be understood computationally is little short of blasphemy.

Not so. Not quite.

The question of just what numbers are – metaphysically, ontologically – is well beyond the scope of this post. But what they are in arithmetic, that’s simple; they’re symbols. Words too are symbols; and literary texts are constituted of words. In this sense, perhaps superficial, but nonetheless real, the reading of literary texts and making arithmetic calculations are the same thing, operations on symbols.

Arithmetic as Symbol Processing

I take it that learning arithmetic calculation has two aspects: 1) learning the relationship between primitive symbols, such as numerals, and the world, and 2) learning rules for manipulating those symbols. Whatever is natural to the human nervous system, arithmetic is not. Children get a good grip on their native tongue little or no explicit teaching; it just comes ‘naturally.’ But it takes children hundreds if not thousands of hours to become fluent in arithmetic. It is not natural in the sense the language, natural language, is.

In what is known as the Arabic notation, we have ten primitive symbols for quantities: 1, 2, 3, 4, 5, 6, 7, 8, 9, 0. That last is a real puzzler and has been regarded as evil at various times and places: How can something, even a mere mark, represent nothing? Children learn the meaning of these symbols by learning to count collections of objects, both real objects (e.g. blocks, pebbles, buttons, whatever), but also objects represented by pictures on a page.

And we have four primitive symbols for operations (+ – × ÷) of which the first two, for addition and subtraction, are the most basic. Children learn their meaning both through manipulating collections of objects (or their visual representations) and through rules of inference.

To be sure that’s not what we call them, but that’s what they are. We call them the addition and subtraction and multiplication and division tables. Each entry in these tables contains a single atomic fact of the form: string1 = string2. String1 always consists of an operator (+ – × ÷) between two numbers. String2 always consists of a single number. To solve an arithmetic problem we must use these atomic facts to make simple inferences. For example:

1 + 1 = 2
5 – 3 = 2
3 × 4 = 12
8 ÷ 2 = 4

And then there are procedures for every more complex cases. My point is simply that arithmetic calculation is symbol processing. Always. Through and through.

Computation: Selection and Combination

The Structuralists talked about the axis of combination and the axis of selection. They were talking about language and, by implication, things that could be analogized to language. But their inspiration is mathematical, as we can see with a bit of elementary algebra. Here’s a simple equation:

x + y = z

The horizontal IS the axis of combination. Here we’re combining three variables, x, y and z, and two operators, + and =. The value of z depends on the value of x and y; it is thus a dependent variable. The values of x and y are not dependent on the values of anything else in this form, and so they are independent variables. The purpose of the form is to tell us just how the value of the dependent variable is related to the values of the independent variables.

The axis of selection is, in effect, the source of values for those variables. Let us say that it is the positive integers plus zero: 0, 1, 2, 3, 4 . . . Now we can select values for x and y along that axis and come up with values for z by using the various rules and procedures of elementary arithmetic. So:

7 + 4 = z: z must be 11
13 + 9 = z: z must 22
4 + 8 = z: z must be 12

And so forth.

Now, consider these expressions from linguistics, where we can use → instead of =, but to a similar effect:

S → NP + VP
NP → N
NP → det + N
VP → V + NP

Those are rules of combination and they are defined in terms of variables: S = sentence, N = noun, NP = noun phrase, V = verb, VP = verb phrase, and det = determiner. Given just those rules, we can generate these forms, among others, for proper sentences:

1) N + V + N
2) N + V + det + N
3) det + N + V + N

To have actual sentences we need to put words into those variables. For example, we can select from these words, among many others:

Nouns: John, boy, Mary, girl, candy, ball
Verbs: like, hit
Determiners: a, the

Buy choosing from the appropriate selection sets we get these sentences, which I’ve indexed to our forms, 1, 2, and 3:

1a) John likes candy.
1b) Mary likes John.
2a) Mary hit the ball.
2b) John hit a girl.
3a) The boy likes candy.
3b) A girl hit John.

For the past half-century linguists have studied syntax from this point of view, broadly speaking — though some will no doubt tell you that I’m now speaking too broadly. There are, in fact, various schools of thinking about syntax and related topics, and they are not mutually consistent. Differences between these schools are deep and, when contested at all, are fiercely contested. But mostly the different schools ignore one another.

And What of Meaning?

Ah, yes, what of it?

Here I would make a distinction between semantics and meaning. Meaning, it seems to me, is fundamentally subjective; that is, it arises only in the interaction of a subject and, in this case, a text (whether written or spoken). Of course, people can communicate with one another and thereby share meanings; and so meaning can be intersubjective as well.

Semantics, on the other hand, is not subjective. To be sure, I’m tempted to say that semantics has to do with the meaning of words, but that would sink me, would it not? If I’ve already said that meaning is subjective, then why would I attempt to assert that semantics IS NOT subjective?

Because it isn’t. Semantics, properly done, is as dumb as rocks. I am thinking of semantics as a domain of study, a topic within linguistics, psychology, philosophy, and computer science. In those contexts it is not subjective. Those investigations may not be fully satisfactory, indeed, they are not; but they are not subjective. Each line of thought, in its own way, objectifies semantics, that is, roughly speaking, the relationship between words and the things and situations to which they (can) refer.

And various computer models of language are among the richest attempts at objectified semantics we’ve got. It is one thing to observe that existing objectifications are inadequate. But one should not infer from that that better, indeed much better, objectifications are impossible in principle. That may be so, but that principle has not, to my knowledge, been demonstrated.

So, semantics is not understood nearly so well as syntax and – as I’ve already indicated – we have major disagreements about syntax. But I don’t think we need to understand semantics deeply and fully in order to assent to the weaker statement that, however it works, it involves symbol processing. That does not, as far as I’m concerned, imply that semantic processing is nothing but symbol processing.

Not at all. It is clear to me that the meaning of symbols is ultimately grounded in non-symbolic schemas, an idea that’s become associated with the notion of embodied cognition. David Hays and his research group (of which I was a member) pursued that line at SUNY Buffalo in the mid-1970s.* And computational investigations of non-processing have been on-going for years, with computer vision being the most richly developed.

And that makes my larger point, for if computation can encompass non-symbolic processing as well as symbolic processing, what else is there? I saying this I do not mean to imply that it’s all smooth sailing from here on out – just hop on the computational bus, weigh anchor, fire the jets, and bombs away hot diggity dawg!! Not at all. There’s still much to learn. In fact over the last half century it’s as though the more we’ve learned, the more we’ve come face to face with our ignorance. That’s how it goes when you’re exploring a new world.

And the only way you can explore this particular new world, is to think in terms of computation. Just how you do that thinking, that depends on your taste, inclination, imagination, and the problems you’re investigating. But, as far as I can tell, computation’s the only game in town.

* * * * *

* See, for example:

William Benzon, Cognitive Science and Literary Semantics, MLN, Vol. 91, pp. 952-982, 1976.

David Hays, Cognitive Structures, HRAF Press 1981.

William Benzon, Cognitive Science and Literary Theory, Dissertation, Department of English, SUNY Buffalo, 1978.

2 thoughts on “Cognitivism and the Critic 2: Symbol Processing”

  1. On numbers v. words.

    Written numbers (Arabic, Roman, tallies, etc.) are logograms, and moreover, not just any kind of logograms, of the kind you see in Chinese or typographical abbreviations like @ and $ and &, but highly systemic logograms that combine in non-phonetic ways in writing. They also have the property of being processed by an exception free (or very nearly exception free if you count division by zero or taking the square root of negative one as an exception) system of processing rules that are themselves represented by logograms. Numbers act the way the 18th century grammerians thought that grammar was supposed to act. Numbers can, of course, be expressed verbally or in written out words, but this generally is not done for the purpose of computation. When we compute we convert numbers into logographic symbols that are part of the system first, then we compute, then we translate the result back into ordinary language. The written number system in absolutely necessary for computation and people who don’t have written numbers don’t do abstract computation – they do physical comparison computation. Moreover people with better written (and oral) number systems are better at computation.

    I had a geometry textbook in high school that brought that point home by providing untranslated math problems from non-English textbooks as illustrations and problems. Newton may have been a co-inventor of calculus, but it would be totally impracticable to do calculus without the logograms that Leibneiz created to represent it.

    Language in the larger sense is not like that. Analysis of large language datasets and efforts to create translators and language processing AI systems have discovered that grammar has a lot more to do with statistical pattern frequency in which phrases have meanings assigned to them than it does with the kind of logical grammatical rules you illustrate. Sure, there are “default” grammatic rules that are often followed, but exceptions are much more pervasive and exist in a much larger universe of cases than the typical native speaker is aware are exceptions or present multiple options because statistical usage habit rather than grammatical rules are primary and grammatical rules are something of a fallback that only loosely guides the exceptions to the rules.

    Language is also (with the exception of written Chinese and isolate logograms in primarily phonetic or syllabic writing systems) about related written markings to word sounds and then word sounds to writing with a certain degree of arbitrariness in the orthography by which word symbols are translated into word sounds. There is some measure of systematization in Chinese characters, and there is some measure of systematization in grammatically related words, but for the most part spoken and written language is a relatively arbitrary assignment of small groups of grammatically related sound clusters to semantic meaning boxes, which are themselves flexible since languages do not insist that one sound pattern have one meaning or that one meaning be represented by one sound pattern. Even the same word in different context has different meaning and is present in more than one box at a time. The link between semantics and words is more like blog post tagging and less like a filing system.

    Also unlike numbers, language is not terribly reductionist. Computation is generally about turning lots of numbers you have into a smaller number of numbers (often just one) that you are seeking. Computation is the process of condensing many numbers into the number you want. It is symbol processing but more than that it is symbol distilling. In contrast, language generally isn’t goal oriented in the same way. The purpose of a story is not to find a summary that is equivalent to the whole (Cliff notes be damned). It is to experience the sequence of sounds that correspond to meanings sequentially in their complete expanse. Looking at the end of a mystery novel to see who did it and how defeats the purpose of reading one, rather than achieving it. It is the absence of a reductionist goal that makes computation a poor metaphor for language processing, and trying to do language processing in a formal logic paradigm often makes obvious problems or issues with a line of reasoning turn invisible (such as words that have more than one meaning or bits of information in an information processing paradigm that have different weight) because the paradigm suppresses features of ordinary language that make it useful.

    If you are sufficiently reductionist to talk about bare symbol processing you don’t necessary have much of value left to talk about that isn’t obvious.

  2. Thanks for the comment. It is most useful.

    On language as symbol processing, it’s complicated. My actual views, whatever they are, are not well represented in the post. The post was intended to make one point, that computation and language are not so very different that the world will fall apart if one even dares to think of them in the same context. Many humanists, alas, seem to be a bit like that.

    There is, of course, the view that semantics and syntax are independent phenomena. And the little bit of syntax in my post comes from that world. But that’s not what I believe. I think that syntax is handmaiden to semantics and that semantics is where the action is. But how does semantics work? Well, there are those who believe it’s some flavor of propositional logic. I’m not one of them.

    What do I believe? That’s complicated. Whatever the process is, it’s fuzzy and stochastic. It’s NOT symbol processing. Perhaps it IS computational; it all depends on what you mean by computation. And that’s a messy one as well. But if it is NOT AT ALL computational, what is it?

Leave a Reply