Thumbnail: “Four Forms of Information” (2003)
Terrence C. Stewart and Andrew Brook
Carleton University Cognitive Science Technical Reports
The foundational assumption of cognitive science is that the mind is an information-processing machine. This simple stipulation enables discussion among psychologists, computer scientists, linguists, and philosophers, since all these people can say smart things about information-processors. Despite this productive common ground, some ambiguity remains as to what exactly “information” is—(‽) which makes some of the reading quite annoying and tautological!—and this paper by Stewart & Brook outlines four specific types of information (summarized in above diagram).
Type 1: Reduction In Local Uncertainty
This is the mathematical sense of “information” from Information Theory:
This definition is that the amount of information (measured in bits) gained by observing an event is the negative log (base two) of the probability of that event occurring. So, if you flip a coin and discover that it has landed on heads, then you have gained one bit of information. If you flip three coins, and discover that they land heads, tails, heads, in that order, then you have gained three bits of information. If you roll a six-sided die and discover that it landed on a value less than 10, then you gain zero bits of information (Cover and Thomas, 1999).
This definition requires knowledge of event-probabilities, and it refers to the amount of information in a signal. A natural line of research in cognition is visual processing, since our eyes receive an unpredictable stream of photons.
The reason for calling this usage a “reduction in local uncertainty” is to highlight the fact that we are talking about reducing uncertainty in the signal itself. We have something very specific and immediate (the signal), and we do not know what the signal is going to be until it arrives.
This is why some messages may give zero “information”.
(‽) This might be why we phonologists are interested in what is unpredictable in a language sound system. What is not predictable is contrastive and thus meaningful. It’s Information!
Type 2: Exploitable Regularity
The next usage of ‘information’ deals with the idea that while most processes in nature are concerned mostly with the flow of energy (rain, gravity, friction, etc.), living creatures, and human beings in particular, are controlled by the flow of ‘information’. This idea is grounded in the physics of entropy. The laws of thermodynamics imply that everything in the universe tends towards decay and breakdown, yet living creatures are able to maintain a stable internal structure. The reason they are able to do this is by exploiting regularities in the environment. So, in order to maintain your structure, you must obtain ordered materials from your environment (by eating and drinking).
Information here refers to the amount of order—patterns and regularities—in the external world that allows organisms to extract material from it’s environment. For us to do very cool human things requires a world stable enough for those cool things to take place. The Game of Life is a helpful model:
It is this sense of information that is used by Chris Langton in his original Game of Life analysis. By using simple simulated universes with different “physical laws”, he showed that complex “life-like” behaviour arose only in situations where the physical laws led to reasonably stable (regular) environments. However, if the laws made things too regular, then the complex structures would be unable to extract the order from their environments in order to maintain themselves.
(‽) The “too regular” problem reminds me of the problem of unmarkedness in Optimality Theory, whereby the simplest conceivable language—3 consonants, 2 vowels, CV syllables—is not optimal at all! Factorially, we have six possible syllables, meaning there are only 1296 possible four syllable words. At least 40,000 words are required for a language to talk about the things that humans need to talk about, so the majority of words in this hyper-regular language will have six syllables! The absolute absence of complexity is counterproductive for communication, and with respect to information, total unmarkedness provides us information-processors with too little order for us to extract and exploit the order needed to maintain our own complexity. (I hope I got that right!)
(‽) The principle of regular patterns comes up in game design as well. Meaningful play says that a game needs to respond to players in a discernible and predictable manner. A game that responds unpredictably to player actions—this button does x some of the time, does y some of the time, but unpredictably in the same context—is a broken game. Without the regularity of the system or clear rules of cause-and-effect, we cannot extract the information from the system that we need in order to have the kind of faith in our actions that makes our actions meaningful.
The paper contrasts this type of information with the mathematical type:
Here, creatures make use of predictable patterns in the environment. Thus, a regularly patterned environment is more useful to an organism than a random one, since there is more order in it. However, by the first definition of information, the random input would contain more information.
And it would really suck if life were like a bad videogame.
Type 3: Data to be Manipulated and Transformed
The idea here is that the brain is a computer with different components tasked with various functions. The brain processes the signal. A vibration works on your eardrum, and the Organ of Cortia translates mechanical force into nerve activation for auditory processing. Categorical perception translates the raw signal into discrete features which are packaged into units of sound. Language processing strings the sounds into packets of potential language, comparing the input to known words and storing new forms. This kind of information is data in the computational sense: input, processing, output, and storage.
There is a clear distinction between this sense of information and the first sense. In the “reduction of local uncertainty” definition, information was something contained in the signal. Here, we are using the term ‘information’ to refer to the signal itself. Another term for this would perhaps be ‘data’, as that term stresses the fact that we are interested in the objective truth about the state of the signal. What is being transmitted from one component in the brain to another is an objective physical truth (as much as any other truth about the real world).
(‽) This is how I always thought about the Information of information-processing, but it seems so quaint compared to those two prior definitions!
Type 4: Aboutness
Aboutness connects the processed signal to things and circumstances in the real world.
This is the idea that the signal we are receiving gives us information about something in the external world. For example, the signal entering a person’s eyes gives that person information about the things around them: their colour, their shape, their motion, and so on.
As soon as the signal takes on a meaning—as soon as a blast of vibration becomes a word and that word tells us something about the world—the information is about something.
(‽) To jump into a comfortable frame of reference, aboutness emerges when the signifier (signal) signifies (means something). Which is really fuzzy territory. If we’re going to think about cognition as information-processing done by an information-processing machine, at what point in a computer program does the input become about something?!
In the diagram above, the dotted lines between the representations and the external environment mean two things:
- There is no direct line between the real world and it’s representations. This is because aboutness is the only kind of information that may be true or false. We may incorrectly mentally represent the world! The ability for the information processor to be wrong or mistakened is essential for a theory of human cognition.
- Representations allow us to cogitate about things we have never sensed firsthand and things that don’t exist in the physical world.
