Chapter 5 - Binding is Inference
Presented by Jona Vance
Part 1 of the book (Chs 1-4) sets out the prediction error minimization (PEM) framework. Part 2 (Chs 5-8) applies the PEM framework to a number of specific problems and phenomena in cognitive science and the philosophy of mind. This post is on Ch 5, which addresses the binding problem (or problems).
Hohwy has two main stated aims in Ch 5. First, he aims to use PEM to give a “reasonably detailed answer” to the binding problem. Second, he aims to use the debate about binding issues and the phenomena it centers on to illustrate how the PEM framework can be applied to various interesting cases. So the chapter aims to use PEM to illuminate how binding works and aims to use binding to illuminate how PEM works.
Hohwy glosses the binding problem in a few ways. On one gloss it concerns “how the brain manages to discern properties of objects in the world and correctly bind these properties together in perception” (p. 101). A second gloss adds that part of the problem is to explain how the brain correctly binds properties “in spite of processing them in different regions throughout the brain” (p. 101). For example, if visual receptors receive information as of something red and as of something round and the olfactory receives information as of something sweet, the brain still has to figure out whether the redness, roundness, and sweetness are properties of the same object or not. And it has to do so despite processing some of the information in different regions.
Hohwy notes that there are numerous approaches to the binding problem (p. 102). It’s also important to note that there is not just one binding problem. There are numerous related binding problems. Binding issues arise in perceptual processing regarding information across space, types of features, sensory modalities, and binding neural signals across cortical space. Binding issues arise for single percepts and more than one percept, at a single time and across time. This is worth emphasizing in PEM’s favor. On Hohwy’s account, PEM solves the binding problems through a very general mechanism: causal inference via prediction error minimization. PEM promises to offer an elegant solution to the full range of binding problems.
Hohwy’s own PEM solution to perceptual binding problems begins by noting that the ambiguity that must be resolved for accurate binding is in principle no different from other ambiguity problems the perceptual system faces. As a result, we can appeal to the same solution for binding problems as for the more general underdetermination problems that motivate constructivist approaches to perception generally, of which PEM is an example. Regarding perceptual binding, Hohwy appeals to the same Bayesian story as is used for other ambiguity resolution: properties will be bound together and to objects represented in perception just in case a proposition expressing them as bound is the mean or maximum a posteriori (MAP) of the Bayesian perceptual inference. For example, whether the perceptual system represents some bit of redness and roundness as properties of the same object or different ones depends on whether the binding hypothesis (a red, round object) is the MAP of the perceptual inference or not. If it is, the properties are represented at bound in the percept; otherwise not. Researchers working on perceptual binding problems have noted that spatiotemporally overlapping properties tend to get bound in perception. This makes sense from a Bayesian perspective, since there is high prior that spatiotemporally overlapping properties are properties of the same distal object.
Not all Bayesian approaches to perception adopt the PEM framework. (It would have been good, I think, if the book had made this point clearer.) However, at first blush PEM seems to provide a particularly elegant solution to the binding problem. PEM posits that hypotheses about the world are represented at various levels of a perceptual hierarchy (where the levels according to Hohwy’s version of PEM are individuated according to causal regularities at different time scales). The hypotheses already bind together relevant properties. So the framework provides for part of a solution to the binding problem almost by default: the system simply builds in that properties are represented at bound at the various levels. In addition, a full solution to the binding problems requires that the properties be bound *accurately*. This aspect of the problem is solved in PEM because the hypotheses are constantly supervised by feedback from prediction error signals up through the hierarchy. According to the model, inaccurate hypotheses are revised in response to error signals generated by the sensory data.
I. WORRY FOR PEM REGARDING CROSS MODAL INCONSISTENCIES IN BINDING
Hohwy address binding both within and across modalities. He illustrates how cross modal binding works according to PEM using the rubber hand illusion (pp. 104-106).
I now want to use the rubber hand illusion to raise a worry for PEM. There seem to be inconsistencies in the representations across modalities in the rubber hand illusion. In the illusion, the following propositions are all simultaneously represented in perception, though not all in the same modality.
Visually, it looks as if
(1) the rubber hand is rubbery.
Haptically, it feels as if
(2) the subject’s hand is not rubbery.
But haptically again, it feels as if
(3) the subject’s hand is the rubber hand.
1, 2 and 3 are inconsistent. So it looks like one’s perceptual system simultaneously represents a set of inconsistent propositions across two modalities (sight and touch).
The rubber hand illusion is not unique in having inconsistent propositions that are represented simultaneously across modalities for a single subject. To see another example, consider the haptic and visual contents that are simultaneously represented when one both looks at and holds a straight stick partly submerged in water. The represented claims include:
(4) The stick is bent.
(5) The stick is unbent.
4 and 5 are inconsistent: the stick can’t be both bent and unbent. So again it looks like there are actual cases in which inconsistent claims are simultaneously represented across different perceptual modalities. (Note that the claim is about inconsistencies across modalities. There is no claim here that a single percept has inconsistent contents, as in some objections to classical sense data theory.)
Here’s how these cases might provide an objection to PEM. If PEM is true, percepts are generated via Bayesian inference from a set of hypotheses represented in the perceptual hierarchy. The hierarchy is unified in important ways, which is essential to Hohwy’s PEM-based explanation of how binding occurs within and across modalities. So the contradictory hypotheses would have to simultaneously derive from a hierarchy of hypotheses that is in an important sense unified. This seems implausible. So on PEM we should expect that these cross-modal contradictions in perceptual representation do not occur. But such cases do occur. So PEM is not true.
One might reply that in some cases of perceptual binding, only hypotheses that bear on representation in one modality are engaged in the perceptual inference. Even if PEM posits a perceptual hierarchy that is unified in important ways, not every perceptual inference draws from the whole hierarchy. So for example, one might argue that PEM does not entail that perceptual inference that yields the haptic representations in the rubber hand illusion utilize the same hypotheses that are utilized for the perceptual inference that yields the visual representations.
However, this reply seems unpromising. Even if it is true that some perceptual inferences are confined to representations that bear on some narrow perceptual task, that isn’t the case in the rubber hand illusion. The point Hohwy makes in the chapter is precisely that in such cases, the perceptual system draws from a core hierarchy of hypotheses and coordinates information across both modalities to generate the relevant percepts in each modality. The problem for PEM that I am raising is that even when such cross-modal binding occurs, the perceptual system still yields inconsistent representational outputs simultaneously. On a Bayesian PEM account, that arguably should not happen.