How Do the Blind 'See'? The Role of Spontaneous Brain Activity in Self-generated Perception

Avital Hahamy; Meytal Wilf; Boris Rosin; Marlene Behrmann; Rafael Malach

Disclosures

Brain. 2021;144(1):340-353. 

In This Article

Results

Does spontaneous brain activity underlie deprivation-related visual hallucinations, and if so, by what mechanism? To answer these questions, five late-onset blind individuals with CBS, 11 late-onset blind controls who did not experience hallucinations, and 13 sighted control participants were recruited for an functional MRI study with three main conditions (Figure 1). CBS participants verbally/manually reported their hallucinations while being scanned. A visual simulation composed of their hallucinatory streams was later presented to sighted controls. In addition, all participants completed a visual imagery scan.

Behavioural Imagery Abilities

Imagery abilities, as assessed using the total score of the VVIQ questionnaire (Marks, 1973), did not differ between the CBS (91.4 ± 12.83) and either the blind (69.64 ± 8.34) or the sighted (78.15 ± 10.64) control groups (P = 0.08 and 0.23, respectively, two-tailed permutation tests) (Table 1). Similarly, the in-scanner reports of the level to which participants succeeded in imagining visual categories during the experiment did not differ between the CBS (16.3 ± 1.6) and either the blind (17.45 ± 0.54) or the sighted (13.23 ± 1.06) control groups (P = 0.2 and 0.08, respectively, two-tailed permutation tests).

Hallucinations and Veridical Vision Activate Similar Visual Areas

We first aimed to compare brain activations evoked by hallucinations in the three CBS participants able to provide a reliable spatiotemporal description of their hallucinations (see 'Materials and methods' section and Table 1) to activations evoked by veridical vision in sighted controls. As depicted in Figure 2A, a whole-brain analysis of hallucination-related activity in the CBS participants revealed significant activations across the entire visual hierarchy (see Supplementary Figures 1 and 2 for single CBS participants' maps). The simulated hallucinations in the sighted control group significantly activated the central visual field representations of early visual areas, as well as areas in the ventral stream, including the FFA. Peripheral visual field representations were significantly deactivated (Figure 2B).

Figure 2.

Hallucination compared to normal vision. Left column: Unthresholded maps; right column: The same maps with a statistical threshold, corrected for multiple comparisons. (A) CBS group maps in the hallucination condition (hallucination versus baseline), projected onto a representative flat cortical surface (the same map projected onto an inflated cortical surface is presented for reference). Wide red/blue contours depict significant areas of activation/deactivation, corrected for multiple comparisons. (B) Sighted control group maps in the simulated hallucination condition (simulated hallucinations versus baseline). Black contours depict significant areas of activation or deactivation, corrected for multiple comparisons. (C) The same group maps of the sighted controls in the simulated-hallucination condition, as presented in B, superimposed with the significant areas activated/deactivated during hallucinations in the CBS group (wide contours, as in A). White contours depict visual landmarks, based on a probabilistic atlas. FEF = frontal eye field; LH = left hemisphere; LO = lateral occipital complex; pIPS = posterior intraparietal sulcus; RH = right hemisphere. See Supplementary Figs 1 and 2.

The apparent group difference in the spatial extent of visual activations between the CBS and sighted controls may have been the result of the difficulty in accurately simulating the internal visual experiences of the CBS participants. For example, because of technical limitations, some of the displays constructed for the simulated hallucinations were much smaller spatially than those reported by the CBS participants, and may thus have led to reduced activation in peripheral visual areas. Nevertheless, in line with our hypothesis that hallucinations and veridical vision would evoke a similar profile of brain activity (Figure 1), superimposing the contours of significantly activated areas in the CBS group over the sighted controls' activation map, revealed that the two patterns of activity overlapped across many regions of the visual hierarchy (Figure 2C). This observation was quantitatively supported by significant inter-group correlation of activity patterns across the entire posterior part of the brain (median of correlations = 0.2, df = 12, P = 0.002, two-tailed Wilcoxon test). This profile of overlap was mostly unaltered when brain activations of sighted controls were not modelled based on the timings of reports made by CBS participants, but rather on the timing of reports made by the sighted controls themselves (median = 0.11, df = 12, P = 0.006, two-tailed Wilcoxon test; Supplementary Figures 1 and 2. Note that modelling of the sighted controls' own reports was done on those scans in which participants reported manually, which comprised half of the data). Furthermore, these correlations were significantly greater than would have been expected by noise, as estimated using shuffled data (P < 0.001/P = 0.01 for CBS-locked/controls-locked protocols, respectively, two-tailed permutation tests).

It could be argued that the hallucination-related activations in the CBS group were not related to the self-generated visual precepts, but rather to the attentional and report-related aspects of the experiment. To test this possibility, CBS participants and blind controls completed a tone discrimination task in which they responded manually or verbally to specific tones. As presented in Supplementary Figure 3A (also see Supplementary Figures 4 and 5 for maps of single CBS participants), blind controls showed weak activations in parts of the visual system during this task, which showed some resemblance to the spatial patterns of hallucination-related activations (median = 0.09, df = 9, P = 0.05, two-tailed Wilcoxon test). These correlations were also greater than would have been expected by noise (P = 0.01, two-tailed permutation test). A qualitative examination of Supplementary Figure 3B and C revealed that these report-related activations were diminished in the CBS group map (here the ability to directly test for similarity to the spatial patterns of hallucination-related activations is compromised due to statistical/methodological reasons, see 'Materials and methods' section). A whole-brain between-group contrast revealed no significant differences between the CBS and blind control groups. Thus, some weak visual activations were evoked by auditory, verbal or motor processing in the blind control group, and perhaps in the CBS group as well. However, it seems unlikely that weak activations that appear in both groups would underlie the emergence of visual hallucinations, which occurs only in the CBS group.

Hallucinations and Visual Imagery in the Blind Activate Similar Visual Areas

The unprompted nature of visual hallucinations differentiates them from veridical vision. However, another difference is that hallucinations are internally generated while veridical vision is evoked by external stimulation. We next set out to compare hallucinations to cued visual imagery, as both are internally generated, but only hallucinations are unprompted. The maps of all experimental groups during the visual imagery condition are presented in Figure 3 (also see single CBS participants' maps during visual imagery in Supplementary Figures 4 and 5). These maps display a network of visual and frontal areas, previously reported to be involved in visual imagery and visual perception in sighted controls (Behrmann, 2000; Ganis et al., 2004; Yellin et al., 2015; Dijkstra et al., 2017). As depicted in Figure 3, left, while the sighted control group tended to activate only higher-order visual areas and deactivate mid-level areas, both the CBS and blind control groups showed activations across the visual system, which reached significance in both lower- and higher-order visual areas. A direct comparison of whole-brain activations between the CBS and each of the control groups revealed no significant differences.f

Figure 3.

Hallucination compared to visual imagery. Left column: Unthresholded maps; right column: the same maps with a statistical threshold, corrected for multiple comparisons. Group maps of the CBS group (A), blind control group (B), and sighted control group (C) during the visual imagery condition (imagery versus baseline). These maps are overlaid with the contours of areas activated/deactivated during hallucinations in the CBS group (as presented in Figure 2, wide contours). Note the spatial overlap between hallucination and imagery activations in both the CBS and blind control groups in the left panel. FEF = frontal eye field; LH = left hemisphere; LO = lateral occipital complex; pIPS = posterior intraparietal sulcus; RH = right hemisphere. See also Supplementary Figs 4 and 5.

As expected by the visual inspection of the sighted controls' group map (Figure 3C), no evidence for similarity between imagery activations in the sighted controls and hallucination-related activations in the CBS group was found in the posterior part of the brain (median = 0.13, df = 12, P = 0.09, two-tailed Wilcoxon test). However, a visual inspection of the left panel of Figure 3 suggested that the contours of hallucination-activated areas in the CBS group largely matched the spatial activations seen during visual imagery in the CBS and blind control groups. This observation was confirmed quantitatively in the blind control group (median = 0.16, df = 9, P = 0.002, two-tailed Wilcoxon test), and correlations were significantly greater than those expected by noise (P ≤ 0.001, two-tailed permutation test). We note that the ability to test for similarities between the spatial activation patterns evoked by hallucinations and imagery in the CBS group is compromised because of statistical and methodological reasons (see 'Materials and methods' section). However, our results suggest that visual hallucinations in CBS and visual imagery in the blind (the population from which CBS emerges) tend to evoke similar spatial patterns of activations in the posterior brain.

The Temporal Dynamics of Hallucinations Differ From Those of Veridical Vision and Visual Imagery

We next hypothesized that although activations evoked by hallucinations show some spatial overlap with those evoked by veridical vision in the sighted and visual imagery in the blind, they may differ in their temporal dynamics. As schematically illustrated in Figure 1, we expected that both the external perception and cued imagery of stimuli would bring about a rise in the BOLD response in visual areas. This would suggest that the optimal fit of an HRF to the data would be expected when there is a negligible lag (of no more than 1 TR) between the data and HRF. Contrary to this, we hypothesized that a rise in the visual BOLD response would precede hallucinations. This would require that the optimal fit of an HRF to the data would be at a negative lag, such that the BOLD signal begins rising prior to hallucination onset.

To test whether the rise of the BOLD signal in CBS participants during the hallucination condition precedes that of controls during the simulated-hallucination condition (Figure 1), we first examined the neural dynamics of a bilateral early/intermediate visual region of interest comprising areas V1–V4 (Figure 4A), which showed a significant BOLD signal increase during hallucinations (P = 0.04, one-tailed permutation test). As depicted in Figure 4B, in the sighted control group, the appearance of stimuli in the simulated-hallucination condition was accompanied by a rise in the BOLD signal (mean optimal lag −0.26 TR). Importantly, and consistent with our hypothesis, the BOLD signal in the CBS group began building up 2.33 TRs on average before the reported onset of hallucinations in the early/intermediate visual region of interest. This difference in the temporal dynamics between the CBS and sighted control groups was statistically significant in this region of interest (P = 0.002, one-tailed permutation test), and also in an additional early/intermediate visual region of interest composed only of voxels which were significantly activated by hallucinations (P = 0.003, one-tailed permutation test). A similar effect was also observed when the modelled events in sighted controls were based on their own responses in the scans involving manual report (half the scans), rather than on the reports of the CBS participants across all scans (sighted controls optimal lag −0.97 TRs, P = 0.056 one-tailed permutation test; Supplementary Figure 6).

Figure 4.

Hallucination, vision and visual imagery have different temporal dynamics. (A) Regions of interest V1–V4 (yellow contours), FFA (purple contours), lip areas (light blue contours). (B) Event-related averaging of group activations within each region of interest. Rows correspond with regions of interest (top row: V1–V4; middle row: FFA; and bottom row: lip region), and columns correspond with experimental conditions (hallucination in CBS participants and simulated hallucination in sighted controls in the left column, and visual imagery in the right column). Groups are represented by green, black and grey lines for CBS, sighted controls, and blind controls, respectively. Dashed lines depict hallucination onset (CBS), stimulus appearance in the visual condition (sighted controls) and onset of imagery instruction in the imagery condition. Note that visual (V1–V4, FFA) activations in the CBS group precede the onset of hallucinations, as marked with dark green arrows. This is unlike non-visual activations (lips region of interest) in the CBS group, simulated-hallucination activations in the sighted control group and imagery activations in all groups. See also Supplementary Figs 6–8.

Importantly, the group difference in temporal dynamics was not observed during the cue-driven imagery scans. During imagery, the temporal dynamics of the CBS group (mean optimal lag 0.6 TRs) did not differ from those of sighted controls (mean optimal lag −0.75 TRs, P = 0.09, two-tailed permutation test) or of blind controls (mean optimal lag: −0.3 TRs, P = 0.07, two-tailed permutation test). A significant group (CBS, Sighted controls) × condition (Hallucination/simulated-hallucination, Imagery) interaction (sighted controls' protocol locked to the CBS participants' report: P = 0.01; protocol locked to the sighted controls' report: P = 0.04, two-tailed permutation tests) indicated that the earlier build-up of activity in the CBS participants during hallucination (compared to veridical vision in the sighted controls) is not a general temporal characteristic of their visual system, but rather a unique manifestation of hallucination.

To examine activations in higher order visual regions, we chose the bilateral FFA (Figure 4A), both because all reporting CBS participants hallucinated faces (Table 1), and because the fusiform gyrus has been previously reported to show a build-up of activity prior to the onset of hallucinations in CBS participants (Ffytche et al., 1998). As depicted in Figure 4C, a build-up of activity in the FFA did indeed precede hallucinations in the CBS group (mean optimal lag: −1 TR), replicating previous findings (Ffytche et al., 1998). While this difference did not reach significance when the protocol of sighted controls was locked to the CBS participants' report in both manual and verbal scans (sighted controls optimal lag: −0.3 TRs, P = 0.32, one-tailed permutation test), significant results were found when using a protocol locked to the sighted controls' reports in the manual scans (sighted controls optimal lag: 0.3 TRs, P = 0.04, one-tailed permutation test). This relatively weak effect is reflected in the bimodal shape of the averaged FFA signal, which captures the higher variability between the temporal dynamics of single CBS participants in this region compared to the dynamics in the early/intermediate visual region of interest (Supplementary Table 2). An opposite effect was observed in the visual imagery condition, as the mean optimal HRF lag of the CBS group (0.8 TRs) was significantly delayed compared to the mean optimal lag of the sighted control group (−1.1 TRs, P = 0.01, two-tailed permutation test, though note this effect is likely due to the reduced FFA activation during imagery in the sighted controls). A similar effect was found when comparing the CBS to the blind control group (optimal lag: 0.3 TRs, P = 0.04, two-tailed permutation test). This slightly delayed rise in signal in the CBS versus blind control participants may reflect slower initiation of imagery upon instruction in CBS participants, possibly due to interference of hallucinatory visual precepts. The observation of an earlier build-up of activity in CBS participants during hallucinations compared to imagery, and compared to sighted control participants was supported by a significant group (CBS, sighted controls) by condition (hallucination/simulated-hallucinations, imagery) interaction (sighted controls' protocol locked to CBS report: P = 0.02, protocol locked to the sighted controls' report: P < 0.001, two-tailed permutation tests).

Of significance, no build-up of activity in the CBS group was observed in areas engaged in the hallucination condition, but located outside the visual system proper. CBS participants showed a slightly delayed BOLD signal in the motor lips area (optimal lag: 0.33 TR) compared to sighted controls (optimal lag: −0.49 TR, P = 0.03 two-tailed permutation test) in the scans involving verbal report of hallucinations. Similarly, CBS participants showed a slightly delayed BOLD signal in the motor hand area (optimal lag: 0 TR) compared to sighted controls (optimal lag: −0.47 TR, P < 0.001, two-tailed permutation test) in the scans involving manual report of hallucinations. A significant interaction between groups (CBS, sighted controls) and regions of interest (early/intermediate visual cortex, lips region: P < 0.001; early/intermediate visual cortex, hand region: P < 0.001, two-tailed permutation tests) confirmed that the early build-up of activity in the CBS compared to the sighted control group was specific to the visual cortex.

Supplementary Figures 7 and 8 present the temporal dynamics in the brains of single CBS participants compared to the relevant sighted controls' data, based on events locked to the CBS or sighted controls' report.

Signals Preceding Hallucinations Diminish Across the Visual Hierarchy

The above analyses of our data revealed that although hallucinations engage the entire visual system, the extent to which neural activity builds up prior to hallucination may vary across visual brain regions. We therefore aimed to examine whether there was a trend in the 'propagation' of this preceding signal across the visual hierarchy. Of importance, when measuring evoked visual responses, synaptic lags along the cortical hierarchy are too rapid to be captured using functional MRI (Dijkstra et al., 2020). However, since spontaneous brain activity is composed of ultra-slow fluctuations (Nir et al., 2008), its decay across the visual system could be captured despite the sluggishness of the BOLD signal. Indeed, as depicted in Figure 5, a significant correlation was found in the CBS group during hallucination, such that lower-level visual areas demonstrated an earlier build-up of activity relative to hallucination report, compared to higher-order visual areas (r = 0.58, P = 0.004, two-tailed permutation test, see Supplementary Table 3 for correlations in single CBS participants).

Figure 5.

Hallucination-related preceding signals decay along the visual hierarchy. The top and bottom plots present the data of the CBS group in the hallucination and imagery conditions, respectively. X-axes represent the hierarchical rank of visual regions of interest (Supplementary Table 1), grey labels depict one representative region of interest from each ranking category. Y-axes represent the optimal lag of the HRF relative to stimulus onset, as fitted to signals from these regions of interest. Dots depict group means for regions of interest and error bars depict standard deviations. IPS = intraparietal sulcus; SPL = superior parietal lobe.

To evaluate whether the ordered propagation of preceding signals across the visual system is distinctive of hallucinations in participants with CBS, we tested for a correlation between visual ranks and optimal lags in the CBS group during the imagery condition, and found no evidence for such a relation (r = 0.24, P = 0.13, two-tailed permutation test).

Comments

3090D553-9492-4563-8681-AD288FA52ACE

processing....