| Home | Background | Programme | Invited Speakers | Application | Registration | Venue | Accomodation | Links | Contact |
It has been shown that language and motor functions are intertwined both neurally and functionally. In particular, the posterior part of the left inferior gyrus (Brodmann area 44) and the directly adjacent inferior precentral gyrus (Brodmann area 6) in both the left and the right hemisphere are considered to play a core role in cognitive tasks which require language and motor integration. To fully grasp this complex interplay associated with various language and motor functions it is therefore convenient to replace the concept of a 'modular organization' of neurocognitive brain functioning by the better paradigmatic notion 'network'. Recent neuroimaging work on speech perception and production (Indefrey et al 2001, Meyer et al. 2002) clearly demonstrates covariant activity in cortical areas which have been described as being associated with language and motor functions.
Here I am going to present further data obtained from two recent fMRI experiments pointing to a functionally and neurally densely interrelated network subserving language and motor functions. The first experiment unveiled an involvement of right premotor areas during perception of pure pitch contour and a recruitment of left premotor and left inferior frontal regions during active rehearsal of dynamic pitch information at the sentential level. This finding provides some evidence that processing prosodic cues available in dynamic pitch contour of spoken utterances (intonation, amplitude, duration, and spectral tilt) activates a temporary network fusing language and motor functions. Further evidence for a mingled cooperation between language and motor system comes from an experiment which was to elucidate the cerebral organization of German sign language (DGS). Congenitally deaf participants were delivered visually presented German sign language. The movies presented to the volunteers showed dialogues recorded from two native speaker of DGS. Data analyses revealed a consistent activation pattern across six participants which includes posterior parts of the temporal lobes and ventral premotor areas in both the left and right hemispheres. This finding raises the question in how far the motion aspects of sign languages contribute to a proper understanding of the linguistic information available in 'signed' sentences.
Since this talk is primarily meant to open the conference I am going to sketch further dimensions (developmental aspects, concept of mirror neurons) which might require a functional and neural integration of language and motor functions to master various kinds of linguistic and non-linguistic tasks. Finally the immense benefit of applying neural modeling techniques to functional brain imaging data (Horwitz et al., 2000) will be outlined.
References
Horwitz, B. et al., Neural Networks 13, 829-846, 2000.
Indefrey, P. et al., Proc Natl Acad Sci U S A 98, 5933-5936, 2001.
Meyer, M. et al., Hum Brain Mapp 17, 73-88, 2002.
In previous research (1), regional cerebral blood flow (rCBF) increases related
to syntactic encoding during language production were found in the left frontal
Rolandic operculum reaching into Brodmann area (BA) 44. The aim of the present
PET study was to investigate whether similar cortical activations can be found
for syntactic parsing of the same sentences as expected under the assumption of
a common syntactic processor for comprehension and production of speech.
Subjects and Procedures
Fourteen (6 female, 8 male) right-handed native speakers of German were
presented with animated scenes involving actions of coloured geometrical
objects. In three conditions, subjects responded to identical stimuli either in
a full sentence (S), e.g. "the red circle launches the blue ellipse", with a
sequence of noun phrases (NP). e.g. "red circle, blue ellipse, launch", or with
a sequence of syntactically unrelated words (W), e.g. "circle red, ellipse blue,
launch". Responses always started with the agent of the action. In three further
conditions, subjects were presented with the same visual stimuli but scene
descriptions were presented auditorily rather than generated by the subjects.
Subject were asked to monitor for occasional errors with respect to the action
(e.g. 'go next to' instead of 'launch') or the semantic roles (e.g. 'blue
ellipse, red circle, launch' instead of 'red circle, blue ellipse, launch').
Errors occurred before and after but not during the 40 sec sampling period of
the PET measurement. Twelve PET scans (2 repetitions per condition) were
performed. Data analysis by SPM96 (2) was performed as in the previous study and
included realignment, stereotactic normalization and spatial smoothing with a
10mm isotropic Gaussian kernel.
Results
The three production conditions did not differ significantly in response onset
or duration. The analysis of the rCBF data yielded one area of significant
activation (p < .05 corrected) for the contrasts S-W, NP-W, and their
conjunction. The area was located in BA 44 caudally extending into the Rolandic
operculum (SPM96-x,y,z-coordinates: -60,14,12) and was slightly more rostral but
overlapping with the area found in the previous study (x,y,z-coordinates:
-54,6,10). The same area was significantly more active in sentence and
noun-phrase level production as compared to comprehension. The reverse
comparison showed extensive bilateral superior temporal activations. No
significant differences were found between listening to S and W scene
descriptions.
Conclusions
The syntactic production area subserving both local and sentence level syntactic
encoding was replicated. The area was not sensitive to the same syntactic
differences in the comprehension conditions, suggesting that (a) their is no
common neural substrate for syntactic encoding and parsing or (b) the activation
of the inferior frontal gyrus during sentence comprehension is modulated by
syntactic vs. non-syntactic task demands (3).
References
(1) Indefrey et al., PNAS, 2001.
(2) Friston et al., Human Brain Map., 1995.
(3) Dapretto & Bookheimer, Neuron, 1999.
Broca's landmark patient Leborgne had a lesion that extended well beyond what is typically considered "Broca's area". Since even today there is no consensus on where precisely Broca's area is, it is not surprising that aphasiology has a hard time determining what type of processing it may be involved in. Some provocative attempts have linked this elusive piece of tissue to Chomsky's model of syntactic autonomy (1). Functional neuroimaging, however, shows that its functional scope is much larger. Indeed, the multitude of cognitive paradigms eliciting a hemodynamic response in this region is puzzling. For a few examples from our own work, we have found activation foci in close proximity within area 44 for such diverse tasks as judgment of semantic association, auditory discrimination of slow frequency glides (2), and visually prompted digit sequence learning (3). Such findings raise basic questions of how a cytoarchitectonically rather homogeneous piece of cortex can participate in so many different cognitive processes. Trying to account for the seemingly disparate set of functions associated with the inferior frontal lobe, I will focus on evidence of lesion-induced developmental plasticity and the neuroscience of experiential effects. I will argue that cognitive neuroscience's obsession with neurofunctional demarcation has blinded our sense of how things belong together in language acquisition. A better understanding of the inferior frontal lobe may thus hinge on our decision on when to look for clues in neurocognitive development.
References
1. Grodzinsky Y. Behav. Brain Sci. 2000; 23: 1-71.
2. Müller R-A, Kleinhans N, Courchesne E. Brain Lang. 2001; 76: 70-76.
3. Müller R-A, Pierce K, Kleinhans N et al. Cog. Brain Res. 2002; 14: 277-93.
One of the central problems of language comprehension is that language is systematically ambiguous. In addition to semantic ambiguities, the syntactic structure of a sentence is not always immediately clear. The brain mechanisms involved in syntactic ambiguity resolution were investigated using PET to look at changes in regional blood flow. Ambiguous sentences resolved to the less preferred structure were compared to unambiguous control sentences. The results of the experiment showed activation of four areas: the left inferior frontal gyrus somewhat dorsal to Broca's, the right basal ganglia, the right cerebellum and the left superior frontal gyrus. The first three areas are most familiar as parts of the motor system. Concentrating primarily on the motor areas, neuropsychological evidence will be discussed which supports the claim that these areas indeed play a role in sentence processing. Then their individual cognitive functions will be discussed, since these functions suggest the most likely contribution each makes to comprehension of ambiguous sentences. Lastly, their involvement in a network to support syntactic ambiguity resolution will be discussed in terms of recent computational models based on the interactions of the basal ganglia and the cerebellum with cortical areas.
A brain's-eye-view on speech perception shows that perceptual and motor planning systems are closely coupled in the processing of speech sounds. This suggests that analysis-by-synthesis approaches might be on the right track. In this type of approach, the ability to consult a forward model (synthesis) is closely tied to (and guides) the perceptual analysis. In the context of a speech perception model that takes distinctive features to be the primitives for representation and processing, I discuss why an analysis-by-synthesis perspective provides the most natural interpretation of neuronal data derived from imaging and neurophysiological experiments.
The interactions between language processing and motor behavior can be viewed along a number of different dimensions. First is the motor system as the final common pathway for all expressions of behavior, including linguistic behavior. Second is the relationship between somatomotor function and sensory systems in language performance, whether the use of gesture in producing language, or the visual perception of the lips, face, mouth, and hands in comprehending language. Third is the nature of the inter-relationship between the systems - are they two systems or part of some intertwined system? To what extent are they separable? Fourth are the theoretical constraints on investigations of language that depend on the separability of these systems, particularly in functional brain imaging. This lecture will discuss these issues and conclude with a number of practical approaches to functional brain imaging within the framework of ecological and situated language understanding.
It is becoming increasingly clear that brain areas once thought to be language-specific are active in various cognitive and sensorimotor tasks, and that damage to such areas can also produce non-linguistic deficits. But how these neural resources are utilized by similar linguistic and non-linguistic processes is far from clear. Here, we will present results from a number of recent studies on the relationship between processing of complex, meaningful environmental sounds and language, including our own data from normals, aphasic patients, and neuroimaging. In general we find that the processes subserving closely related linguistic and non-linguistic auditory tasks overlap to a surprising extent.
In addition to organized sequences of speech sounds (phonemes), verbal utterances are
characterized by a distinct set of suprasegmental features ("speech melody") contributing to
the meaning of a sentence (linguistic prosody) as well as to a speaker's emotional expression
(affective prosody). For example, sentences spoken in a happy tone exhibit, as a rule, larger
pitch fluctuations and higher amplitudes than sentences spoken in a sad mood. Besides
intonation, the temporal pattern of verbal utterances, i.e. their rhythmic structure and the
frequency, contributes to these prosodic aspects of acoustic communication. Based on a series
of case studies, Ross (1981) assumed the right hemisphere (RH) to mediate processing of
speech prosody. In analogy to the classification of the various syndromes of aphasia bound to
left-hemisphere (LH) damage, expressive (motor) and receptive (sensory) variants of
aprosodia were assigned to predominant right-sided anterior or posterior perisylvian lesions.
However, subsequent clinical investigations yielded both in the domains of linguistic and
affective prosody discrepant data with respect to the neural systems underlying prosody
perception and production (for a review see Van Lancker & Sidtis 1992). Conceivably, the
various relevant acoustic cues such as fundamental frequency (F0, the acoustic correlate of
pitch), sound intensity (loudness) and the temporal processing of language (rhythm) are
encoded by differently lateralized cerebral networks.
In order to further delineate the contribution of different cerebral networks, functional
magnetic resonance imaging (fMRI) was performed during (a) acoustical perception / oral
production of isochronuous syllable repetitions and syllable triplets with lengthening either of
the initial or final unit, (b) acoustical perception of click stimuli, and (c) oral production of
syllable repetitions at different production rates (frequency range 2-6 Hz).
References
1. Van Lancker D and Sidtis JJ. 1992. The identification of affective-prosodic stimuli by left-
and right-hemisphere-damaged subjects: all errors are not created equal. J Speech Hear Res
35, 963-970.
2. Ross ED. 1981. The aprosodias. Functional-anatomic organization of the affective
components of language in the right hemisphere. Arch Neurol 38, 561-569.
Mirror neurons were first described in the ventral sector of monkey pre-motor area (area F5). These neurons discharge both when the monkey performs a specific goal-directed hand action and when it observes the same or a similar action performed by an experimenter or a conspecific. It has been proposed that they play a major role in action understanding.
A mirror system also exists in humans. Using transcranial magnetic stimulation (TMS) it was demonstrated an increase of motor evoked potential recorded from hand muscles during the observation of hand actions. Experiments carried out with different techniques (MEG and EEG) showed an involvement of motor areas during both observation and execution of hand actions.
By means of functional magnetic resonance imaging (fMRI), it was shown that during the observation of various actions performed with different effectors (hand, foot, and mouth), there is an activation in the observer of distinct sectors of the pre-motor area, according to the effector used in the observed action. These activated sectors overlap those where a motor representation of the involved effector was classically described. More recently, in an fMRI study, subjects were asked to carefully observe mouth actions performed either by humans or by non-conspecifics (a monkey and a dog, respectively). Common areas were active in both conditions, although to a less extent during the observation of actions performed by non-conspecifics.
Broca’s region, which occupies the posterior part of the left inferior
frontal gyrus, is a dedicated language region. Cytoarchitectonic areas BA44
and 45 are assumed to be the anatomical correlates of Broca’s speech
region. We have developed and applied a method for the combination of SPM99
analysis of fMRI data of a verbal fluency task with a non-SPM, elastic
warping algorithm and cytoarchitectonic maps in order to test the
hypothesis that both left BA44 and 45 participate in verbal fluency.
Methods: Cytoarchitectonic mapping was performed in 10 human brains using
an observer-independent approach for the definition of areal borders [1].
fMRI was performed in 11 normal volunteers who were asked to covertly
produce words [2]. In the non-switching trials, subjects responded with a
series of items from a pre-specified category and from over-learned
sequences. In the switching condition, subjects produced words alternately
from three semantic categories, and from three over-learned sequences. The
design of the experiment was factorial. We focus here on the non-switching
condition. A non-SPM elastic warping algorithm was applied to enable the
transformation of postmortem data, anatomical MR and echo planar imaging
data of the verbal fluency task to a non-SPM reference brain [3].
Probability maps of BA44 and 45 were calculated and superimposed on the
resulting maps of significantly activated areas as obtained by SPM analysis.
Results & Conclusions: Increases in neural activity overlapped with left
BA44 and 45 when fluency was compared with rest. Word production from
superordinate categories thus seems to activate both BA44 and 45. When
semantic fluency was compared with over-learned sequence fluency,
activations overlapped with left BA45. Thus left BA44 and 45 both
participate in verbal fluency, although in different ways. The combination
of SPM analysis with a new elastic warping tool and cytoarchitectonic maps
opens new perspective for analyzing cortical networks involved in language.
References:
[1] Amunts et al. (1999) JCN 412:319
[2] Gurd et al. (2002) Brain, 125:1024
[3] Mohlberg et al. (2002) Neuroimage. Suppl. 492
The UCB/ICSI NTL project has been developing an explicitly neural theory
of language. The core premise is that language is largely determined by
the computational character of neural networks, the structure of our brains,
and our interactions with the physical and social environment. The talk
will cover recent results and their experimental and linguistic consequences.
The Embodied Construction Grammar (ECG) formalism represents
form-meaning relations in a way that supports both neural and conventional
computer enactment. Many linguistic issues that seemed intractable can be
treated in ECG. The related theory of understanding as imaginative
enactment is being tested in behavioral and imaging experiments.
Earlier computational models of word learning have been extended to a system
that models how children learn grammatical constructions as linking sounds
with their experience, including motor activity.
There are quite a number of studies investigating the neural basis of sentence reading. This presentation will focus on syntactic and prosodic processes during auditory sentence comprehension. Neurophysiological studies focusing on the temporal structure of processing demonstrate that syntactic processes although being independent of semantic processes during an early processing phase interact with semantic information during a later processing stage, and that prosodic information can influence phrase structure building processes. Brain imaging studies reveal that both temporal and inferior frontal regions of the left hemisphere support syntactic and semantic processes: while the anterior portion of the superior temporal gyrus in conjunction with the frontal operculum (inferior BA44) are involved in syntactic processes, the posterior portion of the superior temporal gyrus together with the pars triangularis (BA45) serve semantic processes. Prosodic processes, in contrast, are mainly supported by the right hemisphere, again with an involvement of temporal and inferior frontal regions. These data together with those in the literature suggest that temporal and frontal brain region in concert serve syntactic, semantic and prosodic information processing. Interhemispheric communication may hold responsible for the observed interaction of syntactic and prosodic information.
Computer models based on evolutionary neural networks and artificial life
can be used to simulate the emergence of language in populations of
sensorimotor agents (1). Using these models it is also possible to
investigate the interaction between language and other abilities. The
analysis of the agents' neural networks can highlight the neural mechanisms
responsible for the integration of linguistic, cognitive and motor
abilities. For example, it has been shown that the effects of categorical
perception vary when agents evolve language. The similarity space of
proto-verbs is enhanced and optimized with respect to that of proto-nouns
(2). In addition, the relationship between language processing and
sensorimotor abilities has been studied using the method of synthetic brain
imaging (3). Analyses have shown that the neural representations of
sensorimotor categories and syntactic word classes are sensitive to the
level of integration of linguistic information and sensorimotor knowledge.
In these models, the networks show functional organisations that reflect
those observed in human experiments on brain imaging of language processing
(4). The implications of such data and models for testing the role of
sensorimotor integration in the evolution of language and syntax will be
discussed.
References
(1) Cangelosi A. IEEE Trans. Evol. Comp. 2001; 5: 93-101.
(2) Cangelosi A, Parisi D. In Proc. 23rd Cog. Sci. Soc. 2001; 170-175.
(3) Arbib M et al. Neural Networks 2000; 13: 975-997.
(4) Perani D et al. Brain 1999; 122: 2337-44.
We address the possibility of combining the results from hemodynamic and electrophysiological methods for the study of cognitive processing of language. Our ongoing research is focused on language production; the hemodynamic method we use is the Event Related fMRI , and the electrophysiological method measures Event Related EEG Band Potentials. The experimental technique allows us to approach the relation between cortical structure and cognitive function in a new, sophisticated way. In particular, we can formulate original working hypotheses about the role of distributed networks and central processors in language production. We illustrate these hypotheses, and how we test them, with respect to the processing of sound structure (fMRI data) and to syntactic and semantic transformations (combined fMRI/ERBP data). References Dogil, G. et al, 2002. The Speaking Brain. Journal of Neurolinguistics 15, 59-90. Röhm, D. et al, 2001. The role of theta and alpha oscillations for language comprehension in the human EEG. Neuroscience Letters 310, 137-140.
Direct linking of perception with action requires an analysis-by-synthesis model of language. Such an approach has a long and respectable history in speech perception, but has a more checkered history in the area of sentence perception. I argue that an analysis-by-synthesis model of sentence perception is highly feasible, and that it obviates the need for the traditional distinction between competence systems and performance systems for grammatical knowledge. To this end, I present evidence that grammatical competence is better understood as a real-time computation, and that real-time parsing processes show the hallmarks of grammatical sophistication. I also suggest that classic prefrontal language areas of the brain may not be the locus of long-term grammatical information storage, but rather the locus of real-time compositional processes, for grammar and for other domains alike.
This talk will focus on the role of the different parts of the inferior frontal gyrus in various aspects of language functioning (both comprehension and production) and in the production of oral and limb movements. For example, we will demonstrate differences between two cytoarchitectonically-defined subareas of Broca's region (Brodmann areas 44 and 45) in the production of oral speech, American sign language, and complex movement generation. The role of neural functional connectivity, as assessed with functional neuroimaging data, will be stressed. We also will indicate how large-scale, neurobiologically realistic computational models, when coupled explicitly with functional brain imaging data, can be used to help determine the neural substrates of human cognitive behaviors, especially those involved with language.
I present a neural network model for sensori-sensor and sensori-motor integration. The model is based on simple, neurobiologically plausible mechanisms. In modelling sensori-sensor integration, visual and auditory perception are combined to form a unified percept. This process leads to the simultaneous formation of prototypes in each domain, and to interference effects between the domains such as the McGurk effect. The model is extended to one of sensori-motor integration between speech motor commands and their associated sounds. Here, the model develops perceptual and motor prototypes based on the correlational structure between the domains, but also based on the external language environment. In accordance with the ideomotor principle, the model learns to imitate sounds based on the close coupling between perception and production. The model develops motor mirror neurons that are activated when their associated sounds are perceived. An extension of the model to account for visual mirror neurons is suggested.
Two fMRI studies were carried out with the goal of identifying brain regions used in phonetic perception. In a first exeriment, candidate brain regions constituting a neural network for preattentive phonetic perception were identified with fMRI using multiple regression of imaging data. Stimuli contrasted along speech/non-speech, acoustic or phonetic complexity (3 levels each), and natural/synthetic dimensions. Seven distributed brain regions' activity correlated with speech and speech complexity dimensions, including 6 left-sided foci (posterior superior temporal gyrus (STG), angular gyrus, ventral occipitotemporal cortex, inferior-posterior supramarginal gyrus, middle frontal gyrus (MFG)) and 2 right-sided foci (posterior STG and anterior insula). Only the left MFG discriminated natural and synthetic speech. The data also supported a parallel rather than serial model of auditory speech and non-speech perception given that areas sensitive to acoustic spectral complexity were not active during the correlated acoustic/phonetic complexity dimension in the speech. In our follow up experiment, speech and nonspeech were comprised of acoustically matched sinewaves (SW) differing in perceived phonetic content. Chord progressions were also included along with a "silent" baseline. Eight regions were more active for SW speech than SW non-speech. Four of the 8 regions, i.e., left STG/MTG, left IFG, left MFG and right STG/MTG, showed greater activation for speech than all of the other conditions (Sw non-speech, Chords, Silence). All four of these foci were selectively activated by the SW speech stimuli, i.e., neither of the other two conditions (e.g., Swnon, chords) was activated above the silent condition. Each of these four regions was also active in the first experiment for the speech vs. non-speech contrast. Interestingly, each of these foci was contiguous with a region which was active for at least one of the other conditions. This latter result may suggests that speech recognition is a specialization of more general auditory processing. This does not imply, however, that a full auditory analysis is necessary for speech recognition. Results from the first experiment refute the serial processing model. In conclusion, a network consisting of four perisylvian cortical regions appears to be involved in speech perception.
Infants are faced with difficult computational challenges in learning to understand and produce spoken language, due to the highly variable, temporally extended nature of speech and the absence of direct articulatory feedback for production. To address these challenges, a distributed connectionist framework is proposed in which a common set of phonological representations learn to mediate both comprehension and production. Initially, these representations are shaped primarily by the demands of mapping acoustics to semantics during speech comprehension, but become increasingly refined by articulatory factors as production skills improve. Corrective feedback on speech production is derived from the acoustic and phonological consequences of the system's own articulations, via a learned, internal "forward" model of the physical processes relating articulation to acoustics. An implementation of the framework, in the form of a discrete-time simple recurrent network, learned to comprehend, imitate, and produce a corpus of 400 monosyllabic words, and its errors in development showed similar tendencies as those of young children. Although only a first step, the results provide support that the approach may provide the foundation for a comprehensive account of phonological development.