Artificial Neural Networks and Computational Brain Theory Group

Please join us for our last meeting of the semester - we will be discussing plans for next semester, and ways to make the group better.

In recent years, substantial progress has been made in elucidating the neural s ubstrates of selective attention. Much of the research has concerned the roles of, and interactions between, anterior and posterior brain regions, but relatively little interest has been given to the question of how interactions between the cerebral hemispheres are related to attention. With Marie Banich and Michael Coles, I have been investigating the relationships between interhemispheric interaction and selective attention using electrophysiological recordings in humans. I will review some of our findings and discuss how they relate to current views of the neural bases of selective attention.

The function of the VOR (vestibulo-ocular reflex) is to stabilize vision by making eye rotations that oppose head rotations. Like many other sensorimotor systems, the response of the VOR will decrease over time if it is presented with a prolonged stimulus. This response decrease is due to VOR habituation. Recent work by Ernst Dow in my lab has uncovered an array of linear and nonlinear phenomena that are associated with VOR habituation. A model that unifies these observations remains elusive. In this informal talk I will present the data and suggest some possible avenues by which a model of habituation may be approached.

Note room change !!!

This paper is a recent discussion of ideas presented in the well known paper: McClelland, J.L., R.C. O'Reilly, and B.L. McNaughton, 1995, Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights From the Successes and Failures of Connectionist Models of Learning and Memory, Psychological Review 102 no.3:419-457

I have left a copy in the DCL and Beckman folders for optional background reading. For more computational models of hippocampus see Hippocampus vol.6 no.6.

Whilst cases are of great significance in Common Law, there has been minimal research about knowledge discovery in the legal domain. We claim for knowledge discovery to be feasible, the domain must contain an abundance of commonplace cases. We discuss in detail a hybrid rule-based/neural network system, Split_Up which provides advice upon the distribution of property following divorce in Australia. Explanations in Split_Up are provided using the argumentation theory of Stephen Toulmin. We also discuss an in detail evaluation of the Split Up system and consider user interface issues.

Abstract: What is attention? Science typically studies natural phenomena, abstracting the essential aspects from nature and importing them into the laboratory, where they are subject to experimentation. What is the natural phenomenon of attention? Why does it matter what the natura l phenomenon of attention is? Because a good theory explains the natural phenomenon as well as the experimental data. The scientific concept of attention needs to be grounded somewhere, in order to have meaning beyond the operations used to study it in the laboratory. If we can identify the natural phenomenon, we can use it to assess the validity of our theories. The ability to explain the natural phenomenon becomes the acid test. I will explore the possibility that the natural phenomenon of attention is a social. It is something we see and make inferences about in other people. The behaviors that people manifest when we think they are attending are the natural phenomena of attention. The mechanisms of attention are the mechanisms that support these social-behavioral displays of attention. I will try to characterize the broad features of social displays of attention and explore the implications for current issues in the study of attention, contrasting the social approach with mathematical, connectionist, and neuroscientific approaches to attention.

This will be a brief meeting where we will discuss plans for the upcoming semester and welcome visitors. We encourage anyone interested in the group to attend. Refreshments will be served.

Signal averaging makes it possible to observe brain activity time locked to an event in intact, behaving, human subjects. The Event Related Brain Potentials (ERPs) recorded in this manner allow a study of brain function with very fine temporal resolution but with rather poor spatial resolution. Given that different components of the ERP are manifestations of the activity of various processing modules, the challenge is to identify the functional significance and the intracranial loci of the manifested activity. How this can be done will be illustrated by an examination of ERP components that seem to manifest the activity of what might be called the "Homonculus Bureacraticus".

All sensory systems are faced with the task of extracting behaviorally-relevant information from complex sensory environments. This talk will describe our ongoing experimental and modeling studies of the sensory acquisition process in the electric sense of weakly electric fish. Based on infrared video recordings of prey capture behavior in these nocturnal fish, we reconstruct spatiotemporal patterns of electroreceptor activation and relate these patterns to the filtering properties of sensory neurons in the central nervous system. I will speculate on how motor strategies and adaptive filtering properties may work together in this system to enhance signal detection capabilities.

The superior colliculus in mammals (and its homologue in nonmammals, the optic tectum) is a structure in the brain which has been studied extensively for its ability to combine information from different sensory organs. Two models of the superior colliculus are presented. One model consists of a network of perceptrons with a continuous nonlinearity, and a second model consists of explicit representations of dendrites using a discrete version of an RC network. The parameters of these models are trained to synchronize the inputs from different sensory modalities with a novel unsupervised learning algorithm.

This paper is available on-line at http://www.cnbc.cmu.edu/~plaut/papers/PlautKelloINPRESSchap.phon.ps.gz.

"people need not read from the Representations section, from 1/3 of the way down on P. 8, to 1/3 of the way down on P. 13 (it's psycholinguistic stuff that's not relevant to how we'll want to discuss it)."

Neurogen is an architecture that combines benefits (and some drawbacks) of Neuronic and Genetic adaptive learning. The presentation on Thursday will be an explanation of this architecture, followed by a discussion of its possible applications, improvements and bottlenecks.

Questions are welcome before, during and after the talk. This is a work in progress.

Two of the most powerful low-level (and hence tractable!) image processing techniques are clustering and dimensionality reduction. Clustering represents nearby data points with a prototype, whereas dimensionality reduction captures the underlying degrees of freedom in the data. In this talk, I will review these two ideas and their associated algorithms (K-means clustering, mixture-of-Gaussians, principle components analysis, "eigenface" decomposition) and describe a model called a "mixture of factor analyzers" that merges clustering and dimensionality reduction. Each so-called factor analyzer describes a Gaussian distribution in a low-dimensional plane in the input space, and a mixture of these models renders a highly flexible globally nonlinear manifold. This model can be fit to data using the expectation maximization algorithm. I will present an extension of this model that allows different clusters to share directions of variation. This can act as a regularizer that prevents over-fitting of the training data and captures local coherence between the orientations of the linear subspaces. I will present experimental results on small problems throughout the talk to illustrate how these algorithms work, and I will describe recent work on applying these ideas to pattern classification and detection.

This talk will be concerned with the mapping of function to neuroanatomy relative to brain mediation of learning and memory. The ideas to be presented will be based on studies of multi-site neuronal activity recorded during discriminative instrumental learning in rabbits. The basic premise of the talk will be the idea that the behavioral manifestations of learning and memory emerge as a result of information flow and interactions among interconnected brain regions which form the "nodes" of a larger circuit. The main thrust of the talk will be to suggest and document specific mnemonic functions of the following nodes of the limbic memory circuit: cingulate cortex and related areas of thalamus; amygdala; hippocampus; striatum.

Association connections between different areas of cat visual cortex are not spatially continuous, but generally originate from and terminate in what appear to be random patches. The focus of this talk is the patchy pattern of projections from area 18 (a lower-order area of visual cortex) to the lateral suprasylvian visual area ("LS"; a higher- order area of cortex). We present physiological and anatomical evidence that the visual responses of cells in LS patches receiving input from area 18 depend most strongly on the dorsal layers of the lateral geniculate nucleus (LGN), as do the responses of the cells in area 18 that project to these patches. We also found that there tends to be a reciprocal relationship between the extent to which cells at a given site in LS depend on dorsal LGN layers, and the extent to which they depend on a subdivision of the LGN called the medial interlaminar nucleus (MIN). We have previously shown that the MIN is a night-vision specialization, for which high afferent convergence from the retina maximizes sensitivity at the expense of spatial acuity. We propose that independent, partially overlapping patterns of patchy control from the dorsal LGN layers (a high acuity pathway) and the MIN (a high sensitivity pathway) allow a functional competition between these two geniculate relays for control of cortical resources. The balance of inputs to most LS cells would dynamically shift according to light levels, optimizing the trade-off between acuity and sensitivity for the current conditions.

This talk addresses the nature of MODULARITY in neural systems. Part of the talk will deal with the important issues of mixture modeling for sensor and data fusion, and will be devoted to discussion of how the "mixtures of experts" paradigm may be refined and applied to spatiotemporal sequence learning, multimodal integration, and other model-building problems of interest.

In order to bridge this talk with our next discussion, I will first introduce some basic concepts in the "hierarchical mixtures of experts" (HME) paradigm of Jordan and Jacobs, and will survey a number of HME applications by them and their colleagues (Hinton, Barto, Neal, MacKay, etc.). To illustrate the practical applicability of modular neural networks, I will give two example problems (multimodal musical signal identification and agricultural drought monitoring) to which modular mixtures have been applied. Next, I will briefly present mixture modeling aspects of my continuing dissertation research on spatiotemporal sequence learning.

I will then outline a specific aspect of ST sequence learning: extracting CAUSALITY for monitoring and diagnosis, using mixtures of temporal, probabilistic network models. This discussion will focus on two new developments in Bayesian inference which pertain to modular Bayesian learning: a proposed temporal extension of Heckerman's PROBABILISTIC SIMILARITY NETWORKS, and a framework for BAYESIAN LEARNING in ANNs, recently refined by Neal and MacKay. Finally, I will explore the implications of modularity in Bayesian inference and learning (i.e., "locality" and "detachment"), and consider the potential existence of biological analogues.

When people speak, they will tend to repeat the structures of previously uttered sentences even when the sentences differ in prosody as well as in lexical and conceptual content. This is called structural priming. For example, you are more likely to say the passive sentence "A policeman is being hit by an ambulance" rather than an active sentence like "A ambulance hits a policeman", if you had just said another passive sentence beforehand. Our account of this phenomena is that it is the same mechanism that people use to learn language in the first place (implicit learning). To examine this hypothesis, we built a connectionist model of sentence production (using a SRN-like network), which learns to produce sentences from a small grammar. Interestingly, the model required a comprehension task before it could develop representations which would show structural priming. There is a poster describing this model outside of Beckman 1412 and at http://dasparc.cogsci.uiuc.edu/~fchang/comppsych/

Work is presented describing a visual navigation system hosted on a small Khepera robot using flying insect inspired behavior. Using monocular non-directional movement detectors, the robot is able to navigate a field of obstacles. We show by analysis that in a real system (animal or robotic) a blind spot is always present for motion-based obstacle detection. By articulating the body trajectory, i.e. making zig-zag movements, this blind-spot can be functionally eliminated. This behavior as well as an optomotor response and haltere-ocular response are modeled in a neurally plausible way and implemented on a real robot.

In this talk we discuss a "response threshold" model of division of labor in social insects, such as ants and honey bees. To help understand the implications of this model for social insect colonies, we develop and explore an analogy between social insect workers and neurons, and ask how threshold-mediated responses contribute to the functions of workers and neurons within their respective higher level systems.

The general patterns of division of labor in social insects are well known, but little is known about the underlying behavioral rules that generate and maintain the division of labor, the organization that results from these rules, or the functional significance of the either the rules or the organization. Experiments suggest that workers vary in their responsiveness to the stimuli associated with different tasks. A model based on response thresholds that vary among workers can potentially explain observed patterns of behavioral specialization and flexibility among individual workers, and the dynamic stability of division of labor at the level of the colony.

After giving a formal statement of the threshold model, we use the analogy with the nervous system to suggest further questions about how the distribution of thresholds among the workers affects a colony's performance, how thresholds can be modulated, how worker-worker interactions may be involved in modulation of thresholds, and what patterns of "connections" may be found among the workers in a colony.

First Fall'97 Meeting

Our first meeting of the semester is on Friday, September 5, at 3pm, in 2240DCL.
This is an introductory meeting where we will be discussing our plans for the fall semester; we encourage anyone curious about the group to attend.

Open Talk
Title:     "The Hippocampal Formation: an Anatomical Framework for Computational Modeling"
Speaker:   Dr. Paul Patton, Mercer University School of Medicine, Division of Basic Sciences
Date:      Sept. 3 (wednesday)
Time:      4pm - 5pm
Location:  2269 Beckman

The hippocampal formation presents a special opportunity for realistic neural modeling, since its structure, connectivity, and physiology are better understood than that of other cortical components. The first step towards developing such a model is the creation of a framework based on quantitative neuroanatomy. I will discuss the quantitative neuroanatomy of the dentate gyrus and hippocampal area CA3 in the context of the creation of such a framework. For the dentate gyrus, the model includes the principal cells: the granule cells, as well as a number of non-principal cell classes: the basket cells, chandelier cells, mossy cells, and GABAergic peptidergic polymorphic cells.

First Summer Organizational Meeting
Date:          July 9 (wed)
Time:          4pm
Location:      3262 DCL

Attendees are requested to bring 5-6 copies of a paper suitable for next semester's discussions.

Open Talk
Date:		May 7 (wed)
Time:		4pm
Location:	2269 Beckman
Title:		"Lexical access in normal and aphasic speakers"
Speaker:	Gary Dell
Abstract:	

Difficulty in word retrieval is the most pervasive symptom of language breakdown in aphasia. I will present an interactive activation account of this difficulty, one that explains the picture-naming error patterns in aphasic and normal speakers. The theory uses spreading activation in a lexical network to accomplish the mapping between the conceptual representation of a pictured object and the phonological form of the word naming the object. A model developed from the theory was first parameterized to fit normal error patterns. Then it was "lesioned" by globally altering its connection weight and/or decay rates to fit the error patterns of 21 fluent aphasic patients. These fits were then used to derive predictions about other aspects of the patients' behavior. The predictions were confirmed. I argue that simple quantitative alterations to a normal processing model can explain much of the variety among patients' error patterns.

Open Talk
April 30 (wed.), 1997
4pm - 5pm
2269  Beckman
"From Competence to Efficiency: A Tale of GA Progress"David Goldberg

Genetic algorithms are increasingly being used to solve difficult problems in search, optimization, and machine learning, but empirical results in applications are hit and miss. This talk explores theory and computational experiments that help explain why this is currently the case and presents a methodology and particular GA designs that overcome the
difficulty.

Specifically, competent GAs are presented that appear to solve boundedly difficult problems in a PAC-like sense in polynomial times, including messy GAs, gene-expression messy GAs, and linkage learning GAs. As these results are consolidated and carried to practice, it becomes important to move beyond competence and explicitly demand increased efficiency. The talk concludes by briefly considering new results in efficient parallelization, evaluation relaxation, and hybridization.

Open Talk
April 23 (wed.), 1997
4pm
2269 Beckman
"Induction and modeling of periodic alternating nystagmus in intact goldfish"Ernst R. Dow

Nystagmus is a pattern of eye movement consisting of smooth, slow-phase eye rotations in one direction and fast-phase, resetting eye rotations in the opposite direction. Slow-phase eye rotations can be driven by various oculomotor sub-systems including the vestibulo-ocular reflex (VOR). The VOR maintains retinal image stability by making slow-phase eye rotations
that counterbalance head rotations. Periodic alternating nystagmus (PAN) is a congenital or acquired eye movement disorder characterized by uncontrollable nystagmus that alternates direction roughly sinusoidally with a period of several minutes. Researchers have been able to induce PAN in monkey by lesioning the cerebellar nodulus and uvula. We have been able to induce PAN reliably in an intact animal for the first time. Prolonged whole-body rotation of goldfish at specific frequencies (0.05 to 0.1 Hz) in the dark induces PAN, upon which the normal VOR response is superimposed. PAN is infrequently observed at lower frequencies (0.01 and 0.03 Hz) where the process of habituation severely decreases the VOR response to prolonged rotation. Also, PAN is not observed with rotation at higher frequencies where habituation does not take place. The PAN oscillations range in period from 100 to over 1000 sec and can have a peak to peak amplitude of over 95 deg/sec. In some instances, up to 2 cycles of PAN are observed after rotation has ceased. Previously, researchers have reproduced PAN using a nonlinear, limit-cycle model consisting of an unstable velocity storage loop and a central adaptation loop in combination. We were able to simulate rotation-inducable PAN by adding a threshold nonlinearity to the velocity storage loop of this model. By combining these results with our previous studies on habituation of the VOR, we hope to synthesize a more complete model to better understand the neural mechanisms of this adaptive process.

Discussion
April 16 (wed), 1997
4pm,2240 DCL
paper discussion led by Chris Seguin
"Fuzzy ART: Fast Stable Learning ad Categorization of Analog Patterns by an Adaptive Resonance System"
 - Gail Carpenter, Stephen Grossberg, and David Rosen

Open Talk
April 9 (wed.), 1997
4pm
2269  Beckman
"BROKEN SYMMETRY AND HIGH-ORDER DYNAMICS IN THE LINEAR NETWORK MODEL OF THE OCULOMOTOR NEURAL INTEGRATOR"
Tom Anastasio

The integrator of the oculomotor system is a neural network that takes in eye velocity signals and puts out eye position commands that control the position of the eye in the orbit. The oculomotor neural integrator has been modeled as a network of linear elements by Cannon, Robinson, and Shamma. Push-pull inputs to the network alternate regularly, and all neurons make lateral inhibitory connections to each other with the same Gaussian profile.This system symmetry balances network activity and allows it to integrate the push-pull component of its input but not the spontaneous carrier. These are desirable features that the model shares with the real neural integrator. The dynamics of all neurons in this network are identical and are governed by a single time constant (order one). These features are unrealistic, because neurons in the real integrator vary in their dynamics, which appear to be governed not by one but by a large number of time constants (high order) that are distributed in value over several orders of magnitude. The purpose of the work to be described is to use linear systems analysis to show that small breaks in the symmetry of the integrator network allow it to retain its desirable properties while at the same time increasing the variability and the order of the dynamics of its constituent neurons. It will be argued by comparison with neurophysiological data that the network with broken symmetry is a more realistic model of the real oculomotor neural integrator.

Discussion
April 2, 1997
4pm
2240DCL
paper discussion led by Bill Hsu"Toward a Theory of Learning and Representing Causal Inferences in Neural Networks", George E. Mobus.From /Neural Networks for Knowledge Representation and Inference/, 1994, edited by Daniel S. Levine and Manuel Aparicio IV.

Discussion
Mar 19 (wed), 1997
4pm
3262 DCL
paper discussion led by Sam Beshers
"Design and Evolution of Modular Neural Network Architectures" - Bart Happel and Jacob Murrefrom Neural Networks, vol.7,no.6/7

Event
Mar 14&15, 1997
Beckman Institue
Engineering Open House and Cyberfest

Bill Hsu, who was in charge of the ANNCBT presence at Cyberfest/EOH reported that we had
nearly 300 attendees saturday, and over 150 on Friday, with an estimated talk audience of
about 150 people (we delivered about 15 mini-talks to an average audience of 10).

Discussion
Mar 5 (wed), 1997
4pm
2240DCL
paper discussion led by guest faculty"Autopoiesis and Cognition" - Humberto Manturana & Francisco Varela
"On Constructing Reality"

Discussion
Feb.19 (wed.), 1997
4pm
2240DCL
paper discussion led by Jesse Reichler"Selective Recognition Automata", Reeke, Finkel, and Edelman,  
from Neural Connections, Mental Computations, MIT Press 1990

Organizational Meeting
First Meeting of Semester Spring '97
Feb 5, 19974pm
3262 DCL

Open Talk
Dec 4, 1996
"Designing Metabolism:  A Case Study"
Jay Mittenthal

To understand a biological organization one can model its evolution or analyze its design. We have taken an approach through design to understanding the connectivity of a biochemical network. It interconverts five 6-carbon compounds and six 5-carbon compounds. We generated an ensemble of alternative networks and evaluated their performance through a sequence of stages. First we found networks that interconvert the carbon skeletons, ignoring side groups. The networks can be classified into at least 53 families in at least 7 superfamilies, according to the number, input-output relations, and internal structure of their modules. Constraints of the problem generate formulas that specify the families and superfamilies. We then assigned enzymes from known classes to mediate transformations of carbon skeletons and modifications of side groups. As far as we have been able to evaluate the ensemble of alternatives, the real network is optimal. Most favorable alternatives use an enzyme that mediates several side-reactions, reducing the rate of flux through the networks. Analogous approaches through design may aid understanding of real neural networks.

Demonstration
Nov 27, 1996 (wed.)
2240 DCL
4pm
Demonstration on "Perceptual Control Theory."
Gary Ciko

Discussion
Nov. 13 (wed.), 1996
4pm
2240DCL
"A General Feedback Theory of Human Behavior", Powers, Clark, & McFarland, 

Video Showing
Nov. 6 (wed.) 1997

3-5pm
2240 DCL
2 genetic programming videos (from MIT press), in 2240 DCL.

Open Talk
Oct. 30 (wed.), 1997
4pm 
3269 Beckman
"Temporal Sequence Learning and Recognition: Two Network-style Algorithms."
Sylvian R. Ray

Storage(learning), recognition, and retrieval operations with temporal sequences pervade many areas of engineering interest, for example, speech and speaker recognition, music recognition and biomedical signal processing.
We will discuss two network-style algorithms and their application to this type of problem.
The first algorithm is based on an adaptation of reaction-diffusion, a process which has been offered (by A. Turing) to explain the growth of patterned sequential structures in biological systems.
The second algorithm uses a forced unique transition scheme imposed on a Kohonen map, plus 'chunking', to produce hierarchical storage and replay of long sequences, such as complete musical scores.

Discussion
Oct. 23 (wed.), 1996
4pm
2240DCL
"Impairments of Facial Recognition", from Explorations in Cognitive Neuropsychology, by Alan Parkin

Open Talk
Oct. 9 (wed.), 1996
4pm
3269 Beckman
"Topology Representing Networks for Sensor-based Path Planning"
Michael Zeller

An important step toward autonomous robotics is developing ways to generate motion plans for achieving certain goals while satisfying environmental constraints. We present a framework for sensor-based motion planning of robotic manipulators using a Topology Representing Network (TRN). Exploiting the perfectly topology preserving features of the network, the algorithm learns the representation of the Perceptual Control Manifold (PCM), a recently introduced concept for motion planning. This concept allows sensors to be integrated into robot motion planning. A diffusion-based path planning strategy leads to flexible obstacle avoidance. Besides a demonstration of the technical feasibility of motion planning through perfectly topology preserving maps, the capabilities of this approach within an engineering framework, namely the implementation on a pneumatically driven robot arm (SoftArm), are demonstrated.

Organizational Meeting
Sept. 5 (thur.), 1996
2240 DCL
1pm

Open Talk
July 30 (tue.), 1996
2pm
3269 Beckman
"A Feedforward Neural Network Model to Describe Habituation of the Goldfish Vestibulo-Ocular Reflex"
Ernst Dow

The vestibulo-ocular reflex (VOR) is responsible for maintaining the direction of gaze during head rotations. When vertebrates are exposed to low frequency sinusoidal rotations in the dark, the VOR habituates, presumably from Purkinje cell inhibition of the vestibular nuclei. Habituation is observed as a decrease in gain (response amplitude / stimulus amplitude). The goldfish habituates quickly displaying several non-linear phenomena. A simple, non-linear feedforward neural network was constructed to model the habituating response. All of the non-linearities observed before and during habituation can be simulated by varying only the Purkinje cell firing rate.

DiscussionJuly 7 (tue.), 1996
2:30pm
2240 DCL.
"Distributed AI", by Edmund Durfee, from Arbib's Handbook of Brain Theory and Neural Networks

First Open Talk
July 16 (tue.), 1996
2pm
3269 Beckman
"Probabilistic Reasoning and Bayesian Networks"
William H. Hsu

This talk consists of a brief introduction to the probabilistic foundations of uncertain reasoning, and presents Bayesian networks, a very useful formalism for practical modeling of inference and learning in intelligent systems. First, I will outline the causal and associative reasoning principles surrounding Bayesian networks, and discuss their statistical foundations; architecture, function, dynamics; and formal semantics. Next, I will survey some major applications of Bayesian networks to problems at which artificial neural networks excel, and present current research in integrating the two models. We will then look at some crucial issues in machine learning, pattern recognition, and problem solving using Bayesian network approaches. Finally, we will consider future work in computational neuroscience that can be strongly supported by probabilistic modeling.