Algorithms that mimic the computation and learning capabilities of the human brain are feasible solutions to many information-processing problems. We present a theoretical model based on the observation that images of similar visual perceptions reside in a complex manifold in an image space. To model the pattern manifold, we present a novel learning algorithm using a recurrent neural network based on the behavior of the brain. In designing a recurrent neural network, convergence dynamics of the network needs special consideration. We propose to modify this picture: If the brain remembers by converging to the state representing familiar patterns, it should also diverge from such states when presented with an unknown encoded representation of a visual image belonging to a different category. Based on this, we have developed a self-organizing line attractor to learn new patterns. A nonlinear dimensionality reduction technique is used to embed the points to a lower dimensional space that preserves the intrinsic dimensionality and metric structure of the data to enable fast and accurate recognition. Experiments performed on UMIST, CMU AMP, FRGC version-2, Japanese female face expression, and Essex Grimace databases show the effectiveness of the proposed approach in accurate recognition of complex patterns.