A new class of convolutional neural networks based on shunting inhibition with applications to visual pattern recognition

In the contemporary era of increased information overload, there is a growinginterest in a new class of computational intelligence systems. These systems have been proven as powerful and versatile computational tools for solving certain types of problems that are too complex to be analyzed with traditional analytical means. Inspired by the computational mechanism of the human brain, many researchers have looked into neurobiology for new inspiration to solve more complex problems than those based on traditional computational techniques. Artificial neural networks, evolving from neuro-biological insights, give computer systems an amazing capability to actually learn from input data to generate solutions for problems that are too abstract to be understood or too resource-intensive to tackle. Although neural networks have been applied with success in many industries, there is a continuing demand for new types of hierarchical artificial neural networks that can overcome some of the drawbacks of the earlier models. This thesis presents a new class of convolutional neural networks based on the physiologically plausible mechanism of shunting inhibition with its various systematic connection schemes. The network has a generic architecture in which shunting inhibitory neurons are used as feature extraction elements. A series of training algorithms, ranging from first-order gradient methods to Quasi-Newton and hybrid methods, have been implemented to adapt the synaptic weights of the developed networks; all of them have been successfully used to train the convolutional neural networks for a classification task. To demonstrate their capability in real life applications, the convolutional networks are employed in a face detection system and a handwritten digit recognition system. The face detector has 383 trainable network parameters and achieves a detection rate of 98%for detecting human faces on a large set of unconstrained and complex images. The handwritten digit recognition system, on the other hand, has 2722 trainable parameters, and its classification rate is 97.3% for recognizing human handwritten numerals. Besides these two applications, the developed network is analyzed for its built-in invariance, and it is implemented as a rotation invariant face classifier. The network achieves a classification rate of 97.3% in the rotation range ±900, and for 360° in-plane rotation, it has a correct detection rate of 93.6% at 5% false detection rate. These classification results demonstrate that the new class of convolutional neural networks has excellent generalization capability and achieves rotation invariance by adapting its connection weight matrices (receptive fields) as invariant feature detectors.