August 12, 2022

Fast learning

Cover image The paper aims at speeding up Deep Neural Networks (DNN) since this is one of the major bottlenecks in deep learning. This has been achieved by parameterizing the weight matrix using low rank factorization and periodic functions. By parameterization, the weight matrix is split into two matrices of smaller size of rank K with periodic functions. A shrinkage parameter has been introduced which helps in reducing the number of parameters and thus helps in increasing the speed to a great extent. Performance of the proposed parameterization is compared with standard DNN, DNN based on weight factorization alone and on periodic-bounded weights. This has been demonstrated on benchmark datasets MNIST and MNIST variants.

Keywords

Deep learning;
Deep Neural Network;
Denoising autoencoder

B. Chandra is with the Computer Science Group, Department of Mathematics, Indian Institute of Technology, Delhi, India, where she is currently a Professor and was the Department Chair from August 2004 to August 2007. Her specializations include feature selection, pattern classification using neural networks, decision trees, deep learning algorithms and association rule mining. She has been a visiting researcher at NIST Gaithersburg, Maryland during summer 2012. She has been a Visiting Professor with the Graduate School of Business, University of Pittsburgh, Pittsburgh, and Penn State University, University Park. She has also been a Visiting Scientist with the Institut National de Recherche en Informatiqueet en Automatique, France. She has been the Chairman in various sessions on neural networks and machine learning at international conferences held at Hawaii, Washington, DC in the U.S., Bangor, U.K., Montreal, Canada, Singapore, Jeon-Buk, South Korea, and at Bangkok, Thailand. She has been invited to deliver invited lectures at various universities in the U.S., viz. University of Pittsburgh, Penn State University, University of Eastern Illinois, University of Hawaii, University of Texas at Dallas, University of Orleans, and Virginia Tech, and at other institutions like the National University of Singapore, Bangor University, U.K., and Ecole de Mines Paris. She has authored a number of research papers published in reputed international journals published by IEEE, Elsevier, Springer in the area of neural networks, classification, and clustering. She has also authored three books. She has been the Principal Investigator of many sponsored research and consultancy research projects in the field of neural networks and machine learning. She is also actively involved in teaching and curriculum development for the Graduate Program in Computer Applications at the Indian Institute of Technology.