Papers by Mahyar Shahsavari
Auto-tuning HyperParameters of SGD Matrix Factorization-Based Recommender Systems Using Genetic Algorithm
IEEE transactions on multi-scale computing systems, Oct 1, 2018
The brain-inspired spiking neural network neuromorphic architecture offers a promising solution f... more The brain-inspired spiking neural network neuromorphic architecture offers a promising solution for a wide set of cognitive computation tasks at a very low power consumption. Due to the practical feasibility of hardware implementation, we present a memristor-based model of hardware spiking neural networks which we simulate with N2S3 (Neural Network Scalable Spiking Simulator), our open source neuromorphic architecture simulator. Although Spiking neural networks are widely used in the community of computational neuroscience and neuromorphic computation, there is still a need for research on the methods to choose the optimum parameters for better recognition efficiency. With the help of our simulator, we analyze and evaluate the impact of different parameters such as number of neurons, STDP window, neuron threshold, distribution of input spikes and memristor model parameters on the MNIST handwritten digit recognition problem. We show that a careful choice of a few parameters (number of neurons, kind of synapse, STDP window and neuron threshold) can significantly improve the recognition rate on this benchmark (around 15 points of improvement for the number of neurons, a few points for the others) with a variability of 4 to 5 points of recognition rate due to the random initialization of the synaptic weights.
Said Hamdioui and the president of the jury, Prof. Virginie Hoel for their insightful comments an... more Said Hamdioui and the president of the jury, Prof. Virginie Hoel for their insightful comments and encouragement, and reviewing my thesis. My sincere thanks also goes to CRIStAL lab and Émeraude team colleagues, prof. Giuseppe Lipari, Dr Richard Olejnik, Dr Clément Ballabriga and specially to Dr Julien Forget. I will never forget Forget supports specially for the first time teaching at Polytech Lille. I appreciate my Émeraude team friends Antoine Bertout, Khalil Ibrahim Hamzaoui, Houssam Zahaf, Pierre Falez and Yassine sidlakhdar for nice discussions, coffee drinking and playing football together. I would like to thanks to our colleague Dr Fabien Alibart in IEMN lab for his technical consultancy during my research particularly during developing new synapse. I gratefully acknowledge my previous friends and colleagues in TUDelft in The Neterlands, my lovely and kind friends Faisal Nadeem, Arash Ostadzadeh, Mahmood Ahmadi that shared with me valuable knowledge and information. Thanks again to Prof. Said Hamdioui that we started working on memristor together in CE group at TUDelft. Special thanks go to my Iranian friends in Lille Farzan, Hamid, Hamidreza, Ehsan, Babak, and Sina thanks for bing there for me. Eric and Leopoled, my lovely officemates and friends, I never forget your kindnesses in our calm office in M3 building. I am grateful of my Iranian friends and colleagues at Razi University, Prof. Mohsen Hayati Dr Mahmood Ahmadi, Dr Arash Ahmadi and one of my best friend Mazdak Fatahi that we started working on neural network research for his Master thesis. We have done many skype meetings that both of us learned a lot during those scientific discussions. Actually the last chapter of my thesis is related to his master thesis topic. I appreciate my father in law Prof. Mohammad Mehdi Khodaei for his guidance during my research as well as supporting us for starting collaboration between Razi and Lille 1 universities. I really Dr Mahmood Ahmadi, and Mazdak Fatahi for their supports and being kindly present specially during the time our French professors and colleagues i On estime que le nombre d'objets connectés à l'Internet atteindra 50 à 100 milliards en 2020. La recherche s'organise en deux champs principaux pour répondre à ce défi : l'internet des objets et les grandes masses de données. La demande en puissance de calcul augmente plus vite que le développement de nouvelles architectures matérielles en particulier à cause du ralentissement de la loi de Moore. La raison principale en est est le mur de la mémoire, autrement appelé le goulet d'étranglement de Von Neumann, qui vient des différences de vitesse croissantes entre le processeur et la mémoire. En conséquence, il y a besoin d'une nouvelle architecture matérielle rapide et économe en énergie pour répondre aux besoins énormes de puissance de calcul. Dans cette thèse, nous proposons de nouvelles architectures pour les processeurs de prochaine génération utilisant des nanotechnologies émergentes telles que les memristors. Nous étudions des méthodes de calcul non conventionnelles aussi bien numériques qu'analogiques. Notre contribution principale concerne les réseaux de neurones à impulsion (RNI) ou architectures neuromorphiques. Dans la première partie de la thèse, nous passons en revue les memristors existants, étudions leur utilisation dans une architecture numérique à base de crossbars, puis introduisons les architectures neuromorphiques. La deuxième partie contient la contribution principale : le développement d'une simulateur d'architectures neuromorphiques (N2S3), l'introduction d'un nouveau type de synapse pour améliorer l'apprentissage, une exploration des paramètres en vue d'améliorer les RNI, et enfin une étude de la faisabilité des réseaux profonds dans les RNI.
HAL (Le Centre pour la Communication Scientifique Directe), Jan 11, 2017
One of the most promising approaches to overcome the end of Moore's law is neuromorphic computing... more One of the most promising approaches to overcome the end of Moore's law is neuromorphic computing. Indeed, neural networks already have a great impact on machine learning applications and offer very nice properties to cope with the problems of nanoelectronics manufacturing, such as a good tolerance to device variability and circuit defects, and a low activity, leading to low energy consumption. We present here N2S3 (for Neural Network Scalable Spiking Simulator), an open-source simulator that is built to help design spiking neuromorphic circuits based on nanoelectronics. N2S3 is an event-based simulator and its main properties are flexibility, extensibility, and scalability. One of our goals with the release of N2S3 as open-source software is to promote the reproducibility of research on neuromorphic hardware. We designed N2S3 to be used as a library, to be easily extended with new models and to provide a user-friendly special purpose language to describe the simulations.
HAL (Le Centre pour la Communication Scientifique Directe), Mar 13, 2015
HAL (Le Centre pour la Communication Scientifique Directe), May 26, 2016
Benchmarks and datasets have important role in evaluation of machine learning algorithms and neur... more Benchmarks and datasets have important role in evaluation of machine learning algorithms and neural network implementations. Traditional dataset for images such as MNIST is applied to evaluate efficiency of different training algorithms in neural networks. This demand is different in Spiking Neural Networks (SNN) as they require spiking inputs. It is widely believed, in the biological cortex the timing of spikes is irregular. Poisson distributions provide adequate descriptions of the irregularity in generating appropriate spikes. Here, we introduce a spike-based version of MNSIT (handwritten digits dataset), using Poisson distribution and show the Poissonian property of the generated streams. We introduce a new version of evt_MNIST which can be used for neural network evaluation.
Pattern recognition is used to classify the input data into different classes based on extracted ... more Pattern recognition is used to classify the input data into different classes based on extracted key features. Increasing the recognition rate of pattern recognition applications is a challenging task. The spike neural networks inspired from physiological brain architecture, is a neuromorphic hardware implementation of network of neurons. A sample of neuromorphic architecture has two layers of neurons, input and output. The number of input neurons is fixed based on the input data patterns. While the number of outputs neurons can be different. The goal of this paper is performance evaluation of neuromorphic architecture in terms of recognition rates using different numbers of output neurons. For this purpose a simulation environment of N2S3 and MNIST handwritten digits are used. Our simulation results show the recognition rate for various number of output neurons,
Neuromorphic computing systems simulate spiking neural networks that are used for research into h... more Neuromorphic computing systems simulate spiking neural networks that are used for research into how biological neural networks function, as well as for applied engineering such as robotics, pattern recognition, and machine learning. In this paper, we present a neuromorphic system based on an asynchronous event-based hardware platform. We represent three algorithms for implementing spiking networks on our asynchronous hardware platform. We also discuss different trade-offs between synchronisation and messaging costs. A reinforcement learning method known as Rewardmodulated STDP is presented as an online learning algorithm in the network. We evaluate the system performance in a single box of our designed architecture using 6000 concurrent hardware threads and demonstrate scaling to networks with up to 2 million neurons and 400 million synapses. The performance of our architecture is also compared to existing neuromorphic platforms, showing a 20 times speed-up over the Brian simulator on an x86 machine, and a 16 times speed-up over a 48-chip SpiNNaker node.
HAL (Le Centre pour la Communication Scientifique Directe), Jul 18, 2016
With the end of Moore's law in sight, we need new computing architectures to satisfy the increasi... more With the end of Moore's law in sight, we need new computing architectures to satisfy the increasing demands of big data processing. Neuromorphic architectures are good candidates to low energy computing for recognition and classification tasks. We propose an event-based spiking neural network architecture based on artificial synapses. We introduce a novel synapse box that is able to forget and remember by inspiration from biological synapses. Two different volatile and nonvolatile memristor devices are combined in the synapse box. To evaluate the effectiveness of our proposal, we use system-level simulation in our Neural Network Scalable Spiking Simulator (N2S3) using the MNIST handwritten digit recognition dataset. The first results show better performance of our novel synapse than the traditional nonvolatile artificial synapses.
Springer eBooks, 2019
Neuromorphic computation using Spiking Neural Networks (SNN) is proposed as an alternative soluti... more Neuromorphic computation using Spiking Neural Networks (SNN) is proposed as an alternative solution for future of computation to conquer the memory bottelneck issue in recent computer architecture. Different spike codings have been AQ1 discussed to improve data transferring and data processing in neuro-inspired computation paradigms. Choosing the appropriate neural network topology could result in better performance of computation, recognition and classification. The model of the neuron is another important factor to design and implement SNN systems. The speed of simulation and implementation, ability of integration to the other elements of the network, and suitability for scalable networks are the factors to select a neuron model. The learning algorithms are significant consideration to train the neural network for weight modification. Improving learning in neuromorphic architecture is feasible by improving the quality of artificial synapse as well as learning algorithm such as STDP. In this chapter we proposed a new synapse box that can remember and forget. Furthermore, as the most frequent used unsupervised method for network training in SNN is STDP, we analyze and review the various methods of STDP. The sequential order of pre-or postsynaptic spikes occurring across a synapse in an interval of time leads to defining different STDP methods. Based on the importance of stability as well as Hebbian competition or anti-Hebbian competition the method will be used in weight modification. We survey the most significant projects that cause making neuromorphic platform. The advantages and disadvantages of each neuromorphic platform are introduced in this chapter. AQ2 1 Introduction The mammalian nervous system is a network of extreme complexity which is able to perform cognitive computation in a parallel and power-efficient manner. Understanding the principles of the brain processing for computational modeling is one of the
Neuromorphic computing is gaining momentum as an alternative hardware platform for large-scale ne... more Neuromorphic computing is gaining momentum as an alternative hardware platform for large-scale neural simulation. However, with several major devices and systems available and planned, often with very different characteristics, it is not always clear which platform is suitable for which application. Simulating the platform on conventional computers is typically too slow to be of use, but an alternative approach is to implement an 'emulation' of the hardware in FPGAs which can execute at near-hardware speeds but does not commit to a specific hardware architecture. We present an overlay model-a method which superimposes bespoke features on top of a standard template-in both hardware and software to implement neuromorphic architectures using the POETS (Partially Ordered Event Triggered Systems) system. This combination of overlays permits very large-scale simulations to be performed in real time for hardware exploration or application verification, while retaining the flexibility to redefine either the hardware or software layer, if results indicate potential to improve performance, or significant design problems. Using this system we simulate up to 500,000 neurons on a single-box system, that can be scaled to ∼4,000,000 neurons in an 8-box configuration. Results indicate the crucial constraint for real-time simulation: peak input spike rate per neuron; and help to optimise both hardware and software around neural application requirements. The preliminary architecture demonstrates the feasibility of an overlay model, while indicating directions for future neuromorphic systems. With POETS, we introduce a platform that can help to shape and investigate the neuromorphic architectures of the future.
Advancements in Spiking Neural Network Communication and Synchronization Techniques for Event-Driven Neuromorphic Systems
Auto-tuning HyperParameters of SGD Matrix Factorization-Based Recommender Systems Using Genetic Algorithm
2022 IEEE International Conference on Omni-layer Intelligent Systems (COINS)
arXiv: Emerging Technologies, Dec 20, 2016
A memristor is a two-terminal nanodevice that its properties attract a wide community of research... more A memristor is a two-terminal nanodevice that its properties attract a wide community of researchers from various domains such as physics, chemistry, electronics, computer and neuroscience. The simple structure for manufacturing, small scalability, nonvolatility and potential of using in low power platforms are outstanding characteristics of this emerging nanodevice. In this report, we review a brief literature of memristor from mathematic model to the physical realization. We discuss different classes of memristors based on the material used for its manufacturing. The potential applications of memristor are presented and a wide domain of applications are explained and classified.
Task Scheduling Policies in General Distributed Systems: A Survey and Possibilities
ABSTRACT Task scheduling algorithms in distributed and parallel sys-tems play a vital role to pro... more ABSTRACT Task scheduling algorithms in distributed and parallel sys-tems play a vital role to provide better performance platforms for mul-tiprocessor networks. A large number of policies, which can determine best structures of task scheduling algorithms, have been explored so far. These policies have significant value for optimizing system efficiency. The objective of all these approaches are maximizing system throughput with assigning a task to a suitable processor, maximizing resource utilization, and minimizing execution time. In this essay, there are various types of different algorithms for parallel and distributed systems that have been classified by reviewing former surveys. Then, various task scheduling al-gorithms are discussed from different points of view such as dependency among tasks, static vs. dynamic approaches, and heterogeneity of proces-sors. Precedence orders like list heuristics have been studied. Duplication based algorithms, clustering heuristics and scheduling methods inspired by nature's laws like GA (Genetic Algorithm) are other kind of algorithm approaches of this study.
physica status solidi (c), 2014
Memristor is a two-terminal nanodevice that has recently attracted the attention of many research... more Memristor is a two-terminal nanodevice that has recently attracted the attention of many researchers. Its simple structure, non-volatility behavior, high-density integration, and low-power consumption make the memristor a promising candidate to act as a switch in digital gates for future high-performance and low-power nanocomputing applications. In this paper, we model the behavior of memristor by using Verilog-A. To investigate its characteristics in a circuit, we use the HSPICE simulator. Furthermore, a library of digital gates are provided by using two approaches to make digital gates: the first one is based on material implication (IMP) and the second one is based on crossbar arrays. Finally, we perform a comparison and evaluation between the two methods.
1st International Conference on New Research Achievements in Electrical and Computer Engineering, May 26, 2016
Benchmarks and datasets have important role in evaluation of machine learning algorithms and neur... more Benchmarks and datasets have important role in evaluation of machine learning algorithms and neural network implementations. Traditional dataset for images such as MNIST is applied to evaluate efficiency of different training algorithms in neural networks. This demand is different in Spiking Neural Networks (SNN) as they require spiking inputs. It is widely believed, in the biological cortex the timing of spikes is irregular. Poisson distributions provide adequate descriptions of the irregularity in generating appropriate spikes. Here, we introduce a spike-based version of MNSIT (handwritten digits dataset), using Poisson distribution and show the Poissonian property of the generated streams. We introduce a new version of evt_MNIST which can be used for neural network evaluation.
Memristor Technology and Applications:An Overview
A Memristor is a 2-terminal thin-film electrical device that is a short form of memory-resistor. ... more A Memristor is a 2-terminal thin-film electrical device that is a short form of memory-resistor. The Memristor as a basic circuit element has drawn wide attention in the international research community of electrical and computer engineers, physicists, and biologists. The main feature of the Memristor is the ability to remember its state history. Therefore, it is becoming prominent for the realization of nonvolatile VLSI resistive random access memories (RRAM). For the applicability as a synapse in neuromorphic systems the Memristor provides even further surprising properties in electric circuits, which opens up new horizons to future applications. In this report, we study the properties and potentials of Memristor. We explain how it works, how is its physical design behavour, and what are the different implementation and fabrication materials of this nano-device. Furthermore, we investigate potential applications regarding to previous contributed papers. We also classify a novel ca... 
Uploads
Papers by Mahyar Shahsavari