Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.14365/876
Title: | Exploiting heterogeneity in operational neural networks by synaptic plasticity | Authors: | Kiranyaz, Serkan Malik, Junaid Abdallah, Habib Ben İnce, Türker Iosifidis, Alexandros Gabbouj, Moncef |
Keywords: | Operational neural networks Convolutional neural networks Synaptic Plasticity Representations |
Publisher: | Springer London Ltd | Abstract: | The recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an elite ONN can then be configured using the top-ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result, the performance gap over the CNNs further widens. | URI: | https://doi.org/10.1007/s00521-020-05543-w https://hdl.handle.net/20.500.14365/876 |
ISSN: | 0941-0643 1433-3058 |
Appears in Collections: | Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection |
Show full item record
CORE Recommender
SCOPUSTM
Citations
13
checked on Aug 14, 2024
WEB OF SCIENCETM
Citations
12
checked on Aug 14, 2024
Page view(s)
88
checked on Aug 19, 2024
Download(s)
16
checked on Aug 19, 2024
Google ScholarTM
Check
Altmetric
Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.