(click to copy)

Publication

Estimate of the Storage Capacity of q-Correlated Patterns in Hopfield Neural Networks

There has been considerable work lately regarding generalized thermostatistical formalisms arising from generalizations of the standard, Boltzmann-Gibbs entropy, as they are important in the study of complex systems. T

he probability distributions that optimize non-standard entropies are frequently found in Nature and are observed in diverse types of complex systems, including many relevant to neuroscience and artificial intelligence (AI). Probability distributions associated with entropic forms are used in models of artificial neural networks (NNs) and in the study of their basic properties.

Associative memory NN models are relevant both in AI and in the study of mental life, because memory is essential in many phenomena explored in areas of neuroscience, cognition and psychology, as well as in diverse AI techniques, such as recurrent NNs used for handwriting or speech recognition.

The estimation of the storage capacity of memory NNs is crucial, as there is a limitation to the quantity of information that a finite NN can store and retrieve correctly. The storage capacity of the Hopfield associative memory model has been estimated to be proportional to the number of neurons in the network, when the states of the neurons in the stored patterns and the patterns themselves are uncorrelated.

We present here a framework that can be used to estimate the storage capacity of Hopfield-like NNs, for the case where the stored patterns are correlated with a dependency among the states of neurons that leads to Tsallis distributions associated with generalized q-entropies.

R.S. Wedemann, A.R. Plastino, C. Tsallis, E.M.F. Curado, Estimate of the Storage Capacity of q-Correlated Patterns in Hopfield Neural Networks, Lecture Notes in Computer Science 15019() (2024) 137-150.

Constantino Tsallis, External Faculty at the Complexity Science Hub, celebrates his 80th birthday

Constantino Tsallis

0 Pages 0 Press 0 News 0 Events 0 Projects 0 Publications 0 Person 0 Visualisation 0 Art

Signup

CSH Newsletter

Choose your preference
   
Data Protection*