Hopfield net

Hopfield net

A Hopfield net is a form of recurrent artificial neural network invented by John Hopfield. Hopfield nets serve as content-addressable memory systems with binary threshold units. They are guaranteed to converge to a local minimum, but convergence to one of the stored patterns is not guaranteed.

tructure

The units in Hopfield nets are binary threshold units, i.e. the units only take on two different values for their states and the value is determined by whether or not the units' input exceeds their threshold. Hopfield nets can either have units that take on values of 1 or -1, or units that take on values of 1 or 0. So, the two possible definitions for unit "i"'s activation, a_i, are:

(1) a_i leftarrow left{egin{matrix} 1 & mbox {if }sum_{j}{w_{ij}s_j}> heta_i, \-1 & mbox {otherwise.}end{matrix} ight.

(2) a_i leftarrow left{egin{matrix} 1 & mbox {if }sum_{j}{w_{ij}s_j}> heta_i, \0 & mbox {otherwise.}end{matrix} ight.

Where:
* w_{ij} is the strength of the connection weight from unit j to unit i (the weight of the connection).
* s_j is the state of unit j.
* heta_i is the threshold of unit i.

The connections in a Hopfield net typically have the following restrictions:
* w_{ii}=0, forall i (no unit has a connection with itself)
* w_{ij} = w_{ji}, forall i,j (connections are symmetric)

The requirement that weights be symmetric is typically used, as it will guarantee that the energy function decreases monotonically while following the activation rules, and the network may exhibit some periodic or chaotic behaviour if non-symmetric weights are used. However, Hopfield found that this chaotic behaviour is confined to relatively small parts of the phase space, and does not impair the network's abilities to act as a content-addressable associative memory system.

Hopfield nets have a scalar value associated with each state of the network referred to as the "energy", E, of the network, where::E = -frac12sum_{i+sum_i{ heta_i s_i}

This value is called the "energy" because the definition ensures that if units are randomly chosen to update their activations the network will converge to states which are local minima in the energy function (which is considered to be a Lyapunov function). Thus, if a state is a local minimum in the energy function it is a stable state for the network. Note that this energy function belongs to a general class of models in physics, under the name of Ising models; these in turn are a special case of Markov networks, since the associated probability measure, the Gibbs measure, has the Markov property.

Running

At each step, pick a node at random. The node's behavior is then deterministic: it moves to a state to minimize the energy of itself and its neighbors. (In contrast, the Boltzmann machine has a stochastic update rule.)

Training

Training a Hopfield net involves lowering the energy of states that the net should "remember". This allows the net to serve as a content addressable memory system, that is to say, the network will converge to a "remembered" state if it is given only part of the state. The net can be used to recover from a distorted input the trained state that is most similar to that input. This is called associative memory because it recovers memories on the basis of similarity. For example, if we train a Hopfield net with five units so that the state (1, 0, 1, 0, 1) is an energy minimum, and we give the network the state (1, 0, 0, 0, 1) it will converge to (1, 0, 1, 0, 1). Thus, the network is properly trained when the energy of states which the network should remember are local minima.

References

J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities", "Proceedings of the National Academy of Sciences of the USA", vol. 79 no. 8 pp. 2554-2558, April 1982.

ee also

* Associative memory
* Autoassociative memory
* Boltzmann machine — like a Hopfield net but uses annealed Gibbs sampling instead of gradient descent
* Ising model

External links

* Chapter 13 [http://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf The Hopfield model] of [http://page.mi.fu-berlin.de/rojas/neural/index.html.html "Neural Networks - A Systematic Introduction"] by Raul Rojas (ISBN 978-3540605058)
* [http://www.shef.ac.uk/psychology/gurney/notes/l5/l5.html A good introduction to associative memory and Hopfield networks]
* [http://www.comp.nus.edu.sg/~pris/AssociativeMemory/HopfieldModel.html The Hopfield networks page] at [http://www.comp.nus.edu.sg/~pris PRIS, Singapore]
* [http://www.heatonresearch.com/articles/61/page1.html Hopfield Neural Network Applet]
* [http://to-campos.planetaclix.pt/neural/hope.html The Travelling Salesman Problem] - Hopfield Neural Network JAVA Applet
* [https://gna.org/projects/neurallab/ Neural Lab Graphical Interface] - Hopfield Neural Network graphical interface (Python & gtk)


Wikimedia Foundation. 2010.

Игры ⚽ Поможем написать реферат

Look at other dictionaries:

  • Hopfield — may refer to: * Hopfield net, a type of neural network * John Joseph Hopfield (b. 1933), American biologist and physicist, inventor of the Hopfield net * John J. Hopfield (spectroscopist) , American spectroscopist, discoverer of the Hopfield… …   Wikipedia

  • Hopfield (RNA) — Una red de Hopfield es una forma de red neuronal artificial recurrente inventada por John Hopfield. Las redes de Hopfield se usan como sistemas de Memoria asociativa con unidades binarias. Están diseñadas para converger a un mínimo local, pero la …   Wikipedia Español

  • Boltzmann machine — A Boltzmann machine is the name given to a type of stochastic recurrent neural network by Geoffrey Hinton and Terry Sejnowski. Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets. They were one of the first… …   Wikipedia

  • Artificial intelligence — AI redirects here. For other uses, see Ai. For other uses, see Artificial intelligence (disambiguation). TOPIO, a humanoid robot, played table tennis at Tokyo International Robot Exhibition (IREX) 2009.[1] Artificial intelligence ( …   Wikipedia

  • History of artificial intelligence — The history of artificial intelligence begins in antiquity with myths, stories and rumors of artificial beings endowed with intelligence and consciousness by master craftsmen. In the middle of the 20th century, a handful of scientists began to… …   Wikipedia

  • Neural network — For other uses, see Neural network (disambiguation). Simplified view of a feedforward artificial neural network The term neural network was traditionally used to refer to a network or circuit of biological neurons.[1] The modern usage of the term …   Wikipedia

  • Ising model — The Ising model, named after the physicist Ernst Ising, is a mathematical model in statistical mechanics. It has since been used to model diverse phenomena in which bits of information, interacting in pairs, produce collectiveeffects.Definition… …   Wikipedia

  • Biophysics — (also biological physics) is an interdisciplinary science that employs and develops theories and methods of the physical sciences for the investigation of biological systems. Studies included under the umbrella of biophysics span all levels of… …   Wikipedia

  • List of algorithms — The following is a list of the algorithms described in Wikipedia. See also the list of data structures, list of algorithm general topics and list of terms relating to algorithms and data structures.If you intend to describe a new algorithm,… …   Wikipedia

  • Computational neuroscience — is the study of brain function in terms of the information processing properties of the structures that make up the nervous system.[1] It is an interdisciplinary science that links the diverse fields of neuroscience, cognitive science and… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”