How to teach a computer? You can either write a fixed program – or you can enable the computer to learn on its own. Living beings do not have any programmer writing a program for developing their skills, which then only has to be executed. They learn by themselves – without the previous knowledge from external impressions – and thus can solve problems better than any computer today. What qualities are needed to achieve such a behavior for devices like computers? Can such cognition be adapted from biology? History, development, decline and resurgence of a wide approach to solve problems.

Originally, this work has been prepared in the framework of a seminar of the University of Bonn in Germany, but it has been and will be extended. First and foremost, to provide a comprehensive overview of the subject of neural networks.

Zeta2 edition. Also available in German.

- A small preface
- I From biology to formalization – motivation, philosophy, history and realization of neural models
- 1 Introduction, motivation and history
- 1.1 Why neural networks?
- 1.1.1 The 100-step rule
- 1.1.2 Simple application examples

- 1.2 History of neural networks
- 1.2.1 The beginning
- 1.2.2 Golden age
- 1.2.3 Long silence and slow reconstruction
- 1.2.4 Renaissance

- Exercises

- 1.1 Why neural networks?
- 2 Biological neural networks
- 2.1 The vertebrate nervous system
- 2.1.1 Peripheral and central nervous system
- 2.1.2 Cerebrum
- 2.1.3 Cerebellum
- 2.1.4 Diencephalon
- 2.1.5 Brainstem

- 2.2 The neuron
- 2.2.1 Components
- 2.2.2 Electrochemical processes in the neuron

- 2.3 Receptor cells
- 2.3.1 Various types
- 2.3.2 Information processing within the nervous system
- 2.3.3 Light sensing organs

- 2.4 The amount of neurons in living organisms
- 2.5 Technical neurons as caricature of biology
- Exercises

- 2.1 The vertebrate nervous system
- 3 Components of artificial neural networks (fundamental)
- 3.1 The concept of time in neural networks
- 3.2 Components of neural networks
- 3.2.1 Connections
- 3.2.2 Propagation function and network input
- 3.2.3 Activation
- 3.2.4 Threshold value
- 3.2.5 Activation function
- 3.2.6 Common activation functions
- 3.2.7 Output function
- 3.2.8 Learning strategy

- 3.3 Network topologies
- 3.3.1 Feedforward
- 3.3.2 Recurrent networks
- 3.3.3 Completely linked networks

- 3.4 The bias neuron
- 3.5 Representing neurons
- 3.6 Orders of activation
- 3.6.1 Synchronous activation
- 3.6.2 Asynchronous activation

- 3.7 Input and output of data
- Exercises

- 4 Fundamentals on learning and training samples (fundamental)
- 4.1 Paradigms of learning
- 4.1.1 Unsupervised learning
- 4.1.2 Reinforcement learning
- 4.1.3 Supervised learning
- 4.1.4 Offline or online learning?
- 4.1.5 Questions in advance

- 4.2 Training patterns and teaching input
- 4.3 Using training samples
- 4.3.1 Division of the training set
- 4.3.2 Order of pattern representation

- 4.4 Learning curve and error measurement
- 4.4.1 When do we stop learning?

- 4.5 Gradient optimization procedures
- 4.5.1 Problems of gradient procedures

- 4.6 Exemplary problems
- 4.6.1 Boolean functions
- 4.6.2 The parity function
- 4.6.3 The 2-spiral problem
- 4.6.4 The checkerboard problem
- 4.6.5 The identity function
- 4.6.6 Other exemplary problems

- 4.7 Hebbian rule
- 4.7.1 Original rule
- 4.7.2 Generalized form

- Exercises

- 4.1 Paradigms of learning

- 1 Introduction, motivation and history
- II Supervised learning network paradigms
- 5 The perceptron, backpropagation and its variants
- 5.1 The singlelayer perceptron
- 5.1.1 Perceptron learning algorithm and convergence theorem
- 5.1.2 Delta rule

- 5.2 Linear separability
- 5.3 The multilayer perceptron
- 5.4 Backpropagation of error
- 5.4.1 Derivation
- 5.4.2 Boiling backpropagation down to the delta rule
- 5.4.3 Selecting a learning rate

- 5.5 Resilient backpropagation
- 5.5.1 Adaption of weights
- 5.5.2 Dynamic learning rate adjustment
- 5.5.3 Rprop in practice

- 5.6 Further variations and extensions to backpropagation
- 5.6.1 Momentum term
- 5.6.2 Flat spot elimination
- 5.6.3 Second order backpropagation
- 5.6.4 Weight decay
- 5.6.5 Pruning and Optimal Brain Damage

- 5.7 Initial configuration of a multilayer perceptron
- 5.7.1 Number of layers
- 5.7.2 The number of neurons
- 5.7.3 Selecting an activation function
- 5.7.4 Initializing weights

- 5.8 The 8-3-8 encoding problem and related problems
- Exercises

- 5.1 The singlelayer perceptron
- 6 Radial basis functions
- 6.1 Components and structure
- 6.2 Information processing of an RBF network
- 6.2.1 Information processing in RBF neurons
- 6.2.2 Analytical thoughts prior to the training

- 6.3 Training of RBF networks
- 6.3.1 Centers and widths of RBF neurons

- 6.4 Growing RBF networks
- 6.4.1 Adding neurons
- 6.4.2 Limiting the number of neurons
- 6.4.3 Deleting neurons

- 6.5 Comparing RBF networks and multilayer perceptrons
- Exercises

- 7 Recurrent perceptron-like networks (depends on chapter 5)
- 7.1 Jordan networks
- 7.2 Elman networks
- 7.3 Training recurrent networks
- 7.3.1 Unfolding in time
- 7.3.2 Teacher forcing
- 7.3.3 Recurrent backpropagation
- 7.3.4 Training with evolution

- 8 Hopfield networks
- 8.1 Inspired by magnetism
- 8.2 Structure and functionality
- 8.2.1 Input and output of a Hopfield network
- 8.2.2 Significance of weights
- 8.2.3 Change in the state of neurons

- 8.3 Generating the weight matrix
- 8.4 Autoassociation and traditional application
- 8.5 Heteroassociation and analogies to neural data storage
- 8.5.1 Generating the heteroassociative matrix
- 8.5.2 Stabilizing the heteroassociations
- 8.5.3 Biological motivation of heterassociation

- 8.6 Continuous Hopfield networks
- Exercises

- 9 Learning vector quantization
- 9.1 About quantization
- 9.2 Purpose of LVQ
- 9.3 Using codebook vectors
- 9.4 Adjusting codebook vectors
- 9.4.1 The procedure of learning

- 9.5 Connection to neural networks
- Exercises

- 5 The perceptron, backpropagation and its variants
- III Unsupervised learning network paradigms
- 10 Self-organizing feature maps
- 10.1 Structure
- 10.2 Functionality and output interpretation
- 10.3 Training
- 10.3.1 The topology function
- 10.3.2 Monotonically decreasing learning rate and neighborhood

- 10.4 Examples
- 10.4.1 Topological defects

- 10.5 Adjustment of resolution and position-dependent learning rate
- 10.6 Application
- 10.6.1 Interaction with RBF networks

- 10.7 Variations
- 10.7.1 Neural gas
- 10.7.2 Multi-SOMs
- 10.7.3 Multi-neural gas
- 10.7.4 Growing neural gas

- Exercises

- 11 Adaptive resonance theory
- 11.1 Task and structure of an ART network
- 11.1.1 Resonance

- 11.2 Learning process
- 11.2.1 Pattern input and top-down learning
- 11.2.2 Resonance and bottom-up learning
- 11.2.3 Adding an output neuron

- 11.3 Extensions

- 11.1 Task and structure of an ART network

- 10 Self-organizing feature maps
- IV Excursi, appendices and registers
- A Excursus: Cluster analysis and regional and online learnable fields
- A.1 k-means clustering
- A.2 k-nearest neighboring
- A.3 -nearest neighboring
- A.4 The silhouette coefficient
- A.5 Regional and online learnable fields
- A.5.1 Structure of a ROLF
- A.5.2 Training a ROLF
- A.5.3 Evaluating a ROLF
- A.5.4 Comparison with popular clustering methods
- A.5.5 Initializing radii, learning rates and multiplier
- A.5.6 Application examples

- Exercises

- B Excursus: neural networks used for prediction
- B.1 About time series
- B.2 One-step-ahead prediction
- B.3 Two-step-ahead prediction
- B.3.1 Recursive two-step-ahead prediction
- B.3.2 Direct two-step-ahead prediction

- B.4 Additional optimization approaches for prediction
- B.4.1 Changing temporal parameters
- B.4.2 Heterogeneous prediction

- B.5 Remarks on the prediction of share prices

- C Excursus: reinforcement learning
- C.1 System structure
- C.1.1 The gridworld
- C.1.2 Agent und environment
- C.1.3 States, situations and actions
- C.1.4 Reward and return
- C.1.5 The policy

- C.2 Learning process
- C.2.1 Rewarding strategies
- C.2.2 The state-value function
- C.2.3 Monte Carlo method
- C.2.4 Temporal difference learning
- C.2.5 The action-value function
- C.2.6 Q learning

- C.3 Example applications
- C.3.1 TD gammon
- C.3.2 The car in the pit
- C.3.3 The pole balancer

- C.4 Reinforcement learning in connection with neural networks
- Exercises

- C.1 System structure
- Bibliography
- List of Figures
- Index

- A Excursus: Cluster analysis and regional and online learnable fields

#### Free Machine Learning Books

11 Books

- Pattern Recognition and Machine Learning (Information Science and Statistics)
- by Christopher M. Bishop
- Data mining
- by I. H. Witten
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction
- by Various

#### Free Chemistry Textbooks

9 Books

- CK-12 Chemistry
- by Various
- Concept Development Studies in Chemistry
- by John Hutchinson
- An Introduction to Chemistry - Atoms First
- by Mark Bishop

#### Free Mathematics Textbooks

21 Books

- Microsoft Word - How to Use Advanced Algebra II.doc
- by Jonathan Emmons
- Advanced Algebra II: Activities and Homework
- by Kenny Felder
- de2de
- by

#### Free Children Books

38 Books

- The Sun Who Lost His Way
- by
- Tania is a Detective
- by Kanika G
- Firenze_s-Light
- by

#### Free Java Books

10 Books

- Java 3D Programming
- by Daniel Selman
- The Java EE 6 Tutorial
- by Oracle Corporation
- JavaKid811
- by