A Brief Introduction of Neural Networks
Free

A Brief Introduction of Neural Networks

By D. KRIESEL
Free
Book Description

How to teach a computer? You can either write a fixed program – or you can enable the computer to learn on its own. Living beings do not have any programmer writing a program for developing their skills, which then only has to be executed. They learn by themselves – without the previous knowledge from external impressions – and thus can solve problems better than any computer today. What qualities are needed to achieve such a behavior for devices like computers? Can such cognition be adapted from biology? History, development, decline and resurgence of a wide approach to solve problems.

Originally, this work has been prepared in the framework of a seminar of the University of Bonn in Germany, but it has been and will be extended. First and foremost, to provide a comprehensive overview of the subject of neural networks.

Zeta2 edition. Also available in German.

 

Table of Contents
  • A small preface
  • I From biology to formalization – motivation, philosophy, history and realization of neural models
    • 1 Introduction, motivation and history
      • 1.1 Why neural networks?
        • 1.1.1 The 100-step rule
        • 1.1.2 Simple application examples
      • 1.2 History of neural networks
        • 1.2.1 The beginning
        • 1.2.2 Golden age
        • 1.2.3 Long silence and slow reconstruction
        • 1.2.4 Renaissance
      • Exercises
    • 2 Biological neural networks
      • 2.1 The vertebrate nervous system
        • 2.1.1 Peripheral and central nervous system
        • 2.1.2 Cerebrum
        • 2.1.3 Cerebellum
        • 2.1.4 Diencephalon
        • 2.1.5 Brainstem
      • 2.2 The neuron
        • 2.2.1 Components
        • 2.2.2 Electrochemical processes in the neuron
      • 2.3 Receptor cells
        • 2.3.1 Various types
        • 2.3.2 Information processing within the nervous system
        • 2.3.3 Light sensing organs
      • 2.4 The amount of neurons in living organisms
      • 2.5 Technical neurons as caricature of biology
      • Exercises
    • 3 Components of artificial neural networks (fundamental)
      • 3.1 The concept of time in neural networks
      • 3.2 Components of neural networks
        • 3.2.1 Connections
        • 3.2.2 Propagation function and network input
        • 3.2.3 Activation
        • 3.2.4 Threshold value
        • 3.2.5 Activation function
        • 3.2.6 Common activation functions
        • 3.2.7 Output function
        • 3.2.8 Learning strategy
      • 3.3 Network topologies
        • 3.3.1 Feedforward
        • 3.3.2 Recurrent networks
        • 3.3.3 Completely linked networks
      • 3.4 The bias neuron
      • 3.5 Representing neurons
      • 3.6 Orders of activation
        • 3.6.1 Synchronous activation
        • 3.6.2 Asynchronous activation
      • 3.7 Input and output of data
      • Exercises
    • 4 Fundamentals on learning and training samples (fundamental)
      • 4.1 Paradigms of learning
        • 4.1.1 Unsupervised learning
        • 4.1.2 Reinforcement learning
        • 4.1.3 Supervised learning
        • 4.1.4 Offline or online learning?
        • 4.1.5 Questions in advance
      • 4.2 Training patterns and teaching input
      • 4.3 Using training samples
        • 4.3.1 Division of the training set
        • 4.3.2 Order of pattern representation
      • 4.4 Learning curve and error measurement
        • 4.4.1 When do we stop learning?
      • 4.5 Gradient optimization procedures
        • 4.5.1 Problems of gradient procedures
      • 4.6 Exemplary problems
        • 4.6.1 Boolean functions
        • 4.6.2 The parity function
        • 4.6.3 The 2-spiral problem
        • 4.6.4 The checkerboard problem
        • 4.6.5 The identity function
        • 4.6.6 Other exemplary problems
      • 4.7 Hebbian rule
        • 4.7.1 Original rule
        • 4.7.2 Generalized form
      • Exercises
  • II Supervised learning network paradigms
    • 5 The perceptron, backpropagation and its variants
      • 5.1 The singlelayer perceptron
        • 5.1.1 Perceptron learning algorithm and convergence theorem
        • 5.1.2 Delta rule
      • 5.2 Linear separability
      • 5.3 The multilayer perceptron
      • 5.4 Backpropagation of error
        • 5.4.1 Derivation
        • 5.4.2 Boiling backpropagation down to the delta rule
        • 5.4.3 Selecting a learning rate
      • 5.5 Resilient backpropagation
        • 5.5.1 Adaption of weights
        • 5.5.2 Dynamic learning rate adjustment
        • 5.5.3 Rprop in practice
      • 5.6 Further variations and extensions to backpropagation
        • 5.6.1 Momentum term
        • 5.6.2 Flat spot elimination
        • 5.6.3 Second order backpropagation
        • 5.6.4 Weight decay
        • 5.6.5 Pruning and Optimal Brain Damage
      • 5.7 Initial configuration of a multilayer perceptron
        • 5.7.1 Number of layers
        • 5.7.2 The number of neurons
        • 5.7.3 Selecting an activation function
        • 5.7.4 Initializing weights
      • 5.8 The 8-3-8 encoding problem and related problems
      • Exercises
    • 6 Radial basis functions
      • 6.1 Components and structure
      • 6.2 Information processing of an RBF network
        • 6.2.1 Information processing in RBF neurons
        • 6.2.2 Analytical thoughts prior to the training
      • 6.3 Training of RBF networks
        • 6.3.1 Centers and widths of RBF neurons
      • 6.4 Growing RBF networks
        • 6.4.1 Adding neurons
        • 6.4.2 Limiting the number of neurons
        • 6.4.3 Deleting neurons
      • 6.5 Comparing RBF networks and multilayer perceptrons
      • Exercises
    • 7 Recurrent perceptron-like networks (depends on chapter 5)
      • 7.1 Jordan networks
      • 7.2 Elman networks
      • 7.3 Training recurrent networks
        • 7.3.1 Unfolding in time
        • 7.3.2 Teacher forcing
        • 7.3.3 Recurrent backpropagation
        • 7.3.4 Training with evolution
    • 8 Hopfield networks
      • 8.1 Inspired by magnetism
      • 8.2 Structure and functionality
        • 8.2.1 Input and output of a Hopfield network
        • 8.2.2 Significance of weights
        • 8.2.3 Change in the state of neurons
      • 8.3 Generating the weight matrix
      • 8.4 Autoassociation and traditional application
      • 8.5 Heteroassociation and analogies to neural data storage
        • 8.5.1 Generating the heteroassociative matrix
        • 8.5.2 Stabilizing the heteroassociations
        • 8.5.3 Biological motivation of heterassociation
      • 8.6 Continuous Hopfield networks
      • Exercises
    • 9 Learning vector quantization
      • 9.1 About quantization
      • 9.2 Purpose of LVQ
      • 9.3 Using codebook vectors
      • 9.4 Adjusting codebook vectors
        • 9.4.1 The procedure of learning
      • 9.5 Connection to neural networks
      • Exercises
  • III Unsupervised learning network paradigms
    • 10 Self-organizing feature maps
      • 10.1 Structure
      • 10.2 Functionality and output interpretation
      • 10.3 Training
        • 10.3.1 The topology function
        • 10.3.2 Monotonically decreasing learning rate and neighborhood
      • 10.4 Examples
        • 10.4.1 Topological defects
      • 10.5 Adjustment of resolution and position-dependent learning rate
      • 10.6 Application
        • 10.6.1 Interaction with RBF networks
      • 10.7 Variations
        • 10.7.1 Neural gas
        • 10.7.2 Multi-SOMs
        • 10.7.3 Multi-neural gas
        • 10.7.4 Growing neural gas
      • Exercises
    • 11 Adaptive resonance theory
      • 11.1 Task and structure of an ART network
        • 11.1.1 Resonance
      • 11.2 Learning process
        • 11.2.1 Pattern input and top-down learning
        • 11.2.2 Resonance and bottom-up learning
        • 11.2.3 Adding an output neuron
      • 11.3 Extensions
  • IV Excursi, appendices and registers
    • A Excursus: Cluster analysis and regional and online learnable fields
      • A.1 k-means clustering
      • A.2 k-nearest neighboring
      • A.3 -nearest neighboring
      • A.4 The silhouette coefficient
      • A.5 Regional and online learnable fields
        • A.5.1 Structure of a ROLF
        • A.5.2 Training a ROLF
        • A.5.3 Evaluating a ROLF
        • A.5.4 Comparison with popular clustering methods
        • A.5.5 Initializing radii, learning rates and multiplier
        • A.5.6 Application examples
      • Exercises
    • B Excursus: neural networks used for prediction
      • B.1 About time series
      • B.2 One-step-ahead prediction
      • B.3 Two-step-ahead prediction
        • B.3.1 Recursive two-step-ahead prediction
        • B.3.2 Direct two-step-ahead prediction
      • B.4 Additional optimization approaches for prediction
        • B.4.1 Changing temporal parameters
        • B.4.2 Heterogeneous prediction
      • B.5 Remarks on the prediction of share prices
    • C Excursus: reinforcement learning
      • C.1 System structure
        • C.1.1 The gridworld
        • C.1.2 Agent und environment
        • C.1.3 States, situations and actions
        • C.1.4 Reward and return
        • C.1.5 The policy
      • C.2 Learning process
        • C.2.1 Rewarding strategies
        • C.2.2 The state-value function
        • C.2.3 Monte Carlo method
        • C.2.4 Temporal difference learning
        • C.2.5 The action-value function
        • C.2.6 Q learning
      • C.3 Example applications
        • C.3.1 TD gammon
        • C.3.2 The car in the pit
        • C.3.3 The pole balancer
      • C.4 Reinforcement learning in connection with neural networks
      • Exercises
    • Bibliography
    • List of Figures
    • Index
    No review for this book yet, be the first to review.
      No comment for this book yet, be the first to comment
      You May Also Like
      Also Available On
      App store smallGoogle play small
      Categories
      Curated Lists
      • Pattern Recognition and Machine Learning (Information Science and Statistics)
        by Christopher M. Bishop
        Data mining
        by I. H. Witten
        The Elements of Statistical Learning: Data Mining, Inference, and Prediction
        by Various
        See more...
      • CK-12 Chemistry
        by Various
        Concept Development Studies in Chemistry
        by John Hutchinson
        An Introduction to Chemistry - Atoms First
        by Mark Bishop
        See more...
      • Microsoft Word - How to Use Advanced Algebra II.doc
        by Jonathan Emmons
        Advanced Algebra II: Activities and Homework
        by Kenny Felder
        de2de
        by
        See more...
      • The Sun Who Lost His Way
        by
        Tania is a Detective
        by Kanika G
        Firenze_s-Light
        by
        See more...
      • Java 3D Programming
        by Daniel Selman
        The Java EE 6 Tutorial
        by Oracle Corporation
        JavaKid811
        by
        See more...