Artificial Neural Networks

Neural Network - Feed Forward network with input, hidden and output layer

In many cases technology adopts solutions that were generated by evolutionary processes in biology. The same can be applied on Artificial Neural Networks (ANN). This learning resource starts with the comparison of Biological Neuronal Networks (BNN) (e.g. in our brain) and Artificial Neural Networks.

Learning Tasks

• Analyze biological neural networks and explain how the network between cells (neurons) is realized in a BNN.
• Analyze the processes within a neuron. What is threshold for a neuron and how do neurons communicate with eachother?
• In average we have $10^{n}$ neurons and every single neuron is connected with $10^{k}$ other neurons? Search the web to estimate $10^{n}$ and $10^{k}$.
• Explain how the biological processes in a BNN are realized in an ANN. What are the similarities and what are the differences according to different types of Artificial Neural Networks.
• Describe in your own words: "What is learning and apply your definition on different examples of your choice in which you would see learning as a fundamental principle?"
• Explain the role of learning algorithm for an ANN. How does the learning algorithm change the network.
• Machine Learning is a broader concept than Artificial Neural Networks. Identify examples that are related to machine learning but do refer to the concept of an ANN.
• Analyse some basic ANN topologies like Backpropagation Networks, Kohonen Networks, Hopfield nets, ... and describe what kind of biological processes are modelled.
• Decribe the differences of supervised and unsupervised learning.
• Look into Open Source software tools like Octave and R/Studio. What are open source packages that can be used for machine learning (start e.g. with Kohonen networks).
• (Lateral Inhibition) Also for image processing biological neural networks may serve as prototype. Explain the concept of lateral inhibition and how can it be used for edge detection in artificial neural networks.
• (Pattern Recognition) Explain how ANNs can be used for pattern recognition in data and for processing big data. What are the limitation of brute force methods and how can ANN support you in reducing the runtime for pattern recognition.
• What are the limitations of ANNs as black boxes for pattern recognition?
• Analyze the concept of NetTalk implemented by Terrence Sejnowski and Charles Rosenberg. What are the fundamental principles that are incorporate in NetTalk for machine learning and how did those technologies evolve in the 20 years?
• Compare basic principles implemented in OpenSource Speech Recognition and compare the approach with Tensor Flow in current Open Source implementation e.g. Vosk. What are benefits and drawbacks of different approaches?

See also

• Machine Learning
• Pattern Recognition
• Supervised and unsupervised learning,
• Biological Neural Networks,
• Octave and R/R-Studio packages for machine learning,
• Kohonen Network - Clustering of Input Data,
• Learning Algorithm,
• Learning - in general