Does the term “neuron” ring a bell in your mind? It is the building block of the brain and it inspired computer scientists from the 1950s— how to make a computer perform tasks like a brain does? It is not a simple problem and the clue to its complexity is in the brain structure. We need billions of artificial neurons if we were to build an artificial brain. With the increase in computing power, mimicking billions of neurons is now possible. The concept behind an Artificial Neural Network is to define inputs and outputs, feed pieces of inputs to computer programs that function like neurons and make inferences or calculations, then forward those results to another layer of computer programs and so on, until a result is obtained. As part of this neural network, a feedback or difference between intended output and the input is computed at each layer and this difference is used to tune the parameters to each program. This method is called backpropagation and it is an essential component to the Neural Network.
- The concept behind an Artificial Neural Network is to define inputs and outputs, feed pieces of inputs to computer programs that function like neurons and make inferences or calculations, then forward those results to another layer of computer programs and so on, until a result is obtained. As part of this neural network, a difference between intended output and input is computed at each layer and this difference is used to tune the parameters to each program. This method is called backpropagation and is an essential component to the Neural Network.
- It was also observed that instead of CPUs, Graphic Processing Units (GPU) which are good at performing massive parallel tasks can be used for setting up ANNs.
- A few free ANN frameworks are TensorFlow, Keras, PyTorch and Theano. These can be used for both normal Machine Learning tasks like classification or clustering and for Deep Learning/ANN tasks.
The popularity of ANNs
A few more technical phrases will clear up our understanding of this space. Data Science, used interchangeably with Machine Learning, is the computer technology that uses data to detect patterns. Hand-written digit recognition is a good example of machine learning. However, in order for the computer to do this task, large amounts of sample data need to be manually labelled as examples of images of digits. Manual sampling at this scale is not going to be enough. Can there be a technology that can avoid human involvement to label data but can automatically detect patterns in sample data and tune its parameters to an algorithm so that the algorithm is ready to perform automatic tasks? The Artificial Neural Networks (ANN) mentioned above with its backpropagation does exactly this. This is why ANNs have become hugely popular in the past decade. This approach of using neural networks of many layers to automatically detect patterns and parameters is called Deep Learning.
A couple of key developments in the past two decades helped ANNs mature. Cloud computing provided enormous computing resources that are needed for ANNs to “work through” massive volumes of data. Along with this, it was observed that instead of the CPU in a computer, Graphic Processing Unit (GPU) which is good at performing massive parallel tasks can be used for setting up ANNs. In the last two decades, the software for neural networks matured and backpropagation techniques became robust. Combining these concepts, if thousands of GPUs are available that can take up chunks of data and can execute programs on those chunks, then ANNs can be made available for a variety of tasks. Many commercial and free software have become available which use GPUs and Cloud and offer readily available ANNs. A few popular free neural network frameworks are TensorFlow, Keras, PyTorch and Theano.
Free neural networks
TensorFlow was developed by Google. It uses a specific hardware that is optimised to work with GPU and divides the neural network operations and the corresponding data into units called Tensors. It has an architecture that sets up neural network and uses it on our input data to make it “flow” from one state to another and gives options to choose and operate states that are relevant to us. Keras is a software that can be used on top of TensorFlow so that software developers can interact with just the relevant parameters instead of lower-level details that need to be provided to TensorFlow. Popular implementations of TensorFlow are Google’s search algorithm RankBrain and Twitter’s tweet ranking.
The ANN frameworks or software mentioned above can be used for both normal Machine Learning tasks like classification or clustering and for Deep Learning/ANN tasks. Are there tasks that cannot be done with good accuracy by normal Machine Learning and hence need Deep Learning? The answer is yes. Automatic Image Recognition of rich images (instead of only simple hand-written digits) and Speech Recognition are two popular uses of Deep Learning. Convolution Neural Network (CNN), a special type of ANN, is good at Image Recognition. It connects a neuron in a layer to all neurons in the next layer but uses optimisation techniques to weed out unwanted signals from neurons. For Speech Recognition, Recurrent Neural Network (RNN) is used because it is good at handling inputs of variable length like speech.
Deep Learning has progressed to the next level and instead of only working on input data and detections, it can now actually generate creative output like music or paintings. A special type of ANN called Generative Adversarial Network achieves this.
Deep Learning made news in 2016 when an ANN-based product called AlphaGo defeated a player in a game of “Go”.
ANNs are present in many smartphone applications that we use, like voice to type, Siri and Alexa.
If you want to play with neural networks, you can hop on to one of the free frameworks and build a software program. The Cambrian explosion of artificial intelligence is here!
S. Varahasimhan is a senior employee at a software product MNC in Chennai
