Introduction to Pytorch with Neural Networks

PyTorch is an open-source machine learning library for Python developed by Facebook AI Engineers. Facebook primarily built this for Natural Language Processing. PyTorch is developed with the concept of in-built probabilistic programming.

Prerequisites to Learn Pytorch

•     The developer must be familiar to Object detection, bounding box regression and non-maximum suppression
•     User must able to create a simple neural network with ease
•     User must know how neural network works
   

Types of Deep Neural Network

1.     Convolutional Neural Networks
2.     Recurrent Neural Networks

1. Convolutional Neural Networks

Multiple layers of data can be easily processed by CNN. Image recognition and face recognition are some applications of Convolutional Neural Networks.CNN can directly operate on images with the input of two-dimensional array rather than focusing on the extraction of what other neural networks focus on.

Problems of recognition can be solved using CNN. Facebook and Google have also invested in research and development for recognition projects to get activities with high speed and accuracy.
 

The convolutional neural network includes three basic ideas −

•     Local respective fields
•     Convolution
•     Pooling

Implementation with Pytorch

Create a Convolutional Neural Network using Pytorch

Step 1

Import all necessary package
 

from torch.autograd import Variable
import torch.nn.functional as F

Step 2

Convolutional Neural Network for batch representation. Batch Shape for input would be of dimension(3,32,32)
 

class SimpleCNN(torch.nn.Module):
   def __init__(self):
      super(SimpleCNN, self).__init__()
      #Input channels = 3, output channels = 18
      self.conv1 = torch.nn.Conv2d(3, 18, kernel_size = 3, stride = 2, padding = 2)
      self.pool = torch.nn.MaxPool2d(kernel_size = 3, stride = 3, padding = 0)
      
      self.fc1 = torch.nn.Linear(18 * 16 * 16, 64)
      
      self.fc2 = torch.nn.Linear(64, 10)

First Convolution size would change from (3,32,32) to (18,32,32)

Dimensions would changes from (18, 32, 32) to (18, 16, 16). The data dimension of the input layer will change from (18, 16, 16) to (1, 4608).
 

def forward(self, x):
   x = F.relu(self.conv1(x))
   x = self.pool(x)
   x = x.view(-1, 18 * 16 *16)
   x = F.relu(self.fc1(x))
      #Size changes from (1, 64) to (1, 10)
   x = self.fc2(x)
   return(x)

2. Recurrent Neural Network

Recurrent networks are types of deep learning-oriented algorithm which takes a sequential approach.  Input and Output will be considered as independent in neural networks. These types of neural networks are called recurrent because they perform mathematical computations in a sequential manner completing one task after another.

The training approach for our model will contain one data point per input time. 20 data points are taken as input for sequence x.
 

Step 1

Import all necessary package
 

import torch
from torch.autograd import Variable
import numpy as np
import pylab as pl
import torch.nn.init as init

Step 2

Set the size of the input layer to 7. In order to generate the target sequence, it takes consists of 6 context neurons and 1 input neuron
 

dtype = torch.FloatTensor
input_size, hidden_size, output_size = 7, 6, 1
epochs = 300
seq_length = 20
lr = 0.1
data_time_steps = np.linspace(2, 10, seq_length + 1)
data = np.sin(data_time_steps)
data.resize((seq_length + 1, 1))

x = Variable(torch.Tensor(data[:-1]).type(dtype), requires_grad=False)
y = Variable(torch.Tensor(data[1:]).type(dtype), requires_grad=False)

Step 3

Weights are initialized in the recurrent neural network using a normal distribution with zero means.
 

w1 = torch.FloatTensor(input_size,
hidden_size).type(dtype)
init.normal(w1, 0.0, 0.4)
w1 = Variable(w1, requires_grad = True)
w2 = torch.FloatTensor(hidden_size, output_size).type(dtype)
init.normal(w2, 0.0, 0.3)
w2 = Variable(w2, requires_grad = True)

Step 4

Now, we will create a function for feed-forward which uniquely defines the neural network.
 

w1 = torch.FloatTensor(input_size,
hidden_size).type(dtype)
init.normal(w1, 0.0, 0.4)
w1 = Variable(w1, requires_grad = True)
w2 = torch.FloatTensor(hidden_size, output_size).type(dtype)
init.normal(w2, 0.0, 0.3)
w2 = Variable(w2, requires_grad = True)

Step 5

We will start training procedure of recurrent neural network’s sine wave implementation. Outer loop with iterate over the inner loop and inner loop with iterate overall element. Prediction of the continuous variable with the help of Mean Square Error.
 

for i in range(epochs):
   total_loss = 0
   context_state = Variable(torch.zeros((1, hidden_size)).type(dtype), requires_grad = True)
   for j in range(x.size(0)):
      input = x[j:(j+1)]
      target = y[j:(j+1)]
      (pred, context_state) = forward(input, context_state, w1, w2)
      loss = (pred - target).pow(2).sum()/2
      total_loss += loss
      loss.backward()
      w1.data -= lr * w1.grad.data
      w2.data -= lr * w2.grad.data
      w1.grad.data.zero_()
      w2.grad.data.zero_()
      context_state = Variable(context_state.data)
   if i % 10 == 0:
      print("Epoch: {} loss {}".format(i, total_loss.data[0]))

context_state = Variable(torch.zeros((1, hidden_size)).type(dtype), requires_grad = False)
predictions = []

for i in range(x.size(0)):
   input = x[i:i+1]
   (pred, context_state) = forward(input, context_state, w1, w2)
   context_state = context_state
   predictions.append(pred.data.numpy().ravel()[0])

 
Now, You can create the Sinewave way you want.

Oodles Technologies, providing high-end tech solutions. Disrupt the digital future with Python Development Services. Our tech experts help you unlock the untapped and unimagined potential of Python, IoT & Mobile App Development solutions for your business. Drop us a line today at [email protected]