Github Page

FPGA Research Background

This research is being conducted at Arizona State University and I (Mark Ashinhust) am the primary person working on it. The main purpose of this research is to empower autonomous navigation with radar sensing and machine learning.

The short term goal of this project is to run simple Convolutional Neural Networks using Brevitas and FINN. This is then imported to the Pynq-Z1 FPGA board.

Quantized Convolutional Neural Network

A Quantized Convolutional Neural Network (QCNN) is a deep learning model that uses quantization to reduce the size and compuatational complexity of standard CNN’s. In quantization, the weights and activations of a CNN are represented with fewer bits than the usual 32- or 64-bit floating-point numbers, which allows for faster processing on devices with limited computing power and memory, such as mobile devices and embedded systems. The quantization process can also result in a smaller model size, making it easier to deploy in memory-constrained environments.

Therefore, by using these QCNN’s we are better able to port information to the FPGA device which has low amounts of memory.

Overview Of Contents

There are three main things that need to be working and all come together for this project to work. The first is Brevitas followed by FINN and lastly the Pynq board. Since the pynq board has been slightly covered I will move to describing Brevitas and FINN.

Brevitas

Brevitas is the main PyTorch library that is used for network quantization. This Brevitas network has a focus on quantization-aware-training (QAT).

FINN

FINN is an experimental framework from the Xilinx Research Labs to explore deep neural network inference on FPGA devices. FINN specifically targets quantized neural networks with emphasis on generating dataflow-style architectures customized for each network. The resulting FPGA accelerators are highly efficient and can yield high throughput and low latency. The framework is fully open-source in order to give a higher degree of flexibility, and is intended to enable neural network research spanning several layers of the software/hardware abstraction stack.

Pynq

This is the final part of our three main categories to this project.

PYNQ is an open-source project from Xilinx that makes it easy to design embedded systems with Zynq All Programmable Systems on Chips (APSoCs). Using the Python language and libraries, designers can exploit the benefits of programmable logic and microprocessors in Zynq to build more capable and exciting embedded systems. PYNQ users can now create high performance embedded applications with

• parallel hardware execution
• high frame-rate video processing
• hardware accelerate algorithms
• real-time signal processing
• high bandwidth IO
• low latency control

Follow for more information Pynq Docs

Projects

All projects are located in the current Github Repository

The first project deals with Cifar-10 Explorations. The first exploration of Cifar-10 was done using PyTorch instead of Brevitas to get a better understanding. Then this was converted to using Brevitas which provided better information.

PyTorch Cifar-10 Implementation

The first Cifar-10 implementation with PyTorch found here has an accuracy of about 54% after training.

Brevitas Cifar-10 Implementation

There are many ways that the Cifar-10 dataset was implemented with Brevitas. The whole directory is found here.

The initial Brevitas implementation uses a lower channel size and less epochs. Therefore, the final accuracy was only about 53%.

However, when using a quantized version with updated bit widths and channel sizes the testing accuracy got to approximately 66%. This is great and means that out of 10,000 images 6,666 will be correctly identified.

The code for the CNN definition is located below with the full notebook here

####################################################################################################################
# Information Used for Brevitas
#
#
####################################################################################################################

import brevitas.nn as qnn
from brevitas.quant import Int32Bias

# setup class for the neural network building
class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()

        self.quant_inp = qnn.QuantIdentity(bit_width = 4, return_quant_tensor = True)

        # Three input channels for the 32x32 images in color
        # Using Brevitas QuantConv2d
        self.conv1 = qnn.QuantConv2d(3, 16, 5, bias = True, weight_bit_width = 4, bias_quant = Int32Bias)   # in_channels = 3, out_channels = 6, kernel_size = 5

        self.relu1 = qnn.QuantReLU(bit_width = 4, return_quant_tensor = True)

        # input channels is the output of the last Conv2d
        # using Brevitas,
        self.conv2 = qnn.QuantConv2d(16, 32, 5, bias = True, weight_bit_width = 4, bias_quant = Int32Bias)  # in_channels = 6, out_channels = 16, kernel_size = 5

        self.relu2 = qnn.QuantReLU(bit_width = 4, return_quant_tensor = True)

        # Fully Connected Layers using Brevitas
        self.fc1 = qnn.QuantLinear(32 * 5 * 5, 120, bias = True, weight_bit_width = 4, bias_quant = Int32Bias)

        self.relu3 = qnn.QuantReLU(bit_width = 4, return_quant_tensor = True)

        self.fc2 = qnn.QuantLinear(120, 10, bias = True, weight_bit_width = 4, bias_quant = Int32Bias)

    # feed forward
    def forward(self, x):

        # forward pass
        x = self.quant_inp(x)
        x = self.relu1(self.conv1(x))
        x = F.max_pool2d(x, 2)
        x = self.relu2(self.conv2(x))
        x = F.max_pool2d(x, 2)
        x = x.reshape(x.shape[0], -1)
        x = self.relu3(self.fc1(x))
        x = self.fc2(x)

        return x      # output

net = Net()

Full implementation of the research is continuing but will be updated as time goes on. All updates take place on my personal repository. Feel free to click the shield here or at the top to look at the repo.

Github Page