RkBlog

Overview of PYNQ project offering FPGA capabilities to Python and data engineers.

2020-05-11

FPGA is an chip that can be configured via a hardware programming language to make nearly any digital circuit. Hardware engineer can use them while Python developer can’t. But there is a distinct group of Xilinx FPGA based devices that can be used by software developers and data engineers, even with Python. This is possible thanks to PYNQ for Xilinx Zynq chips. In this article I'll showcase this platform and it capabilities.

Introduction

FPGA is configured by HDL - hardware description language like Verilog. When as on a microcontroller given pin is for example SPI on a FPGA it can be whatever it was programmed to be - wherever it's SPI, I2C or even USB, HDMI, Ethernet and alike. Aside of such I/O FPGA can do various type of processing and logic.

EEVblog #496 - What Is An FPGA?

FPGA isn't something we can use as easily as Raspberry Pi, Micro:bit and similar boards. But there is an option to use Xilinx Zynq based chips that contain an FPGA.

Xilinx created an architecture called Zynq. It combines an FPGA with ARM cores and I/O into one product. The ARM part is called Processing System (PS) while the FPGA is called Programmable Logic (PL). Such concept isn't limited only to this company but Xilinx went further and created PYNQ (pink).

PYNQ is a hardware-software stack that allows using an FPGA via Python and Jupyter notebooks running on the chip itself. It doesn't replace Verilog, it doesn't allow you to create designs for the FPGA but it allows interfacing and using designs made by hardware engineers.

PYNQ from the software developer side is aimed mostly at edge data processing and analysis (video, image), data processing function acceleration and machine learning. Depending on design it can compete with edge NPU chips or CPU/GPU platforms like Nvidia Jetson (in terms of power efficiency and or flexibility. One of example usage cases are drones and autonomous robots - processing images and video streams to detect obstacles, collision detection, controlling drone flight and more... but there are simpler usage cases too ;)

PYNQ

PYNQ is an open source project made by Xilinx for their Zynq based boards. As the project website states you can use it to create high performance applications with:

PYNQ big picture is quite simple. A Linux distribution is running on the ARM cores, it has Jupyter web server and configuration for the FPGA so that when the chip boots the FPGA is ready to work:

How PYNQ works
How PYNQ works

Looking at it in more detail reveals how each layer connects with others - from low level hardware to Python applications in Jupyter notebooks:

PYNQ software stack
PYNQ software stack

PYNQ overlay is a hardware design of the FPGA - it implements the logic on the FPGA - with Verilog and Vivado designer application. Hardware engineers can make various overlays and software engineers can use them in their applications. In-between we have Linux Kernel drivers and system libraries connect them together.

When you download and use the default PYNQ-Z2 board system image the overlay will configure the FPGA to work like so:

PYNQ-Z2 default overlay
PYNQ-Z2 default overlay

When using other overlays the pins on the board can function differently or have no logic assigned to them. This is the FPGA part where nothing is set in stone.

PYNQ Boards

There are four supported boards and few more supported by the community:

Right now PYNQ-Z2 is most popular and most accessible. If you want to try out PYNQ I would recommend this board. All of my tutorials will be based on this board.

PYNQ-Z2

The board has Zynq Z7020 chip with 512MB of DDR3 memory. I/O consists of HDMI in/out (PL), ADAU1761 audio codec, GigE, USB HOST (PS), Arduino and Raspberry Pi headers, 2 Pmod headers, 6 user LEDs, 4 pushbuttons and 2 Dip switches. And it's pink.

PYNQ-Z2 board
PYNQ-Z2 board
Ethernet and micro USB serial port
Ethernet and micro USB serial port

Board setup

In the box you will get the board only. You will need a microSD card (16GB or more), micro USB cable and an Ethernet connection. Aside of that a PC from which you will use the Jupyter notebooks.

A 16GB or bigger microSD card is needed. Download latest image for your board. Extract the image from the archive and burn it onto the microSD card. Etcher is the recommended application for this. When it's done insert it into the microSD card slot on the board.

PYNQ images for development boards
PYNQ images for development boards

Next connect the Ethernet cable - the basic setup assumes that the board and your PC/laptop will be connected to the same network (router, access point). You can also use a direct PC-board Ethernet connection but that requires bit of configuration - you will find more info in the official documentation.

When that's done connect the USB cable to the micro USB port next to it. Connect that cable to your PC and flip the power switch:

Power switch
Power switch
Red LED will turn after powering on the device
Red LED will turn after powering on the device
Green LED will turn on when the system boots
Green LED will turn on when the system boots

When you get the green light the board is ready to work. Depending on your system you may be already able to access it by opening http://pynq:9090/ in your browser. If it's not working no problem. We will get the IP address in a moment.

Serial connection

To install third party projects or to check the IP of the device a serial connection can be made in a terminal. Depending on the operating system the method will differ.

The USB connections provides power but also functions as a serial port. On Linux you can connect to the board via terminal using:

sudo screen /dev/ttyUSB1 115200

Where /dev/ttyUSB1 is the serial port created by the board. If you have more UARTs connected the number will be different. You can check dmesg logs after connecting the board to see what numbers it got:

USB serial device on Linux after connecting PYNQ-Z2
USB serial device on Linux after connecting PYNQ-Z2

When connection is made you should see the shell prompt (if nothing shows up try pressing Enter). Now just type ifconfig to see what IP the board got:

IP address of the PYNQ board
IP address of the PYNQ board

Now open http://IP:9090/ in your browser to access the notebooks. The password to Jupyter is xilinx

Jupyter notebook working on PYNQ-Z2
Jupyter notebook working on PYNQ-Z2

You can now start using the board. Jupyter is used to write and execute code while the serial connection can be used to install notebooks of other developers. sudo password on the board is xilinx. If you have any issues with the setup check the documentation. The whole stack looks like so:

PYNQ-Z2 stack overview
PYNQ-Z2 stack overview

On macOS you also can use the terminal application but the USB serial device has a different name:

ls /dev/tty.usb*
sudo screen /dev/tty.usbDEVICE_ID_HERE 115200

If you have problems with screen you can try one of serial console apps from app store.

Serial connection on late 2014 Mac Mini was quite unreliable but still managed to get the IP
Serial connection on late 2014 Mac Mini was quite unreliable but still managed to get the IP

On Windows the pynq:9090 URL should work, where as to make a serial connection you will have to install Putty or other app for serial connections. UART chip drivers may also be necessary.

To list serial devices on Windows you can use:

wmic path Win32_SerialPort

Usually it will be COM1 if no other serial device is connected.

pynq:9090 should work on Windows out of the box
pynq:9090 should work on Windows out of the box
Putty can be used to make the serial connection but you may also need UART chip driver for it to work
Putty can be used to make the serial connection but you may also need UART chip driver for it to work

Alternatively you can use the router software to check the IP of the PYNQ board - if it provides such information. Similarly IP/Port scanners could be used if needed.

SSH connection

Having the IP address of the device you can make a SSH connection via terminal or an app:

ssh xilinx@IP

The password is xilinx. SFTP in file managers that support it also works:

sftp://xilinx@IP

Working with Jupyter notebooks and PYNQ

The PYNQ-Z2 system comes with a set of example notebooks showcasing features and capabilities of the board. You can navigate through folders and open an interesting notebook.

To get started open base/board/board_btns_leds.ipynb - this notebook contains a simple code binding LED actions to press events of the buttons on the board.

Notebook interface
Notebook interface
Code cell while executing has an asterisk as a status
Code cell while executing has an asterisk as a status

Select the cell with the code and run it. The status will then change to an asterisk. Non-blocking code blocks will finish executing and you will see a number instead of the asterisk. This example is blocking and will end when you press the fourth button. Pressing first will change the LED colors while button 2 and 3 will light up LEDs back and forth.

Notebook has to be explicitly stopped to stop running. Closing the browser window won't stop it. Either use the File - close and halt option or halt the notebook from notebooks list after closing the tab with open notebook.

More advanced notebooks will have multiple code cells. You have to execute them in order one by one to get to the desired result. All of the included notebooks also have some markdown cells with description and documentation. When making a notebook you want to share be sure to add your own description and split the code into smaller cells as needed.

To install a third party notebook you need the SSH or serial connection and do it like so:

sudo pip3 install URL_HERE_TO_THE_REPOSITORY

This may take a while as dependencies being installed. When it's done reload the opened Jupyter listing and you should see the notebook being available. Some community notebook projects are on PYNQ website while many more are on Github.

In a subsequent article I'll try to go over interesting projects as well as overview the capabilities of the base overlay of PYNQ-Z2.

Board getting slightly warm during use
Board getting slightly warm during use

Additional resources

From Verilog and Vivado to Python notebook
Comment article