The SQRL Acorn is an M.2 form factor FPGA accelerator card originally designed for cryptocurrency mining acceleration. But due to poor performance and little support, it failed, causing a big scandal. Fortunately for us that know how to use FPGAs, we can re-purpose the hardware to do other kinds of acceleration, such as DSP processing.
Under the enormous heat-sink, it contains a fairly high-end Xilinx Artix-7 FPGA and some ram but due to limited IO, it definitely needs to be used in an accelerator-like fashion utilizing the PCIe connector to perform data transfer to the device.
You can get one from the original designer on Amazon for ~$100 from the link below (Click the picture below to purchase from Amazon), but you can also get them from eBay as there were tons of cryptocurrency people trying to offload them.
The designer of the board has some information on their GitHub but it’s a bit sparse on details. It does contain purchasing links to their dedicated web store or Amazon.
First Contact
First, we need to reverse engineer the interface to make sure we can program the card and interact with it. This post on the Element 14 forum was super helpful basically doing all the hard work and convinced me to purchase one for experimentation now that I have my server set up.
Element14: Do Mighty Oaks from SQRL Acorns Grow
The article solves the basics of what JTAG connector you will need along with the pinout. To get access to JTAG we need a specific Molex cable, which you can get from DigiKey: Molex Cable
we will also need a JTAG pod so you can talk to the JTAG interface and upload bitstreams to the FPGA device. You can get one on Amazon or while ordering the Molex connector add it to your Digikey cart. DigiKey: Digilent HS3 JTAG Programmer
It also might be a good idea to get a 2mm pitch connector as .1in pin headers and jumper cables do not fit in the connector. I had some that kinda fit but could damage the JTAG programmer pod if forced so I am not sure if this exact model is correct. DigiKey: 2mm Pitch Header
Or you can get the hs2 JTAG module that has .1in headers instead
Digikey: Digilent HS2 JTAG Programmer
Here is how you connect up the pins on the board to the hs3 JTAG module if that is the one that you already have.
Another helpful link for setting up the JTAG interface is the guide from Enjoy-Digital who has been very prominent in creating features around this repurposed FPGA.
Connection Diagram from Enjoy-Digital
Once wired up and installed back in the server for power, we can open up Vivado to see if our board is alive.
Now that it is showing up in the hardware manager we can check the die temperatures to make sure our board is not overheating, since I had decided to remove the annoying fan. I might need to add it back once the FPGA is under heavier loads.
Gateware
Now that we have the ability to reprogram the FPGA we need to create some designs to take advantage of the hardware and the special interfaces on the SQRL Acorn board. The FPGA is a Xilinx Artix-7 XC7A200T which has 4 Gen 2 PCIe lanes a couple general purpose IO and not much else… So any design that we create for this board really needs to take advantage of those PCIe lanes first and foremost. There has been some work to repurpose the FPGA SERDES interfaces that perform the PCIe transfers for other things like SATA or optical networking but it requires a special board and some more hardware hacking. So I decided to focus on using the FPGA device inside of a desktop computer.
This is because the SQRL Acorn was designed as an M.2 form factor device, similar to the SSD drives that are common on most modern motherboards and laptops, as their primary memory storage device. This meant it is pretty easy to install in an off-the-shelf desktop computer either directly on the motherboard or in a PCIe expansion slot using an adapter board.
Vivado
Since this device uses a Xilinx FPGA we need the Vivado software in order to synthesize bitstreams for the device. Thankfully the free version of Vivado can be used to create designs for this part since a full license can cost thousands of dollars! This is a huge piece of software in the tens of Gigabytes so be prepared with some significant disk space. The other thing is it can be quite difficult to install on Linux. My version of Ubuntu 20.04 has some funny issues that I had to overcome but I was able to get it running. On Linux, you may also need to create a script that sources the directory that the installer places the software but that can be easily done. Here is one guide that could be helpful but you might need to troubleshoot a bit and may no longer be relevant.
Vivado 2020.x Ubuntu 20.04 Install guide
LiteX
While we could start from scratch using VHDL or Verilog and define our own constraints file and create our own PCIe core there is an amazing project, called LiteX, that has done most of this work for us!
The LiteX framework provides a convenient and efficient infrastructure to create FPGA Cores/SoCs, to explore various digital design architectures, and create full FPGA-based systems.
https://github.com/enjoy-digital/litex
One of the most difficult things in FPGAs is just getting the first interactions, this is because we are basically starting from a blank slate. So the LiteX software has some pre-defined interfaces and processor cores that you can instantiate. The main one that piqued my interest was the lite-pcie interface.
This is because the PCIe interface not only needs a core on the FPGA but a driver and software on your PC to perform data transfers back and forth. Xilinx has some documentation for this but it is notoriously difficult to implement and work with. There are also DRAM and Ethernet controllers, SD Card and SPI interfaces, and UART serial. The software can also instantiate a plethora of RISCV processor cores that you can even use to run Linux.
The other really cool thing about LiteX is that it is super easy to install, all you need to do is run this command that takes care of all the core code but you may need to install some other packages.
wget https://raw.githubusercontent.com/enjoy-digital/litex/master/litex_setup.py
chmod +x litex_setup.py
./litex_setup.py --init --install --user (--user to install to user directory)Linux on Litex VexRISCV
This project bridges the gap between the processor core that LiteX generates and running Linux on it. Most of the requirements are handled by the litex_setup.py process. You can clone the repo here if it was not already installed in the LiteX setup.
GItHub: Linux on LiteX VexRISCV
We can use this extension of LiteX to build all of the gateway, firmware, software, and drivers! This command below creates the bootloader that can be used to recover from a failed bitstream if it was loaded into the flash via the PCIe utility.
./make.py --board=acorn_pcie --build --flashIt flashes the fallback bitstream at 0 with OpenOCD and the operational bitstream can then be flashed at 0x400000 with litepcie_util flash_write (or with OpenOCD and a 0x400000 offset). This creates a build directory where you can build and install the PCIe driver as well.
LiteX Boards
This repo contains the plethora of compatible boards that people have created LiteX designs for and have made open source to the community. From small USB-sized boards all the way to massive accelerator boards costing thousands. Thankfully one of these is the SQRL Acorn so we should be able to quickly get started using the board.
You can also build a bitstream without the Linux portion using the instructions in the sqrl_acorn.py file in litex-boards, you can cd into the “targets” directory and run the following instructions. This python file can also be helpful to build the litepcie_util for future flashing if you started with the bootloader that we flashed in the previous section.
# Build/Use ----------------------------------------------------------------------------------------
# Build/Load bitstream:
# ./sqrl_acorn.py --uart-name=crossover --with-pcie --build --driver --load (or --flash)
#
#.Build the kernel and load it:
# cd build/<platform>/driver/kernel
# make
# sudo ./init.sh
#
# Test userspace utilities:
# cd build/<platform>/driver/user
# make
# ./litepcie_util info
# ./litepcie_util scratch_test
# ./litepcie_util dma_test
# ./litepcie_util uart_testLitex Terminal
If you want to access the UART terminal over the PCIe interface you need to also install the kernel module for that in the same build/<platform>/driver/kernel folder. From there you can open the port that the kernel creates.
sudo insmod liteuart.ko
litex_term /dev/ttyLXU0LiteX Overview
While LiteX is some amazing software it can feel a bit disjointed starting out due to all the different repo locations and starting places. This is because for FPGAs you can start almost anywhere and so the projects and use cases can start to explode from networking and hardcore DSP to game emulation, to video processing and music creation, etc. But if you just start from the basic LiteX repo and start exploring first you should be able to find the information you need to get started on your application.
Server Configuration
I have a unique server that I planned on using this FPGA in which is based on the Unraid OS. This operating system is used as a networked file server as well as a VM server that hosts my main Ubuntu system that I use for development and main computer. The issue was that in order to get a PCIe device to connect with the VM I needed to reserve it so that the Unraid OS does not connect with it and can then be connected to the VM once it has started later. This process is mainly done with Graphics cards and USB controllers but should work with any PCIe device in theory.
One of the obstacles I needed to handle was flashing the board’s internal memory so that on boot up the FPGA would get the correct bitstream to implement the lite-pcie interface. This is because otherwise, the old bitstream from SQRL would create a PCIe interface that would not have the same properties as the lite-pcie and could not be reserved in the Unraid OS. Thankfully I was able to overcome this and flashed the memory using the Vivado hardware manager, but new flashing capabilities should be available in the previous sections.
The Unraid reservation was super easy, all you needed to do was find the Memory controller from Xilinx and copy the identifier “10ee:7024” into the boot configuration as a VFIO device for reservation similar to how it is done for a USB controller.
The only other problem I ran into when adding the device to the VM was that it did not come up as an “other PCIe device” in the VM GUI so I needed to manually enter the details in the XML format. Not super difficult but you need to keep this in mind if you try to update the VM from the GUI this will need to be re-entered manually.
Results
I was able to successfully program the FPGA and flash the memory with the Vivado hardware manager. I now need to update the flash with the bootloader version of the gateway and try to program over the PCIe using the utility.
Now I just need to find a project that would allow me to take advantage of this architecture to solve some problems. I will most likely look into some kind of digital signal processing acceleration using GNURadio.
Extra Resources
This is a helpful resource to the original Twitter post from Enjoy Digital That has some information from the designer.
There is also a baseboard in development that turns the Acorn M.2 FPGA board into a normal FPGA development board with much greater IO such as an SFP port.
Someone also created a GNURadio accelerator skeleton for accelerating digital signal processing for software-defined radios.
There is also another project that where the creator was able to put the M.2 FPGA board into a Thunderbolt enclosure so that this device could be used with a laptop as well.
