AOHW 2024

Team information

• Team number: AOHW-200
• Project name: RISC-V based GPU
• University name: Universitat Politècnica de Catalunya (UPC)
• Participant(s):
  • Javier Beiro Piñón
  • Beatriz Navidad Vilches
  • Nicolás Zhilie Zhao
• Supervisor: Dr. Leonidas Kosmidis

Vortex GPGPU

Vortex is a full-stack open-source RISC-V GPGPU. For our submission to the AOHW2024, we have added support for tensor instructions for loading, storing and performing multiplication and addition of matrices. We have also added warp-level intrinsics that make use of these instructions.

Specifications

• Supports RISC-V RV32IMAF and RV64IMAFD
• Microarchitecture:
  • configurable number of cores, warps, and threads.
  • configurable number of ALU, FPU, LSU, and SFU units per core.
  • configurable pipeline issue width.
  • optional shared memory, L1, L2, and L3 caches.
  • default configuration:
    • 1 core, 4 warps and 4 threads/warp
    • 4 ALU, 4 FPU, 4 LSU and 4 SFU units per core
    • issue width of 4
    • L1 enabled, L2 and L3 disabled
• Software:
  • OpenCL 1.2 Support.
• Supported FPGAs:
  • Altera Arria 10
  • Altera Stratix 10
  • Xilinx Alveo U50, U250, U280
  • Xilinx Versal VCK5000

Directory structure

• ci: Continuous integration scripts.
• docs: Documentation.
• hw: Hardware sources.
• kernel: RISC-V device runtime.
• miscs: Miscellaneous resources.
• runtime: Host drivers implementations.
• sim: Simulators repository.
• tests: Tests repository.
• runTests.sh: script for executing all tests performed in the report.
• runTests_cache.sh: script for measuring the memory access patterns in the implemented instruction.

Build Requirements

Supported OS Platforms

• Ubuntu 18.04, 20.04
• CentOS 7

Toolchain Dependencies

• POCL LLVM
• Our Customized LLVM
• RISCV-GNU-TOOLCHAIN
• Verilator
• FpNew
• SoftFloat
• Ramulator
• Yosys
• Sv2v

Build Instructions

We already ship a docker image based on Ubuntu 20.04 that has the Vortex repository cloned and all the dependencies installed.

The build process is described below.

Within docker container

It's only necessary to build Vortex's sources:

cd vortex
make -s -j $(nproc)

From scratch

1. Get Vortex codebase

   git clone --recursive https://github.com/Beanavil/vortex.git vortex
   cd vortex

2. Install dependencies

   sudo apt-get install build-essential zlib1g-dev libtinfo-dev libncurses5 uuid-dev libboost-serialization-dev libpng-dev libhwloc-dev ninja-build cmake

   and upgrade gcc to 11:

   sudo apt-get install gcc-11 g++-11

   Multiple gcc versions on Ubuntu can be managed with update-alternatives, e.g.:

   sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 9
   sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 9
   sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11
   sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 11

3. Set up prebuilt toolchain

   By default, the toolchain will be installed to the /opt folder, which requires sudo access. You can install the toolchain to a different location of your choice by setting TOOLDIR (e.g. export TOOLDIR=$HOME/tools).

   export TOOLDIR=/opt

   ./ci/toolchain_install.sh --all

   source ./ci/toolchain_env.sh

4. Set up custom LLVM

   git clone https://github.com/Beanavil/vortex-llvm llvm-vortex && cd llvm-vortex

   cmake -G Ninja -S llvm -B build -DLLVM_INSTALL_UTILS=ON -DCMAKE_INSTALL_PREFIX=$TOOLDIR/llvm-vortex -DCMAKE_BUILD_TYPE=Release -DLLVM_DEFAULT_TARGET_TRIPLE="riscv32-unknown-elf" -DLLVM_TARGETS_TO_BUILD="RISCV" -DLLVM_ENABLE_PROJECTS="clang"

   ninja -C build install

5. Build Vortex's sources

   make -s -j $(nproc)

Execute the tensor core test

For executing the tensor core test for $2\times 2$ matrices with hardware simulation:

./ci/blackbox.sh --driver=rtlsim --app=tmul --args="-n2"