Skip to content

Compiler Setup

Jump to bottom
Gautam Sharma edited this page Jun 6, 2022 · 8 revisions

Overview of Compiler

The Popcorn compiler is built on top of clang+LLVM and builds binaries suitable for migration between heterogeneous-ISA systems. It requires a modified version of clang & LLVM and several supporting tools/libraries for building these binaries. These will be installed and used by a Makefile included with the compiler to generate multi-ISA binaries. For more information, see the READMEs in the repository (and sub-folders), which describe how the compiler, libraries and tools operate. We assume in this document the Popcorn cluster is comprised of two nodes; node 0 is running x86 at 10.4.4.100, and node 1 is running ARM at 10.4.4.101.

Install the Compiler

  • Firstly, we'll install several prerequisites:
$ sudo apt-get install build-essential flex bison subversion cmake zip texinfo

  • Should have gcc installed for both x86-64 and ARM64. If you have followed from installing VM, the compiler suites will be already installed on the host. Otherwise, refer to Internet documents for installing the crossbuild suite.

  • Should install g++ compiler on Ubuntu

$ sudo apt-get install x86_64-linux-gnu-g++
  • Download the popcorn compiler and NPB benchmarks, and place them in the same directory on the host.
$ git clone -b main --single-branch https://github.com/ssrg-vt/popcorn-compiler.git
Item Filename URL
Example - NPB NPB3.3-SER-C-FLAT.zip https://drive.google.com/open?id=1oyFep7igjRaJvl3nx7BRREphcF8Y1dvD
  • Go to the compiler repository and look at the contents
$ cd popcorn-compiler
$ ls
common  INSTALL  install_compiler.py  ISSUES  lib  patches  README  tool  util
  • Install the Popcorn compiler toolchain. /usr/local/popcorn is the default path where the toolchain is installed. Change <MYID> with your login ID. If you're not using popcorn kernel 5.2, please update the value of kernel_version in install_compiler.py accordingly.
$ sudo mkdir -p /usr/local/popcorn
$ sudo chown <MYID> /usr/local/popcorn
$ ./install_compiler.py --install-all --threads 8
  • Or install into a directory of your choosing (denoted as <POPCORN PATH>)
$ ./install_compiler.py --install-path <POPCORN PATH> --install-all --threads 8
  • The compiler, including supporting libraries and tools, is now installed at <POPCORN PATH>. You can use the the provided Makefile in "popcorn-compiler/util/Makefile.pyalign.template" to build applications. The makefile template expects that the entire application's source is contained in a single directory.

Example: NPB

We provide a port of NPB benchmark suite modified to run core computation on a remote node. The modified benchmark is comprised of 9 programs (bt, cg, ep, ft, is, lu , mg, sp, and ua) and three workloads (S, A, and B).

  • Untar the source code and have a look at the contents:
$ tar -xzf npb-popcorn.tar.gz
$ cd npb-popcorn
$ ls
bt  dc  ft  lu        mg                      setclass.sh  ua
cg  ep  is  Makefile  migration_points.patch  sp
  • Set the workload to run using make make {S | A | B} 
$ make A    # Set the benchmark suite to run using workload A

If you have problems in the linking stage, you probably need to update the Makefile to use the new x86_64-popcorn-linux-gnu-ld.gold instead of ld.gold.

  • Or .setclass.sh {S | A | B} 
$ ./setclass.sh S  # Set to use workload S
  • Build the benchmark suite
$ make
  • You will find <PROGRAM>_aarch64 and <PROGRAM>_x86-64 on each <PROGRAM> directory. To run the programs, place them in the Popcorn nodes. Taking "ep" as an example:
$ cd ep
$ scp ep_aarch64 ep_x86-64 [email protected]:
$ scp ep_aarch64 ep_x86-64 [email protected]:
  • Set executables for each ISA.
popcorn@x86~$ cp ep_x86-64 ep
popcorn@arm~$ cp ep_aarch64 ep

  • DO NOT use soft-link nor hard-link for setting up the executable. Popcorn is so fragile for now that doing that may disallow the thread migration.

  • Now, you can run the program on node 0:

popcorn@x86:~$ ./ep
  • By default, kernels print out many debug messages to ensure the program execution and to help debugging. You can turn off them by unsetting CONFIG_POPCORIN_DEBUG_PROCESS_SERVER, CONFIG_POPCORN_DEBUG_PAGE_SERVER, and CONFIG_POPCORN_DEBUG_VMA_SERVER from the kernel configuration (e.g., .config) or Popcorn Distributed Execution Support/Log debug messages for Popcorn from the menuconfig.

Initiate thread migration

Programmers can instruct threads to be relocated to a specified node by putting migration points. In the NPB example above, such migration points can be found from migration_points.patch file.

The migration can be triggered by calling the following function:

#include "migrate.h"

void migrate(int nid, void (*callback)(void *), void *callback_data);

callback is invoked with callback_data as the argument after the thread is migrated. You can pass NULL if you are not interested in the callback. The execution is resumed at the next line of the migrate function on node nid.

Limitations

  • Bring back migrated-away threads to their original node before exiting from the program.

  • A migrated thread should be brought back to the original node to be relocated to other node. For example, to migrate a thread which was created at node 0 and is running at node 1, the thread should be migrated back to node 0 before it is migrated to node 2 again.

  • You can do file I/O at any location, but file descriptors are not synchronized between migrated threads. This means, console is not migrated and printf() might not work neither. To check the execution progress, we recommend to open() a file at an absolute path, write() to the file, and close() for each print-out. File operations (e.g., fopen(), fwrite()) might not work neither.

  • You cannot create a thread while the thread is running at a remote node.

  • Networking is not supported.

Build your own applications

The compiler toolchain provides a template Makefile for building the multi-ISA binary. The template file can be found at popcorn-compiler/util/Makefile.pyalign.template. Copy the file to your project directory and edit accordingly.

  • In most cases, it is sufficient to put source files into SRC and set the executable name to BIN. For example:
BIN := test

SRC := hello.c world.c
  • Build multi-ISA binaries
$ make
  • You will find $(BIN)_aarch64 and $(BIN)_x86-64 in the directory, which are the executables.
$ ls
... test_aarch64   test_x86-64  ...
  • The executable should be at the same absolute path arcoss the machines.
$ scp test_x86-64 test_aarch64 [email protected]:
$ scp test_x86-64 test_aarch64 [email protected]:
  • Make test point to the proper-ISA executable on each machine:
popcorn@x86:~$ cp test_x86-64  test
popcorn@arm:~$ cp test_aarch64 test
  • Now, you can run the executable with ${BIN} which is test in this example.
Clone this wiki locally