3. Tool build- and install scripts¶
The entire project consists mainly of folders, which contain two scripts and sometimes a configuration file. The folder is named after the tool or the collection of tools, which are installed by the scripts contained within. One script does install the build essentials, using the apt package manager as it’s primary source. The other script pulls, configures, builds and installs the tool in question. All scripts can be found in this documentation in Script and configuration index. The usage of those tool build and install scripts is described in section Tool build and install scripts.
In addition to scripts for every single tool, a major fully configurable script exists, which automatically builds and installs all tools and projects, for which a tool build script exists and for which the installation flag is toggled in the configuration file. For more details, skip to section Fully automated and configurable tools and projects install script
3.2. Tool build and install scripts¶
This section describes how to configure and use the tool build and install scripts.
Before attempting to install the tools, you have to install some build-essentials like make, compilers and the python interpreter. You only have to execute this script once on a specific machine. Locally browse to build_tools and execute install_build_essentials.sh as a superuser:
The scripts are structured similarly and most of the time offer identical configuration options. Let us simulate the usage of one tool together, using explanations of the configuration options and what the script does internally. Browse to build_tools/verilator. This folder contains the two script:
This is a common naming pattern in this project, you can replace verilator by the names of other tools supported by this project. Both scripts require superuser privileges. To install the build essentials, the apt install command is used, that requires superuser privileges. Furthermore to install the built script, superuser privileges are required. The script could be designed such that superuser privileges are requested when required. By using this alternative approach, a fully automatic sequential installation of all tools would not be possible if the user does forget to run the scripts as superuser, because after a certain time the user must type in the superuser credentials again. You should install the software required to build the tool before building it by invoking the install_<toolname>_essentials.sh script, in this case:
After the build essentials have been installed, we can build and install the tool. Let’s check out the parameters by executing the script with the -h option:
This prints the following output (for verilator):
install_verilator.sh [-h] [-c] [-d dir] [-i path] [-t tag] -- Clone latested tagged verilator version and build it. Optionally select the build directory and version, install binaries and cleanup setup files. where: -h show this help text -c cleanup project -d dir build files in "dir" (default: build_and_install_verilator) -i path install binaries to path (use "default" to use default path) -t tag specify version (git tag or commit hash) to pull (default: Latest tag)
The -c, -d, -i and -t options are default options that are available for every tool build and install script.
The script creates a build folder, in which the source code for the project is being pulled into and in which temporary files might be stored. The name of the build folder can be specified by using the -d flag.
The source code version that should be pulled can be specified by using the -t flag. You can specify a branch name, tag, commit hash or one of the following options:
default/latest: Pulls the default branch
stable: Pulls the latest tag
The default behaviour (in case -t was not specified) is to pull the default branch. Before using the stable option, be sure to check whether the repository stopped to use tags at some point in time. If this is the case, the script will pull and use an outdated version, because it does not check timestamps. If no tags are found, the default branch is used.
The scripts only builds the tools by default. To also install them (using the default path specified in the tool itself), execute the script with the -i flag. The -i flag takes one parameter, which is used to specify the install path. Set it to default to use the default install path preconfigured within the tool in question.
The last default flag is the -c flag, which deletes all files after the tool has been successfully installed. It is only relevant if the -i flag is supplied at the same invocation. Otherwise a tool that was build but not installed would be removed, which is obviously pointless because it is equivalent to no changes at all.
Some tools have additional parameters which should be documented well enough in the output of the -h flag.
If the tool build essentials have been installed and the invocation of the tool is realized with superuser privileges and correct parameters, the script will fully automatically install the tool in question. Note that the build and/or installation process can be canceled by the SIGINT or SIGTERM signals, the default behavior of the scripts is to remove any files created by the script though. Therefore any progress will be lost.
3.3. Fully automated and configurable tools and projects install script¶
This section describes how to configure and use the major tools and projects install script.
The script depends on a configuration file, which specifies which tools and projects should be installed and how they are configured. This file is located in build_tools/config.cfg. The configuration parameters should be commented well enough to be understood, but let’s take a look at Verilators configuration section
184.108.40.206. Tool configuration¶
## Verilator # Build and (if desired) install Verilator? VERILATOR=true # Build AND install Verilator? VERILATOR_INSTALL=true # Install path (default = default path) VERILATOR_INSTALL_PATH=default # Remove build directory after successful install? VERILATOR_CLEANUP=true # Folder name in which the project is built VERILATOR_DIR=default # Specify project version to pull (default/latest, stable, tag, branch, hash) VERILATOR_TAG=default
The configuration parameter names for tools follow the name conception TOOLNAME_PARAMETER=VALUE. The
TOOL=true flag specifies whether this tool should be build and optionally installed or whether it should be ignored. Other than that, the four basic tool build and install script flags, that were described in Tool build and install script parameters, are mirrored by the config parameters followed by
TOOL=true. This is the minimal configuration, at the same time it is the complete set of configuration parameters for most of the tools.
220.127.116.11. Project configuration¶
Beside configuration entries for tools, projects can also be configured. The configuration is identical for every project and looks like this:
## Pqvexriscv project # Download git repository PQRISCV_VEXRISCV=false # Git URL PQRISCV_VEXRISCV_URL="https://github.com/mupq/pqriscv-vexriscv.git" # Specify project version to pull (default/latest, stable, tag, branch, hash) PQRISCV_VEXRISCV_TAG=default # If default is selected, the project is stored in the documents folder # of each user listed in the variable PQRISCV_VEXRISCV_USER PQRISCV_VEXRISCV_LOCATION=default # Space separated list of users (in quotation marks) to install the project for # in /home/$user/Documents (if PQRISCV_VEXRISCV_LOCATION=default). # default = all logged in users. Linking to desktop is also based on this list. PQRISCV_VEXRISCV_USER=default # Symbolic link to /home/$user/Desktop PQRISCV_VEXRISCV_LINK_TO_DESKTOP=true
The configuration parameter names for projects follow the name conception PROJECT_PARAMETER=VALUE. You can toggle whether you’d like the project to be installed by specifying
PROJECT=true. Currently the projects are limited to projects that can be pulled by using git. The git repository url can be specified as an HTTP-link in the
PROJECT_URL=HTTPURL parameter. The state of the git repository that should be used is reflected in the
PROJECT_TAG=STATE parameter. STATE can take the same values as the -t flag from the Tool build and install script parameters. By specifying
PROJECT_LOCATION=PATH you can control where the project is copied to. Leaving this value at default does use the documents folder inside the home directory of the user specified in the variable
PROJECT_USER=USER. If PROJECT_USER is default, any logged on user will be regarded. Finally, it is possible to configure whether the project is linked to the desktop of the user by specifying
After configuring the tools and projects that shall be installed by adjusting config.cfg, execute the install script install_everything.sh and toggle the -h parameter (note that the real execution requires superuser privileges):
It should emit the following output:
install_everything.sh [-c] [-h] [-o] [-p] [-v] [-d dir] -- Build and install QuantumRisc toolchain. where: -c cleanup, delete everything after successful execution -h show this help text -o space seperated list of users who shall be added to dialout (default: every logged in user) -p space seperated list of users for whom the version file shall be copied to the desktop (default: every logged in user) -v be verbose (spams the terminal) -d dir build files in "dir" (default: build_and_install_quantumrisc_tools)
The parameters -c and -d are equal to the default parameters mentioned in Tool build and install script parameters.
The -o parameter is used to specify the users who are added to the dialout group. By default (if -o is not set), the install script installs all tools and projects for every user who is logged in during the installation process. -o can by used in a scenario where the install script is configured to install the tools and projects for a single user or a set of users.
The -p parameter lets us control which users get a copy of the version file. This file is explained in the following section Version file. Identical to the behavior of -o, -p does target all logged on users by default.
The -v parameter enables or disables the verbose output. By default, only the current operations are printed to the console. This keeps the console relatively clean. Note that errors are still logged in a file (see Error file). By setting the -v parameter, every output is passed to the console. This includes compiler logs, which spam the console.
The default behavior of the script in case it receives SIGINT or SIGTERM signals, is to leave everything as it was before receiving the signal and to terminate the script. Nevertheless, the tool build script will delete the tool build folder in that case.
18.104.22.168. Version file¶
Every single tool installation script does log the version the tool was build for in a file called installed_version.txt. The major tools and projects installation script, that is covered in this chapter, does collect the information from the version file of every tool that was build into a file called installed_versions.txt. The file is copied to the desktop of each user, who was specified by the -p parameter (every logged on user by default). This file can be used for instance when releasing a new QuantumRisc-VM version or when publishing a paper. The contents of the version file look like this:
Yosys: 0.9 Project-Trellis: fef7e5fd16354c2911673635dd78e2dae3a775c0 Icestorm: d12308775684cf43ab923227235b4ad43060015e Nextpnr-ice40: e6991ad5dc79f6118838f091cc05f10d3377eb4a Nextpnr-ecp5: b39a2a502065ec1407417ffacdac2154385bf80f Ujprog: 0698352b0e912caa9b8371b8f692e19aac547a69 OpenOCD: 9ed6707716b72a88ba6b31219b766c1562aec8d0 OpenOCD-Vexriscv: b77b41cf06d8981f3cf10c639d0f65d8ee6498b8 Verilog: v4.038 GTKWave: e049b936203c5a9b8e48de48a3d505e4e33e3d65 RiscV-GNU-Toolchain-linux-multilib: 256a4108922f76403a63d6567501c479971d5575 qemu-linux-multilib: 134b7dec6ec2d90616d7986afb3b3b7ca7a4c383 riscv_binutils-linux-multilib: 2.34 riscv_dejagnu-linux-multilib: 1.6 riscv_gcc-linux-multilib: 10.1.0 riscv_gdb-linux-multilib: 9.1 riscv_glibc-linux-multilib: 2.29 RiscV-GNU-Toolchain-newlib-multilib: 256a4108922f76403a63d6567501c479971d5575 qemu-newlib-multilib: 134b7dec6ec2d90616d7986afb3b3b7ca7a4c383 riscv_binutils-newlib-multilib: 2.34 riscv_dejagnu-newlib-multilib: 1.6 riscv_gcc-newlib-multilib: 10.1.0 riscv_gdb-newlib-multilib: 9.1 riscv_newlib-newlib-multilib: 3.2.0
22.214.171.124. Error file¶
Any errors that occur during the execution of the install_everything.sh script are logged in the build directory, whose name is specified by the -d or whose name is set to the default value “build_and_install_quantumrisc_tools” if -d was not set. The file is named “errors.log”. If -v is not set, the error messages are only redirected to this file. If -v is set, the error messages are additionally printed in the console.
The install_everything.sh script does remember which tools or projects have been successfully installed. By default, this information is stored inside the build directory in a file that’s called “latest_success_tools.txt”. For projects, by default a file named “latest_success_projects.txt” is used. If the execution of this script is canceled by the user or an error, the script remembers the state and during the next execution offers the user to continue were it stopped. The user can either decide to go on or start over. If the script terminated successfully, the user can only decide to install the latest tool or project in case the build directory was not cleaned up (id est -c was not set).
All projects are only downloaded using the version that was specified in the configuration file config.cfg. The downloaded files are placed in the “Documents” folder inside the home folder of all users who were specified in the configuration file. In addition, a symbolic link to the projects is placed on the desktop. Currently this part only works on English systems, because the folder names “Documents” and “Desktop” are hard-coded.
126.96.36.199. Deriving a configuration file with fixed tool versions¶
The project offers a script that does derive a configuration from another configuration file and a version file. The final configuration file is a duplicate of the original configuration file, but it locks the tool versions to those used in the version file.
Once you installed a set of tools using the install_everything.sh script and the corresponding config.cfg file, a file called installed_versions.txt is created in the build folder and in the same folder the install_everything.sh file is located in. A script called versiondump_to_config.sh is located in the misc_tools folder. This script can be invoked to derive the configuration file that locks the tool versions:
versiondump_to_config.sh [-h] versionfile configfile outputfile
Given a specific git commit hash or tag of the QuantumRisc-VM build tools and the derived configuration file, anybody can recreate the exact same set of tools on their own system.