added github actions dependencies

README.md

@@ -1,21 +1,56 @@
-Drivechain (BIPs 300+301)
--------------------------
+## CI Scripts
 
-Drivechain allows Bitcoin to create, delete, send BTC to, and receive BTC from “Layer-2”s called “sidechains”. Sidechains are Altcoins that lack a native “coin” – instead, BTC must first be sent over.
+This directory contains scripts for each build step in each build stage.
 
-BIP 300:
-https://github.com/bitcoin/bips/blob/master/bip-0300.mediawiki
+### Running a Stage Locally
 
-BIP 301:
-https://github.com/bitcoin/bips/blob/master/bip-0301.mediawiki
+Be aware that the tests will be built and run in-place, so please run at your own risk.
+If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
 
-Learn more about Drivechain here:
-http://drivechain.info
+The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory.
+While it should be fine to run
+the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and
+testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci
+system in a virtual machine with a Linux operating system of your choice.
 
-For an example sidechain implementation, see: https://github.com/drivechain-project/sidechains
+To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
+requires `bash`, `docker`, and `python3` to be installed. To install all requirements on Ubuntu, run
 
-License
--------
+```
+sudo apt install bash docker.io python3
+```
 
-Bitcoin Core (and Drivechain) are released under the terms of the MIT license. See [COPYING](COPYING) for more
-information or see https://opensource.org/licenses/MIT.
+It is recommended to run the ci system in a clean env. To run the test stage
+with a specific configuration,
+
+```
+env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
+```
+
+### Configurations
+
+The test files (`FILE_ENV`) are constructed to test a wide range of
+configurations, rather than a single pass/fail. This helps to catch build
+failures and logic errors that present on platforms other than the ones the
+author has tested.
+
+Some builders use the dependency-generator in `./depends`, rather than using
+the system package manager to install build dependencies. This guarantees that
+the tester is using the same versions as the release builds, which also use
+`./depends`.
+
+It is also possible to force a specific configuration without modifying the
+file. For example,
+
+```
+env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
+```
+
+The files starting with `0n` (`n` greater than 0) are the scripts that are run
+in order.
+
+### Cache
+
+In order to avoid rebuilding all dependencies for each build, the binaries are
+cached and reused when possible. Changes in the dependency-generator will
+trigger cache-invalidation and rebuilds as necessary.

ci/README.md

@@ -0,0 +1,56 @@
+## CI Scripts
+
+This directory contains scripts for each build step in each build stage.
+
+### Running a Stage Locally
+
+Be aware that the tests will be built and run in-place, so please run at your own risk.
+If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
+
+The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory.
+While it should be fine to run
+the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and
+testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci
+system in a virtual machine with a Linux operating system of your choice.
+
+To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
+requires `bash`, `docker`, and `python3` to be installed. To install all requirements on Ubuntu, run
+
+```
+sudo apt install bash docker.io python3
+```
+
+It is recommended to run the ci system in a clean env. To run the test stage
+with a specific configuration,
+
+```
+env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
+```
+
+### Configurations
+
+The test files (`FILE_ENV`) are constructed to test a wide range of
+configurations, rather than a single pass/fail. This helps to catch build
+failures and logic errors that present on platforms other than the ones the
+author has tested.
+
+Some builders use the dependency-generator in `./depends`, rather than using
+the system package manager to install build dependencies. This guarantees that
+the tester is using the same versions as the release builds, which also use
+`./depends`.
+
+It is also possible to force a specific configuration without modifying the
+file. For example,
+
+```
+env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
+```
+
+The files starting with `0n` (`n` greater than 0) are the scripts that are run
+in order.
+
+### Cache
+
+In order to avoid rebuilding all dependencies for each build, the binaries are
+cached and reused when possible. Changes in the dependency-generator will
+trigger cache-invalidation and rebuilds as necessary.

ci/lint/04_install.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+#
+# Copyright (c) 2018-2022 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+export LC_ALL=C
+
+export PATH=$PWD/ci/retry:$PATH
+
+${CI_RETRY_EXE} apt-get update
+# Lint dependencies:
+# - curl/xz-utils (to install shellcheck)
+# - git (used in many lint scripts)
+# - gpg (used by verify-commits)
+${CI_RETRY_EXE} apt-get install -y curl xz-utils git gpg
+
+PYTHON_PATH="/python_build"
+if [ ! -d "${PYTHON_PATH}/bin" ]; then
+  (
+    ${CI_RETRY_EXE} git clone https://github.com/pyenv/pyenv.git
+    cd pyenv/plugins/python-build || exit 1
+    ./install.sh
+  )
+  # For dependencies see https://github.com/pyenv/pyenv/wiki#suggested-build-environment
+  ${CI_RETRY_EXE} apt-get install -y build-essential libssl-dev zlib1g-dev \
+    libbz2-dev libreadline-dev libsqlite3-dev curl llvm \
+    libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
+    clang
+  env CC=clang python-build "$(cat "./.python-version")" "${PYTHON_PATH}"
+fi
+export PATH="${PYTHON_PATH}/bin:${PATH}"
+command -v python3
+python3 --version
+
+export LINT_RUNNER_PATH="/lint_test_runner"
+if [ ! -d "${LINT_RUNNER_PATH}" ]; then
+  ${CI_RETRY_EXE} apt-get install -y cargo
+  (
+    cd ./test/lint/test_runner || exit 1
+    cargo build
+    mkdir -p "${LINT_RUNNER_PATH}"
+    mv target/debug/test_runner "${LINT_RUNNER_PATH}"
+  )
+fi
+
+${CI_RETRY_EXE} pip3 install \
+  codespell==2.2.5 \
+  flake8==6.1.0 \
+  lief==0.13.2 \
+  mypy==1.4.1 \
+  pyzmq==25.1.0 \
+  vulture==2.6
+
+SHELLCHECK_VERSION=v0.8.0
+curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | \
+    tar --xz -xf - --directory /tmp/
+mv "/tmp/shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/

ci/lint/06_script.sh

@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+#
+# Copyright (c) 2018-2022 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+export LC_ALL=C
+
+set -ex
+
+if [ -n "$LOCAL_BRANCH" ]; then
+  # To faithfully recreate CI linting locally, specify all commits on the current
+  # branch.
+  COMMIT_RANGE="$(git merge-base HEAD master)..HEAD"
+elif [ -n "$CIRRUS_PR" ]; then
+  COMMIT_RANGE="HEAD~..HEAD"
+  echo
+  git log --no-merges --oneline "$COMMIT_RANGE"
+  echo
+  test/lint/commit-script-check.sh "$COMMIT_RANGE"
+else
+  COMMIT_RANGE="SKIP_EMPTY_NOT_A_PR"
+fi
+export COMMIT_RANGE
+
+RUST_BACKTRACE=1 "${LINT_RUNNER_PATH}/test_runner"
+
+if [ "$CIRRUS_REPO_FULL_NAME" = "bitcoin/bitcoin" ] && [ "$CIRRUS_PR" = "" ] ; then
+    # Sanity check only the last few commits to get notified of missing sigs,
+    # missing keys, or expired keys. Usually there is only one new merge commit
+    # per push on the master branch and a few commits on release branches, so
+    # sanity checking only a few (10) commits seems sufficient and cheap.
+    git log HEAD~10 -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
+    git log HEAD~10 -1 --format='%H' > ./contrib/verify-commits/trusted-git-root
+    mapfile -t KEYS < contrib/verify-commits/trusted-keys
+    git config user.email "[email protected]"
+    git config user.name "ci"
+    ${CI_RETRY_EXE} gpg --keyserver hkps://keys.openpgp.org --recv-keys "${KEYS[@]}" &&
+    ./contrib/verify-commits/verify-commits.py;
+fi

ci/lint/container-entrypoint.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+#
+# Copyright (c) The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or https://opensource.org/license/mit/.
+
+export LC_ALL=C
+
+# Fixes permission issues when there is a container UID/GID mismatch with the owner
+# of the mounted bitcoin src dir.
+git config --global --add safe.directory /bitcoin
+
+export PATH="/python_build/bin:${PATH}"
+export LINT_RUNNER_PATH="/lint_test_runner"
+
+if [ -z "$1" ]; then
+  LOCAL_BRANCH=1 bash -ic "./ci/lint/06_script.sh"
+else
+  exec "$@"
+fi

ci/lint_imagefile

@@ -0,0 +1,24 @@
+# Copyright (c) The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or https://opensource.org/license/mit/.
+
+# See test/lint/README.md for usage.
+
+FROM debian:bookworm
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LC_ALL=C.UTF-8
+
+COPY ./.python-version /.python-version
+COPY ./ci/lint/container-entrypoint.sh /entrypoint.sh
+COPY ./ci/lint/04_install.sh /install.sh
+COPY ./test/lint/test_runner /test/lint/test_runner
+
+RUN /install.sh && \
+  echo 'alias lint="./ci/lint/06_script.sh"' >> ~/.bashrc && \
+  chmod 755 /entrypoint.sh && \
+  rm -rf /var/lib/apt/lists/*
+
+
+WORKDIR /bitcoin
+ENTRYPOINT ["/entrypoint.sh"]

ci/lint_run_all.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+#
+# Copyright (c) 2019-2020 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+export LC_ALL=C.UTF-8
+
+set -o errexit; source ./ci/test/00_setup_env.sh
+set -o errexit; source ./ci/lint/04_install.sh
+set -o errexit
+./ci/lint/06_script.sh