From 7e00a498d582a845a6a7dd6f2a58f8a93816111d Mon Sep 17 00:00:00 2001 From: Rune Morling Date: Wed, 17 Apr 2024 11:32:44 +0200 Subject: [PATCH] samba: Add autobuild.yml files for dep resolution MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **Summary** This is the initial attempt at showing how to use the power of autobuild for getting the Solus monorepo under control re. dependency cycles. The purely additive nature of the autobuild.yml configuration files added with this commit, should make it trivial to modify and extend the ignored set of dependencies for a package as our cycle-discovery journey advances. To gain an appreciation of just how useful this tool might become for us, consider the following: ``` ermo@solbox:~/repos/getsolus/soluspkgs $ autobuild -q query src:packages/ samba ldb tdb talloc tevent talloc tdb cifs-utils notmuch tevent ldb samba acccheck ffmpeg \ python-pysmbc gvfs kio-extras kodi mpd rhythmbox vlc \ gnome-control-center nautilus-share nemo-extensions budgie-control-center ``` The above ^ shows how autobuild might resolve a build + rebuild order for when someone is doing a samba stack upgrade and pushing it to a single builder; like, say, the local packager's machine or the current Solus build infrastructure, which is limited to one builder. Note that autobuild also already supports committing+pushing to the Solus builder, which implies that it can replace the current `build_pr.sh` script in short order, if the the appetite to do so is present within Solus as an organisation. But that's not all! In fact, `autobuild` also supports generating a "tiered" builder order (notice the `-t` argument), which those of you packaging KDE might already be familiar with: ``` ermo@solbox:~/repos/getsolus/soluspkgs $ autobuild query -t src:packages/ samba ldb tdb talloc tevent (.. some output omitted ..) 🗸  Successfully parsed state! 🗸  Found all requested packages in state! 🗸  Successfully built dependency graph! 🗸  Build order: 🗸  Tier 1: talloc tdb 🗸  Tier 2: cifs-utils notmuch tevent 🗸  Tier 3: ldb 🗸  Tier 4: samba 🗸  Tier 5: acccheck ffmpeg python-pysmbc 🗸  Tier 6: gvfs kio-extras kodi mpd rhythmbox vlc 🗸  Tier 7: gnome-control-center nautilus-share nemo-extensions 🗸  Tier 8: budgie-control-center ``` Tiers (called "Precedence Constraint Groups" in graph theory literature) effectively show which packages can be built in parallel since they do not interdepend on each other. With the -q argument like above, this build order could potentially drive a set of builders (like summit does in the Serpent infra currently): ``` ermo@solbox:~/repos/getsolus/soluspkgs $ autobuild -qt query src:packages/ samba ldb tdb talloc tevent talloc tdb cifs-utils notmuch tevent ldb samba acccheck ffmpeg python-pysmbc gvfs kio-extras kodi mpd rhythmbox vlc budgie-control-center gnome-control-center nautilus-share nemo-extensions ``` I hope this serves to show how useful autobuild can potentially become to Serpent and Solus in terms of automating the discovery of build and rebuild orders. This commit is mostly intended as a way to showcase what an autobuild- driven packaging process might look like for Solus and Serpent, and as a starting point for the discussion of how to tag "Auxiliary Dependencies" in a way that's compatible with both the Solus and Serpent monorepos and tooling. See also: #2175 and #2176 autobuild.yaml format design notes here: https://github.com/GZGavinZhao/autobuild/issues/16 Massive props to @GZGavinZhao for his interest in studying the necessary graph theory at my suggestion, and also for his being ready, able, and willing to design and build this tool, and for his willingness to take on board constructive feedback re. the documentation and UX necessary to productise this solution for a wider audience of Solus staff and community packagers. Signed-off-by: Rune Morling --- common/autobuild-ignore-doxygen.yml | 6 ++++++ common/autobuild-ignore-python-pytest.yml | 5 +++++ packages/l/llvm-bolt/autobuild.yml | 7 +++++++ packages/l/llvm/autobuild.yml | 1 + packages/m/mesalib/autobuild.yml | 5 +++++ packages/o/openjdk-17/autobuild.yml | 4 ++++ packages/p/pulseaudio/autobuild.yml | 7 +++++++ packages/py/python-chardet/autobuild.yml | 1 + packages/py/python-jinja/autobuild.yml | 7 +++++++ packages/py/python-requests/autobuild.yml | 1 + packages/py/python-wheel/autobuild.yml | 1 + packages/t/tpm2-tss/autobuild.yml | 1 + 12 files changed, 46 insertions(+) create mode 100644 common/autobuild-ignore-doxygen.yml create mode 100644 common/autobuild-ignore-python-pytest.yml create mode 100644 packages/l/llvm-bolt/autobuild.yml create mode 120000 packages/l/llvm/autobuild.yml create mode 100644 packages/m/mesalib/autobuild.yml create mode 100644 packages/o/openjdk-17/autobuild.yml create mode 100644 packages/p/pulseaudio/autobuild.yml create mode 120000 packages/py/python-chardet/autobuild.yml create mode 100644 packages/py/python-jinja/autobuild.yml create mode 120000 packages/py/python-requests/autobuild.yml create mode 120000 packages/py/python-wheel/autobuild.yml create mode 120000 packages/t/tpm2-tss/autobuild.yml diff --git a/common/autobuild-ignore-doxygen.yml b/common/autobuild-ignore-doxygen.yml new file mode 100644 index 00000000000..209a40a4f75 --- /dev/null +++ b/common/autobuild-ignore-doxygen.yml @@ -0,0 +1,6 @@ +# doxygen is a common enough cause for cycles that it deserves its own file +# which can be trivially symlinked to. +solver: + ignore: + - doxygen + diff --git a/common/autobuild-ignore-python-pytest.yml b/common/autobuild-ignore-python-pytest.yml new file mode 100644 index 00000000000..67bd2769235 --- /dev/null +++ b/common/autobuild-ignore-python-pytest.yml @@ -0,0 +1,5 @@ +# python-pytest is a common cycle-creator, in that it's typically a checkdep, +# not a true builddep. +solver: + ignore: + - python-pytest diff --git a/packages/l/llvm-bolt/autobuild.yml b/packages/l/llvm-bolt/autobuild.yml new file mode 100644 index 00000000000..b30bf785048 --- /dev/null +++ b/packages/l/llvm-bolt/autobuild.yml @@ -0,0 +1,7 @@ +# A rebuild in git shouldn't really trigger a rebuild of llvm-bolt, and upon +# inspection of llvm-bolt's recipe, we see that the only reason git is in its +# builddeps is probably because of build system requirements, so we can safely +# ignore git here. +solver: + ignore: + - git diff --git a/packages/l/llvm/autobuild.yml b/packages/l/llvm/autobuild.yml new file mode 120000 index 00000000000..36c576567c8 --- /dev/null +++ b/packages/l/llvm/autobuild.yml @@ -0,0 +1 @@ +../../../common/autobuild-ignore-doxygen.yml \ No newline at end of file diff --git a/packages/m/mesalib/autobuild.yml b/packages/m/mesalib/autobuild.yml new file mode 100644 index 00000000000..13c530e5ad8 --- /dev/null +++ b/packages/m/mesalib/autobuild.yml @@ -0,0 +1,5 @@ +# linux-driver-management is a rundep of mesalib, but a rebuild of LDM +# really shouldn't trigger a rebuild of mesalib, so ignore it. +solver: + ignore: + - linux-driver-management diff --git a/packages/o/openjdk-17/autobuild.yml b/packages/o/openjdk-17/autobuild.yml new file mode 100644 index 00000000000..738698fab49 --- /dev/null +++ b/packages/o/openjdk-17/autobuild.yml @@ -0,0 +1,4 @@ +# jtreg is a test harness for testing the JDK, so can be ignored in builddeps. +solver: + ignore: + - jtreg diff --git a/packages/p/pulseaudio/autobuild.yml b/packages/p/pulseaudio/autobuild.yml new file mode 100644 index 00000000000..f21004a99ad --- /dev/null +++ b/packages/p/pulseaudio/autobuild.yml @@ -0,0 +1,7 @@ +# python3-qt5 is a rundep of pulseaudio-equalizer, so we can ignore python3-qt5 +# when considering pulseaudio rebuilds. +# If we don't, a python3-qt5 rebuild could trigger a qt6-base rebuild via PA +# which does not seem reasonable. +solver: + ignore: + - python3-qt5 diff --git a/packages/py/python-chardet/autobuild.yml b/packages/py/python-chardet/autobuild.yml new file mode 120000 index 00000000000..522caca4c20 --- /dev/null +++ b/packages/py/python-chardet/autobuild.yml @@ -0,0 +1 @@ +../../../common/autobuild-ignore-python-pytest.yml \ No newline at end of file diff --git a/packages/py/python-jinja/autobuild.yml b/packages/py/python-jinja/autobuild.yml new file mode 100644 index 00000000000..c852de82ab0 --- /dev/null +++ b/packages/py/python-jinja/autobuild.yml @@ -0,0 +1,7 @@ +# python-pytest and python-markupsafe shouldn't trigger a rebuild of this, +# as these are marked as checkdeps. +solver: + ignore: + - python-markupsafe + - python-pytest + diff --git a/packages/py/python-requests/autobuild.yml b/packages/py/python-requests/autobuild.yml new file mode 120000 index 00000000000..522caca4c20 --- /dev/null +++ b/packages/py/python-requests/autobuild.yml @@ -0,0 +1 @@ +../../../common/autobuild-ignore-python-pytest.yml \ No newline at end of file diff --git a/packages/py/python-wheel/autobuild.yml b/packages/py/python-wheel/autobuild.yml new file mode 120000 index 00000000000..522caca4c20 --- /dev/null +++ b/packages/py/python-wheel/autobuild.yml @@ -0,0 +1 @@ +../../../common/autobuild-ignore-python-pytest.yml \ No newline at end of file diff --git a/packages/t/tpm2-tss/autobuild.yml b/packages/t/tpm2-tss/autobuild.yml new file mode 120000 index 00000000000..36c576567c8 --- /dev/null +++ b/packages/t/tpm2-tss/autobuild.yml @@ -0,0 +1 @@ +../../../common/autobuild-ignore-doxygen.yml \ No newline at end of file