Skip to content

Commit

Permalink
samba: Add autobuild.yml files for dep resolution (#2279)
Browse files Browse the repository at this point in the history
**Summary**
This is the initial attempt at showing how to use the power of autobuild
for getting the Solus monorepo under control re. dependency cycles. It
is based on [this
gist](https://gist.github.com/GZGavinZhao/4981fb5ba874ac82364b18b039e3f495).

The purely additive nature of the autobuild.yml configuration files
added with this commit, should make it trivial to modify and extend the
ignored set of dependencies for a package as our cycle-discovery journey
advances.

To gain an appreciation of just how useful this tool might become for
us, consider the following:

```
ermo@solbox:~/repos/getsolus/soluspkgs
$ autobuild -q query src:packages/ samba ldb tdb talloc tevent
talloc tdb cifs-utils notmuch tevent ldb samba acccheck ffmpeg python-pysmbc gvfs kio-extras kodi mpd rhythmbox vlc gnome-control-center nautilus-share nemo-extensions budgie-control-center
```

The above ^ shows how autobuild might resolve a build + rebuild order
for when someone is doing a samba stack upgrade and pushing it to a
single builder; like, say, the local packager's machine or the current
Solus build infrastructure, which is limited to one builder.

Note that autobuild also already supports committing+pushing to the
Solus builder, which implies that it can replace the current
`build_pr.sh` script in short order, if the the appetite to do so is
present within Solus as an organisation.

But that's not all!

In fact, `autobuild` also supports generating a "tiered" builder order
(notice the `-t` argument), which those of you packaging KDE might
already be familiar with:

```
ermo@solbox:~/repos/getsolus/soluspkgs
$ autobuild query -t src:packages/ samba ldb tdb talloc tevent
 (.. some output omitted ..)
 🗸  Successfully parsed state!
 🗸  Found all requested packages in state!
 🗸  Successfully built dependency graph!
 🗸  Build order:
 🗸  Tier 1: talloc tdb
 🗸  Tier 2: cifs-utils notmuch tevent
 🗸  Tier 3: ldb
 🗸  Tier 4: samba
 🗸  Tier 5: acccheck ffmpeg python-pysmbc
 🗸  Tier 6: gvfs kio-extras kodi mpd rhythmbox vlc
 🗸  Tier 7: gnome-control-center nautilus-share nemo-extensions
 🗸  Tier 8: budgie-control-center
```

Tiers (called "Precedence Constraint Groups" in graph theory literature)
effectively show which packages can be built in parallel since they do
not interdepend on each other.

With the -q argument like above, this build order could potentially
drive a set of builders (like summit does in the Serpent infra
currently):

```
ermo@solbox:~/repos/getsolus/soluspkgs
$ autobuild -qt query src:packages/ samba ldb tdb talloc tevent
talloc tdb
cifs-utils notmuch tevent
ldb
samba
acccheck ffmpeg python-pysmbc
gvfs kio-extras kodi mpd rhythmbox vlc
budgie-control-center gnome-control-center nautilus-share nemo-extensions
```

I hope this serves to show how useful autobuild can potentially become
to Serpent and Solus in terms of automating the discovery of build and
rebuild orders.

This commit is mostly intended as a way to showcase what an
autobuild-driven packaging process might look like for Solus and
Serpent, and as a starting point for the discussion of how to tag
"Auxiliary Dependencies" in a way that's compatible with both the Solus
and Serpent monorepos and tooling.

See also: #2175 and #2176

autobuild.yaml format design notes here: GZGavinZhao/autobuild#16

Massive props to @GZGavinZhao for his interest in studying the necessary
graph theory at my suggestion, and also for his being ready, able, and
willing to design and build this tool, and for his willingness to take
on board constructive feedback re. the documentation and UX necessary to
productise this solution for a wider audience of Solus staff and
community packagers.
  • Loading branch information
ermo authored May 1, 2024
2 parents 4673aff + 7e00a49 commit c5175af
Show file tree
Hide file tree
Showing 12 changed files with 46 additions and 0 deletions.
6 changes: 6 additions & 0 deletions common/autobuild-ignore-doxygen.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# doxygen is a common enough cause for cycles that it deserves its own file
# which can be trivially symlinked to.
solver:
ignore:
- doxygen

5 changes: 5 additions & 0 deletions common/autobuild-ignore-python-pytest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# python-pytest is a common cycle-creator, in that it's typically a checkdep,
# not a true builddep.
solver:
ignore:
- python-pytest
7 changes: 7 additions & 0 deletions packages/l/llvm-bolt/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# A rebuild in git shouldn't really trigger a rebuild of llvm-bolt, and upon
# inspection of llvm-bolt's recipe, we see that the only reason git is in its
# builddeps is probably because of build system requirements, so we can safely
# ignore git here.
solver:
ignore:
- git
1 change: 1 addition & 0 deletions packages/l/llvm/autobuild.yml
5 changes: 5 additions & 0 deletions packages/m/mesalib/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# linux-driver-management is a rundep of mesalib, but a rebuild of LDM
# really shouldn't trigger a rebuild of mesalib, so ignore it.
solver:
ignore:
- linux-driver-management
4 changes: 4 additions & 0 deletions packages/o/openjdk-17/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# jtreg is a test harness for testing the JDK, so can be ignored in builddeps.
solver:
ignore:
- jtreg
7 changes: 7 additions & 0 deletions packages/p/pulseaudio/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# python3-qt5 is a rundep of pulseaudio-equalizer, so we can ignore python3-qt5
# when considering pulseaudio rebuilds.
# If we don't, a python3-qt5 rebuild could trigger a qt6-base rebuild via PA
# which does not seem reasonable.
solver:
ignore:
- python3-qt5
1 change: 1 addition & 0 deletions packages/py/python-chardet/autobuild.yml
7 changes: 7 additions & 0 deletions packages/py/python-jinja/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# python-pytest and python-markupsafe shouldn't trigger a rebuild of this,
# as these are marked as checkdeps.
solver:
ignore:
- python-markupsafe
- python-pytest

1 change: 1 addition & 0 deletions packages/py/python-requests/autobuild.yml
1 change: 1 addition & 0 deletions packages/py/python-wheel/autobuild.yml
1 change: 1 addition & 0 deletions packages/t/tpm2-tss/autobuild.yml

0 comments on commit c5175af

Please sign in to comment.