Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A nested pipeline's environment does not pass through to a uses step. #1548

Open
smoser opened this issue Oct 8, 2024 · 0 comments
Open

Comments

@smoser
Copy link
Contributor

smoser commented Oct 8, 2024

I was trying to use environment from a nested pipeline in a uses stanza. The environment variable didnt' pass through as I wanted.

$ melange version | grep ^[A-Z]
GitVersion:    v0.12.1-7-g94476be
GitCommit:     94476be97a34c3d0b0dece6c4b2ec596960b1b34
GitTreeState:  clean
BuildDate:     '2024-09-26T17:58:32Z'
GoVersion:     go1.23.1
Compiler:      gc
Platform:      linux/amd64

Given the two files:

  1. test-me.yaml
# test-me.yaml
package:
  name: test-me
  version: 1.0
  epoch: 0

environment:
  contents:
    packages:
      - busybox
  environment:
    BAR: "the other day I saw a BAR"

pipeline:
  - environment:
      FOO: "WINNER WINNER"
    pipeline:
      - runs: |
          echo "==== FOO=$FOO ==="
          echo "==== BAR=$BAR ==="
      - uses: test/dump-env
  1. pipelines/test/dump-env
name: dump your environment
pipeline:
  - name: "show env"
    runs: |
      fails=0
      [ -n "$FOO" ] && echo "PASS: FOO got $FOO" ||
        { echo "FAIL: FOO was empty"; fails=$((fails+1)); }
      [ -n "$BAR" ] && echo "PASS: BAR got $BAR" ||
        { echo "FAIL: BAR was empty"; fails=$((fails+1)); }
      exit $fails

build command and output are below. you can see that the 'run' section gets both the global environment (BAR) and the "local" environment ("FOO"), but the uses pipeline does not get access to the local variable (FOO).

$ melange build test-me.yaml --repository-append=./packages \
  --keyring-append=local-melange.rsa.pub \
  --signing-key=local-melange.rsa \
  --arch=x86_64 --pipeline-dir=./pipelines/  \
  -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub \
  -r https://packages.wolfi.dev/os
...
2024/10/08 15:00:11 INFO ==== FOO=WINNER WINNER ===
2024/10/08 15:00:11 INFO ==== BAR=the other day I saw a BAR ===
2024/10/08 15:00:11 INFO running step "test/dump-env"
2024/10/08 15:00:11 INFO running step "show env" uses=test/dump-env
2024/10/08 15:00:11 INFO FAIL: FOO was empty uses=test/dump-env name="show env"
2024/10/08 15:00:11 INFO PASS: BAR got the other day I saw a BAR uses=test/dump-env name="show env"
2024/10/08 15:00:11 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/10/08 15:00:11 INFO   workspace dir: /tmp/melange-workspace-3334864414
2024/10/08 15:00:11 INFO   guest dir: /tmp/melange-guest-1999173818
2024/10/08 15:00:11 ERRO failed to build package: unable to run package test-me pipeline: unable to run pipeline: unable to run pipeline: unable to run pipeline: exit status 1
smoser added a commit to smoser/wolfi-os that referenced this issue Oct 8, 2024
A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

       CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.
smoser added a commit to smoser/wolfi-os that referenced this issue Oct 8, 2024
A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

       CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.
smoser added a commit to smoser/wolfi-os that referenced this issue Oct 8, 2024
A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

       CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.
smoser added a commit to wolfi-dev/os that referenced this issue Oct 8, 2024
…30285)

A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.

7. provide multiple versions of the python modules, and make them
provide 'pyarrow' for backwards compat.
gdonval pushed a commit to gdonval/wolfi-os that referenced this issue Oct 9, 2024
…olfi-dev#30285)

A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.

7. provide multiple versions of the python modules, and make them
provide 'pyarrow' for backwards compat.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant