USHIFT-4336: Implement embedded container build support in bootc tests #3921

ggiguash · 2024-09-13T13:05:56Z

When rehearsing the current change, detected a few unrelated problems and opened follow-up issues:

openshift-ci-robot · 2024-09-13T13:06:01Z

@ggiguash: This pull request references USHIFT-4336 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.18.0" version, but no target version was set.

In response to this:

Which issue(s) this PR addresses:

Closes #

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-09-13T13:06:33Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

ggiguash · 2024-09-13T13:10:15Z

Everything seems to be working fine, except the router pod that fails to start with the following error.

[pod/router-default-74c8bbb85b-fs5bw/router] I0913 13:09:11.500833       1 router.go:537] "msg"=
"calling reload function" "fn"=0 "logger"="template"
[pod/router-default-74c8bbb85b-fs5bw/router] E0913 13:09:11.501025       1 haproxy.go:418] can't
 scrape HAProxy: dial unix /var/lib/haproxy/run/haproxy.sock: connect: no such file or directory
[pod/router-default-74c8bbb85b-fs5bw/router] I0913 13:09:11.501047       1 router.go:541] "msg"=
"reloading the router" "logger"="template"
[pod/router-default-74c8bbb85b-fs5bw/router] E0913 13:09:11.522860       1 limiter.go:165] error
 reloading router: exit status 1
[pod/router-default-74c8bbb85b-fs5bw/router] [NOTICE]   (9) : haproxy version is 2.8.10-f28885f
[pod/router-default-74c8bbb85b-fs5bw/router] [NOTICE]   (9) : path to executable is /usr/sbin/ha
proxy
[pod/router-default-74c8bbb85b-fs5bw/router] [ALERT]    (9) : Binding [/var/lib/haproxy/conf/hap
roxy.config:63] for frontend public: cannot bind socket (Permission denied) for [0.0.0.0:80]
[pod/router-default-74c8bbb85b-fs5bw/router] [ALERT]    (9) : Binding [/var/lib/haproxy/conf/hap
roxy.config:98] for frontend public_ssl: cannot bind socket (Permission denied) for [0.0.0.0:443
]
[pod/router-default-74c8bbb85b-fs5bw/router] [ALERT]    (9) : [/usr/sbin/haproxy.main()] Some pr
otocols failed to start their listeners! Exiting.

ggiguash · 2024-09-13T13:34:36Z

/test ?

openshift-ci · 2024-09-13T13:34:40Z

@ggiguash: The following commands are available to trigger required jobs:

/test footprint-and-performance
/test images
/test metal-bootc-periodic-test
/test metal-bootc-periodic-test-arm
/test metal-bootc-test
/test metal-bootc-test-arm
/test metal-periodic-test
/test metal-periodic-test-arm
/test microshift-metal-cache
/test microshift-metal-cache-arm
/test microshift-metal-tests
/test microshift-metal-tests-arm
/test ocp-conformance-rhel-eus
/test ocp-conformance-rhel-eus-arm
/test ocp-full-conformance-rhel-eus
/test ocp-full-conformance-serial-rhel-eus
/test test-rpm
/test test-unit
/test verify

The following commands are available to trigger optional jobs:

/test test-rebase

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-microshift-main-images
pull-ci-openshift-microshift-main-metal-bootc-periodic-test
pull-ci-openshift-microshift-main-metal-bootc-periodic-test-arm
pull-ci-openshift-microshift-main-metal-bootc-test
pull-ci-openshift-microshift-main-metal-bootc-test-arm
pull-ci-openshift-microshift-main-metal-periodic-test
pull-ci-openshift-microshift-main-metal-periodic-test-arm
pull-ci-openshift-microshift-main-microshift-metal-tests
pull-ci-openshift-microshift-main-microshift-metal-tests-arm
pull-ci-openshift-microshift-main-test-unit
pull-ci-openshift-microshift-main-verify

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ggiguash · 2024-09-13T13:34:57Z

/test metal-bootc-periodic-test-arm

ggiguash · 2024-09-16T16:55:20Z

/test metal-bootc-periodic-test

ggiguash · 2024-09-17T06:03:31Z

/test metal-bootc-periodic-test

ggiguash · 2024-09-17T10:41:15Z

/test metal-bootc-periodic-test-arm
/test metal-bootc-periodic-test

ggiguash · 2024-09-17T10:57:17Z

/test verify

ggiguash · 2024-09-17T12:39:11Z

/test metal-bootc-periodic-test
/test metal-bootc-periodic-test-arm
/test verify

ggiguash · 2024-09-17T13:35:54Z

The isolated container builds exhibit failures. Asked about it in the PBI tracker issue.

STEP 1/14: FROM localhost/cos9-bootc-source:latest
STEP 2/14: RUN --mount=type=secret,id=pullsecret,dst=/run/secrets/pull-secret.json     podman pull     --authfile /run/secrets/pull-secret.json     --root=/var/lib/containers/storage-preloaded     "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0c12ddea1c28a7287700c19e623c4bf66ce9931a6b8a8c882fd80204393996c6"
time="2024-09-17T13:05:47Z" level=warning msg="\"/\" is not a shared mount, this could cause issues or missing mounts with rootless containers"
Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0c12ddea1c28a7287700c19e623c4bf66ce9931a6b8a8c882fd80204393996c6...
Getting image source signatures
Copying blob sha256:4f05dc73d94cbbd0cd91587184e671ba4112087f93f86139ebdcb3afa3b1aae7
Copying blob sha256:d8e5548c38deb03c17871de35e63257444e05109754c25725185b8bf2630cfce
Copying blob sha256:0ea779877a0c9ea774298c27e45ffa6589e7b157fef28dad9d6869859cf8077f
Copying blob sha256:abced7ee58448ba09b150b6a5d0083ceb67e2e834226c951cc0c678891596db9
Copying config sha256:fd1a10b5a8443d0436fb6447b24b6aee75848995909962aa1c681e5942c70a72
Writing manifest to image destination
fd1a10b5a8443d0436fb6447b24b6aee75848995909962aa1c681e5942c70a72
--> 79b9daa6d479
STEP 3/14: RUN --mount=type=secret,id=pullsecret,dst=/run/secrets/pull-secret.json     podman pull     --authfile /run/secrets/pull-secret.json     --root=/var/lib/containers/storage-preloaded     "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:69219ccc460d06c01cfbbf5617795ed28eb3834704af811e8bf3469ace763bd3"
cannot set user namespace

ggiguash · 2024-09-18T12:57:21Z

/test metal-bootc-periodic-test-arm
/test metal-bootc-periodic-test

ggiguash · 2024-09-19T07:32:28Z

/test metal-bootc-periodic-test-arm

openshift-ci · 2024-09-19T07:32:32Z

@ggiguash: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-footprint-and-performance
/test e2e-aws-tests
/test e2e-aws-tests-arm
/test e2e-aws-tests-bootc
/test e2e-aws-tests-bootc-arm
/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm
/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm
/test e2e-aws-tests-periodic
/test e2e-aws-tests-periodic-arm
/test images
/test ocp-conformance-rhel-eus
/test ocp-conformance-rhel-eus-arm
/test ocp-full-conformance-rhel-eus
/test ocp-full-conformance-serial-rhel-eus
/test test-rpm
/test test-unit
/test verify

The following commands are available to trigger optional jobs:

/test test-rebase

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-microshift-main-e2e-aws-tests
pull-ci-openshift-microshift-main-e2e-aws-tests-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-periodic
pull-ci-openshift-microshift-main-e2e-aws-tests-periodic-arm
pull-ci-openshift-microshift-main-images
pull-ci-openshift-microshift-main-test-unit
pull-ci-openshift-microshift-main-verify

In response to this:

/test metal-bootc-periodic-test-arm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ggiguash · 2024-09-19T07:32:50Z

/test e2e-aws-tests-bootc-periodic

ggiguash · 2024-09-20T07:18:54Z

/test e2e-aws-tests-bootc-periodic e2e-aws-tests-bootc-periodic-arm
/test verify

pacevedom

/lgtm

openshift-ci-robot · 2024-09-20T11:18:45Z

/retest-required

Remaining retests: 0 against base HEAD 8f33f9a and 2 for PR HEAD cf2d62d in total

openshift-ci-robot · 2024-09-20T14:00:33Z

/retest-required

Remaining retests: 0 against base HEAD 2931085 and 2 for PR HEAD cf2d62d in total

pacevedom

/lgtm
Good change!

openshift-ci-robot · 2024-09-20T21:57:33Z

/retest-required

Remaining retests: 0 against base HEAD 2931085 and 2 for PR HEAD 8aa122f in total

ggiguash · 2024-09-21T03:57:51Z

/test e2e-aws-tests-bootc-periodic-arm

openshift-ci · 2024-09-22T05:13:36Z

@ggiguash: Overrode contexts on behalf of ggiguash: ci/prow/e2e-aws-tests-bootc-periodic-arm

In response to this:

Overriding the test as its failures have nothing to do with the current change.
/override ci/prow/e2e-aws-tests-bootc-periodic-arm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot · 2024-09-22T05:27:21Z

@ggiguash: This pull request references USHIFT-4336 which is a valid jira issue.

In response to this:

When rehearsing the current change, detected a few unrelated problems and opened follow-up issues:

USHIFT-4518

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2024-09-22T05:31:53Z

@ggiguash: This pull request references USHIFT-4336 which is a valid jira issue.

In response to this:

When rehearsing the current change, detected a few unrelated problems and opened follow-up issues:

The ostree2bootc upgrade test may fail when switching the layer

The osconfig-lifecycle test may fail in CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ggiguash · 2024-09-22T09:38:00Z

/test metal-bootc-periodic-test metal-bootc-periodic-test-arm

openshift-ci · 2024-09-22T09:38:04Z

@ggiguash: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-footprint-and-performance
/test e2e-aws-tests
/test e2e-aws-tests-arm
/test e2e-aws-tests-bootc
/test e2e-aws-tests-bootc-arm
/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm
/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm
/test e2e-aws-tests-periodic
/test e2e-aws-tests-periodic-arm
/test images
/test ocp-conformance-rhel-eus
/test ocp-conformance-rhel-eus-arm
/test ocp-full-conformance-rhel-eus
/test ocp-full-conformance-serial-rhel-eus
/test test-rpm
/test test-unit
/test verify

The following commands are available to trigger optional jobs:

/test test-rebase

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-microshift-main-e2e-aws-tests
pull-ci-openshift-microshift-main-e2e-aws-tests-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-periodic
pull-ci-openshift-microshift-main-e2e-aws-tests-periodic-arm
pull-ci-openshift-microshift-main-images
pull-ci-openshift-microshift-main-test-unit
pull-ci-openshift-microshift-main-verify

In response to this:

/test metal-bootc-periodic-test metal-bootc-periodic-test-arm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ggiguash · 2024-09-22T09:38:25Z

/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

ggiguash · 2024-09-22T11:09:31Z

/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

ggiguash · 2024-09-22T12:36:23Z

/test all

ggiguash · 2024-09-22T16:38:20Z

/test metal-bootc-periodic-test-arm
/test metal-bootc-periodic-test

openshift-ci · 2024-09-22T16:38:23Z

@ggiguash: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-footprint-and-performance
/test e2e-aws-tests
/test e2e-aws-tests-arm
/test e2e-aws-tests-bootc
/test e2e-aws-tests-bootc-arm
/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm
/test e2e-aws-tests-cache
/test e2e-aws-tests-cache-arm
/test e2e-aws-tests-periodic
/test e2e-aws-tests-periodic-arm
/test images
/test ocp-conformance-rhel-eus
/test ocp-conformance-rhel-eus-arm
/test ocp-full-conformance-rhel-eus
/test ocp-full-conformance-serial-rhel-eus
/test test-rpm
/test test-unit
/test verify

The following commands are available to trigger optional jobs:

/test test-rebase

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-microshift-main-e2e-aws-tests
pull-ci-openshift-microshift-main-e2e-aws-tests-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic
pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic-arm
pull-ci-openshift-microshift-main-e2e-aws-tests-periodic
pull-ci-openshift-microshift-main-e2e-aws-tests-periodic-arm
pull-ci-openshift-microshift-main-images
pull-ci-openshift-microshift-main-test-unit
pull-ci-openshift-microshift-main-verify

In response to this:

/test metal-bootc-periodic-test-arm
/test metal-bootc-periodic-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ggiguash · 2024-09-22T16:39:00Z

/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

ggiguash · 2024-09-22T19:56:16Z

/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

ggiguash · 2024-09-23T04:40:24Z

/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

openshift-ci · 2024-09-23T06:05:19Z

@ggiguash: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/metal-bootc-periodic-test-arm	`0e65fa7`	link	true	`/test metal-bootc-periodic-test-arm`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

ggiguash · 2024-09-23T06:42:25Z

Now that we have VM install retry-on-error fix merged, let's see the results. #3953

/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

P.S. There might still be errors due to #3952

openshift-ci-robot · 2024-09-23T06:44:23Z

@ggiguash: This pull request references USHIFT-4336 which is a valid jira issue.

In response to this:

When rehearsing the current change, detected a few unrelated problems and opened follow-up issues:

Exit with non-zero status from onerror kickstart handler

The ostree2bootc upgrade test may fail when switching the layer

The osconfig-lifecycle test may fail in CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2024-09-23T06:45:18Z

@ggiguash: This pull request references USHIFT-4336 which is a valid jira issue.

In response to this:

When rehearsing the current change, detected a few unrelated problems and opened follow-up issues:

The onerror handler in CI kickstart reboots the VM without returning failure

The ostree2bootc upgrade test may fail when switching the layer

The osconfig-lifecycle test may fail in CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

pacevedom

/lgtm

openshift-ci · 2024-09-23T07:15:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ggiguash, pacevedom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ggiguash,pacevedom]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2024-09-23T09:57:44Z

/retest-required

Remaining retests: 0 against base HEAD fa8e7de and 2 for PR HEAD 7a98ec5 in total

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 13, 2024

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 13, 2024

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 13, 2024

ggiguash force-pushed the isolated_images branch from 0e65fa7 to f581359 Compare September 20, 2024 06:57

ggiguash marked this pull request as ready for review September 20, 2024 09:25

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 20, 2024

openshift-ci bot requested review from jerpeter1 and pliurh September 20, 2024 09:26

pacevedom approved these changes Sep 20, 2024

View reviewed changes

openshift-ci bot assigned pacevedom Sep 20, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 20, 2024

pacevedom reviewed Sep 20, 2024

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 20, 2024

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Sep 22, 2024

Add retry to skopeo copy command

4020bdb

Limit skopeo max process number to 8

7a98ec5

pacevedom approved these changes Sep 23, 2024

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 23, 2024

openshift-merge-bot bot merged commit 3bea1fc into openshift:main Sep 23, 2024
12 checks passed

ggiguash deleted the isolated_images branch September 23, 2024 16:31

USHIFT-4336: Implement embedded container build support in bootc tests #3921

USHIFT-4336: Implement embedded container build support in bootc tests #3921

Conversation

ggiguash commented Sep 13, 2024 • edited Loading

openshift-ci-robot commented Sep 13, 2024 • edited by openshift-ci bot Loading

openshift-ci bot commented Sep 13, 2024

ggiguash commented Sep 13, 2024

ggiguash commented Sep 13, 2024

openshift-ci bot commented Sep 13, 2024

ggiguash commented Sep 13, 2024

ggiguash commented Sep 16, 2024

ggiguash commented Sep 17, 2024

ggiguash commented Sep 17, 2024

ggiguash commented Sep 17, 2024

ggiguash commented Sep 17, 2024

ggiguash commented Sep 17, 2024

ggiguash commented Sep 18, 2024

ggiguash commented Sep 19, 2024

openshift-ci bot commented Sep 19, 2024

ggiguash commented Sep 19, 2024

ggiguash commented Sep 20, 2024

pacevedom left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Sep 20, 2024

openshift-ci-robot commented Sep 20, 2024

pacevedom left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Sep 20, 2024

ggiguash commented Sep 21, 2024

openshift-ci bot commented Sep 22, 2024

openshift-ci-robot commented Sep 22, 2024 • edited by openshift-ci bot Loading

openshift-ci-robot commented Sep 22, 2024 • edited by openshift-ci bot Loading

ggiguash commented Sep 22, 2024

openshift-ci bot commented Sep 22, 2024

ggiguash commented Sep 22, 2024

ggiguash commented Sep 22, 2024

ggiguash commented Sep 22, 2024

ggiguash commented Sep 22, 2024

openshift-ci bot commented Sep 22, 2024

ggiguash commented Sep 22, 2024

ggiguash commented Sep 22, 2024

ggiguash commented Sep 23, 2024

openshift-ci bot commented Sep 23, 2024 • edited Loading

ggiguash commented Sep 23, 2024

openshift-ci-robot commented Sep 23, 2024 • edited by openshift-ci bot Loading

openshift-ci-robot commented Sep 23, 2024 • edited by openshift-ci bot Loading

pacevedom left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Sep 23, 2024

openshift-ci-robot commented Sep 23, 2024

ggiguash commented Sep 13, 2024 •

edited

Loading

openshift-ci-robot commented Sep 13, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 22, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 22, 2024 •

edited by openshift-ci bot

Loading

openshift-ci bot commented Sep 23, 2024 •

edited

Loading

openshift-ci-robot commented Sep 23, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 23, 2024 •

edited by openshift-ci bot

Loading