Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Refactor to OCB based agent #1755

Merged
merged 21 commits into from
Aug 2, 2024
Merged

feat: Refactor to OCB based agent #1755

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
341dece
wip
dpaasman00 Jul 8, 2024
c67327c
refactor repo
dpaasman00 Jul 11, 2024
788dedb
update workflow go versions
dpaasman00 Jul 11, 2024
16db092
update build flow to install-tools
dpaasman00 Jul 11, 2024
0c0421b
update build license check
dpaasman00 Jul 12, 2024
401ae72
new make target
dpaasman00 Jul 15, 2024
a901484
rename to manifests
dpaasman00 Jul 15, 2024
638dd22
update release workflow
dpaasman00 Jul 16, 2024
4a48e30
package supervisor with agent
dpaasman00 Jul 17, 2024
fd376a3
resolve errs with gcp exporters
dpaasman00 Jul 17, 2024
0f9731b
update version to have go.mod
dpaasman00 Jul 17, 2024
aba2439
get google exporters correctly importing version
dpaasman00 Jul 17, 2024
1f382ac
manifest documentation
dpaasman00 Jul 17, 2024
f818c47
use updated version commit in google exporters
dpaasman00 Jul 17, 2024
c00e2fc
include supervisor as prebuilt binary
dpaasman00 Jul 18, 2024
9308e53
remove config.yaml from release pkg since its not used by supervisor/…
dpaasman00 Jul 18, 2024
9df9443
feedback 1/n
dpaasman00 Jul 29, 2024
5436dbc
rebase cleanup
dpaasman00 Jul 29, 2024
3227ba5
update go version in a few actions, use https git clone for contrib
dpaasman00 Jul 31, 2024
fdecdaf
change tool cache and fix release-test
dpaasman00 Aug 1, 2024
90989c4
replace rest of tool caches
dpaasman00 Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
go-version: "1.21"
check-latest: true
- name: Cache Tools
id: tool-cache
uses: actions/cache@v3
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/codeql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
go-version: "1.21"
check-latest: true

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/manual_msi_build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
go-version: "1.21"
check-latest: true
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
Expand Down
58 changes: 57 additions & 1 deletion .github/workflows/multi_build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,58 @@ on:
pull_request:

jobs:
setup-tools:
runs-on: "ubuntu-20.04"
steps:
- name: Checkout Sources
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: "1.21"
check-latest: true
- name: Cache Tools
id: tool-cache
uses: actions/cache@v3
with:
path: /home/runner/go/bin
key: tooling-${{ runner.os }}-${{ hashFiles('./Makefile') }}
BinaryFissionGames marked this conversation as resolved.
Show resolved Hide resolved
- name: Install Tools
if: steps.tool-cache.outputs.cache-hit != 'true'
run: make install-tools

build_linux:
runs-on: ubuntu-latest-4-cores
needs:
- setup-tools
steps:
- name: Checkout Sources
uses: actions/checkout@v4
- name: Install Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
go-version: "1.21"
check-latest: true
- name: Cache Tools
id: tool-cache
uses: actions/cache@v3
with:
path: /home/runner/go/bin
key: tooling-${{ runner.os }}-${{ hashFiles('./Makefile') }}
BinaryFissionGames marked this conversation as resolved.
Show resolved Hide resolved
- name: Install Tools
if: steps.tool-cache.outputs.cache-hit != 'true'
run: make install-tools
- name: Build
run: make build-linux
- name: Scan Third Party Dependency Licenses
run: |
go install github.com/uw-labs/[email protected]
lichen --config=./license.yaml $(find dist/collector_* dist/updater_*)

build_darwin:
runs-on: macos-14
needs:
- setup-tools
steps:
- name: Checkout Sources
uses: actions/checkout@v4
Expand All @@ -31,14 +66,26 @@ jobs:
with:
go-version: "1.21"
check-latest: true
- name: Cache Tools
id: tool-cache
uses: actions/cache@v3
with:
path: /home/runner/go/bin
key: tooling-${{ runner.os }}-${{ hashFiles('./Makefile') }}
- name: Install Tools
if: steps.tool-cache.outputs.cache-hit != 'true'
run: make install-tools
- name: Build
run: make build-darwin
- name: Scan Third Party Dependency Licenses
run: |
go install github.com/uw-labs/[email protected]
lichen --config=./license.yaml $(find dist/collector_* dist/updater_*)

build_windows:
runs-on: ubuntu-20.04
needs:
- setup-tools
steps:
- name: Checkout Sources
uses: actions/checkout@v4
Expand All @@ -47,6 +94,15 @@ jobs:
with:
go-version: "1.21"
check-latest: true
- name: Cache Tools
id: tool-cache
uses: actions/cache@v3
with:
path: /home/runner/go/bin
key: tooling-${{ runner.os }}-${{ hashFiles('./Makefile') }}
- name: Install Tools
if: steps.tool-cache.outputs.cache-hit != 'true'
run: make install-tools
- name: Build
run: make build-windows
- name: Scan Third Party Dependency Licenses
Expand Down
21 changes: 6 additions & 15 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,14 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: "**/go.sum"
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
# either 'goreleaser' (default) or 'goreleaser-pro'
distribution: goreleaser-pro
version: "v1.22.1"
args: build --single-target --skip=validate --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
go-version: "1.21"
check-latest: true
- name: Build Windows Binaries
run: make build-binaries
- name: Copy Windows Collector Binary
run: cp dist/windows_amd64/collector_windows_amd64_v1/observiq-otel-collector.exe windows/observiq-otel-collector.exe
run: cp dist/collector_windows_amd64.exe windows/observiq-otel-collector.exe
- name: Copy Windows Updater Binary
run: cp dist/windows_amd64/updater_windows_amd64_v1/updater.exe windows/updater.exe
run: cp dist/updater_windows_amd64.exe windows/updater.exe
- name: Copy Plugins to MSI Build Directory
run: cp -r release_deps/plugins windows/
- name: Copy Example Config
Expand Down Expand Up @@ -79,7 +71,6 @@ jobs:
with:
go-version: "1.21"
check-latest: true
cache-dependency-path: "**/go.sum"
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
go-version: "1.21"
check-latest: true
jsirianni marked this conversation as resolved.
Show resolved Hide resolved
- name: Run Tests
run: make test
- name: Run Updater Integration Tests (non-linux)
Expand Down
7 changes: 6 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,9 @@ release_deps
# OpAmp Files
collector*.yaml
manager*.yaml
logging*.yaml
logging*.yaml

#Supervisor & Builder Files
builder
agent*.log
effective*.yaml
69 changes: 40 additions & 29 deletions .goreleaser.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,18 @@ project_name: observiq-otel-collector
before:
hooks:
- make release-prep CURR_VERSION={{ .Version }}
- make build-all OUTDIR="tmp"

after:
hooks:
- cmd: rm -rf tmp
BinaryFissionGames marked this conversation as resolved.
Show resolved Hide resolved

# https://goreleaser.com/customization/build/
builds:
# https://goreleaser.com/customization/builds/#import-pre-built-binaries
- id: collector
builder: prebuilt
binary: observiq-otel-collector
main: ./cmd/collector
env:
- CGO_ENABLED=0
mod_timestamp: "{{ .CommitTimestamp }}"
goos:
- windows
Expand All @@ -35,18 +39,11 @@ builds:
goarch: ppc64
- goos: darwin
goarch: ppc64le
ldflags:
- -s -w
- -X github.com/observiq/bindplane-agent/internal/version.version=v{{ .Version }}
- -X github.com/observiq/bindplane-agent/internal/version.gitHash={{ .FullCommit }}
- -X github.com/observiq/bindplane-agent/internal/version.date={{ .Date }}
no_unique_dist_dir: false
prebuilt:
path: tmp/collector_{{ .Os }}_{{ .Arch }}{{ .Ext }}
- id: updater
builder: prebuilt
binary: updater
dir: ./updater/
main: ./cmd/updater
env:
- CGO_ENABLED=0
mod_timestamp: "{{ .CommitTimestamp }}"
goos:
- windows
Expand All @@ -71,12 +68,36 @@ builds:
goarch: ppc64
- goos: darwin
goarch: ppc64le
ldflags:
- -s -w
- -X github.com/observiq/bindplane-agent/updater/internal/version.version=v{{ .Version }}
- -X github.com/observiq/bindplane-agent/updater/internal/version.gitHash={{ .FullCommit }}
- -X github.com/observiq/bindplane-agent/updater/internal/version.date={{ .Date }}
no_unique_dist_dir: false
prebuilt:
path: tmp/updater_{{ .Os }}_{{ .Arch }}{{ .Ext }}
- id: supervisor
builder: prebuilt
binary: opampsupervisor
goos:
- windows
- linux
- darwin
goarch:
- amd64
- arm64
- arm
- ppc64
- ppc64le
ignore:
- goos: windows
goarch: arm
- goos: windows
goarch: arm64
- goos: windows
goarch: ppc64
- goos: windows
goarch: ppc64le
- goos: darwin
goarch: ppc64
- goos: darwin
goarch: ppc64le
prebuilt:
path: release_deps/supervisor_bin/opampsupervisor_{{ .Os }}_{{ .Arch }}{{ .Ext }}

# https://goreleaser.com/customization/archive/
archives:
Expand All @@ -87,9 +108,6 @@ archives:
- src: release_deps/opentelemetry-java-contrib-jmx-metrics.jar
dst: "."
strip_parent: true
- src: release_deps/config.yaml
dst: "."
strip_parent: true
Comment on lines -90 to -92
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you have to remove this from the nfpms section as well?

make release-test errors for me: nfpm failed for observiq-otel-collector_v1.56.0-SNAPSHOT-3edff07f_linux_ppc64.rpm: matching "./release_deps/config.yaml": file does not exist

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea good call, I think I removed it in one of my later pr's but forgot to do so here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this file is still referenced in the dockers section, which causes release-test to fail. Can we remove config.yaml form there too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, blanked on that

- src: release_deps/logging.yaml
dst: "."
strip_parent: true
Expand Down Expand Up @@ -138,13 +156,6 @@ nfpms:
mode: 0755
owner: observiq-otel-collector
group: observiq-otel-collector
- src: release_deps/config.yaml
dst: /opt/observiq-otel-collector/config.yaml
file_info:
mode: 0640
owner: observiq-otel-collector
group: observiq-otel-collector
type: config|noreplace
- src: release_deps/logging.yaml
dst: /opt/observiq-otel-collector/logging.yaml
file_info:
Expand Down
Loading
Loading