OTEL refactor

[andylibrian]

This PR is currently not ready for merge.

• Agent: Use OpenTelemetry logs to publish events in favor of direct access to NATS
• Client: Introduce OpenTelemetry collector in the KubViz client cluster
• Implement a basic version of OpenTelemetry NATS exporter
• Update helm charts to accomodate the new infra

[dryrunsecurity]

DryRun Security Summary

The pull request includes a wide range of changes to improve the observability, monitoring, and overall security posture of the application, including GitHub Actions workflows for building and pushing Docker images, Kubernetes deployment configurations, OpenTelemetry and Kuberhealthy integrations, and Dgraph database integration.

Expand for full summary

Summary:

The code changes in this pull request cover a wide range of components and features, including GitHub Actions workflows for building and pushing Docker images, Kubernetes deployment configurations, OpenTelemetry and Kuberhealthy integrations, and Dgraph database integration. From an application security perspective, the changes appear to be focused on improving the observability, monitoring, and overall security posture of the application.

Key security considerations include:

1. Ensuring secure configuration and management of the OpenTelemetry Collector, NATS messaging, and Dgraph database components to prevent data exposure and unauthorized access.
2. Implementing robust input validation and data sanitization mechanisms to mitigate potential vulnerabilities, such as injection attacks, during the processing of Kubernetes resources and other data.
3. Maintaining up-to-date dependencies and container images to address known security vulnerabilities.
4. Implementing appropriate access controls and least-privilege principles for the GitHub Actions workflows and Kubernetes deployments.
5. Continuous monitoring and logging of the application's security-related events and activities.

Overall, the changes seem to be a step in the right direction for improving the security and reliability of the application, but ongoing vigilance and review will be necessary to ensure the application's security posture remains strong.

Files Changed:

1. GitHub Actions Workflows:

   • .github/workflows/agent-container-pr.yml: Changes related to building and pushing a Docker container image for the "container-agent" component.
   • .github/workflows/agent-git-pr.yml: Changes related to building and pushing a Docker image for a Git agent.
   • .github/workflows/client-pr.yml: Changes related to building and pushing a Docker image for the "Client" component.
   • .github/workflows/agent-kubviz-pr.yml: Changes related to building and pushing a Docker image for the "kubviz-agent" component.
   • .github/workflows/migration-pr.yml: Changes related to building and pushing a Docker image for a migration component.
   • .github/workflows/otel-collector-image.yml: Changes related to building, signing, and pushing a Docker image for the OpenTelemetry Collector.
   • .github/workflows/otel-collector-release.yml: Changes related to building, signing, and pushing a Docker image for the OpenTelemetry Collector during a release.
   • .github/workflows/otel-collector-pr.yml: Changes related to building and pushing a Docker image for the OpenTelemetry Collector on pull requests.

2. Configuration Files:

   • .gitignore: Changes related to ignoring certain files and directories in the Git repository.
   • agent/kubviz/k8smetrics_agent.go: Changes related to the functionality of the "kubviz" agent, including the use of a dynamic Kubernetes client and the addition of new plugins.
   • agent/config/config.go: Changes related to the configuration of the "agent" application, including the addition of new configuration parameters.
   • agent/kubviz/scheduler/scheduler.go: Changes related to the scheduler component of the "kubviz" agent, including the addition of a new job to retrieve and process all Kubernetes resources.
   • agent/kubviz/scheduler/scheduler_watch.go: Changes related to the implementation of the "KubeAllResourcesJob" in the "kubviz" agent scheduler.
   • charts/agent/values.yaml: Changes related to the configuration of the "agent" component, including the enablement of OpenTelemetry and Kuberhealthy.
   • charts/client/templates/configmap-otel-collector-config.yaml: Changes related to the configuration of the OpenTelemetry Collector for the "client" component.
   • charts/client/templates/deployment-otel-collector.yaml: Changes related to the Kubernetes Deployment for the OpenTelemetry Collector in the "client" component.
   • charts/client/templates/service-otel-collector.yaml: Changes related to the Kubernetes Service for the OpenTelemetry Collector in the "client" component.
   • charts/client/templates/deployment.yaml: Changes related to the Kubernetes Deployment for the "client" component, including the integration with Dgraph and OpenTelemetry.
   • charts/client/values.yaml: Changes related to the configuration of the "client" component, including the integration with Dgraph, Grafana, and various

Code Analysis

We ran 9 analyzers against 30 files and 0 analyzers had findings. 9 analyzers had no findings.

View PR in the DryRun Dashboard.

[github-actions]

Message that will be displayed on users' first pull request

[devopstoday11]

@andylibrian please check the failed CI jobs which are relevant to us for now