Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: Refactor for better maintainability #17

Merged
merged 6 commits into from
Oct 4, 2024

Conversation

anurag-rajawat
Copy link
Contributor

@anurag-rajawat anurag-rajawat commented Sep 15, 2024

Description

Part of #26

This PR introduces a significant change to the access-log observability implementation:

  • Replaces istio telemetry with envoy wasm plugin:

    • This PR replaces the existing Istio telemetry object-based approach with the envoy wasm plugin for access-log observability.
    • This enables deployment of SentryFlow without sidecar configuration injection via annotation, configmap creation containing the wasm filter binary, telemetry object creation, or workload restarts (assuming an existing Istio sidecar mesh).
    • This enables SentryFlow to gain visibility into the Layer 7 behaviour of sidecar-injected K8s workloads within the cluster, without requiring the creation of any Istio telemetry objects.
  • Configurable Knobs:

    • Adds configuration options to configure functionalities within receivers, exporter or SentryFlow itself.
  • New APIEvent Protobuf Message:

    • Introduces a new protobuf message APIEvent for structured API call event data.
  • Exposes API Event Collection Endpoints:
    Out of the box, SentryFlow exposes two endpoints for receiving API events from external sources:

    • HTTP Endpoint: An HTTP POST endpoint at /api/v1/events on port 8081 allows external sources to send API events in the defined APIEvent structure.
    • gRPC Endpoint: A gRPC service method SendAPIEvent (running on the configured .exporter.grpc.port) provides another option for sending API events.

Does this PR introduce a breaking change?

Yes, this PR introduces a breaking change due to the shift from istio telemetry objects to the Wasm plugin for access log observability.

Additional Information for Reviewers

Docs and test cases will be incorporated after core functionalities are reviewed and approved.

To try it locally

  1. Install Istio
  2. Build image and deploy it by applying deployments/sentryflow.yaml manifest file:
    Execute following from project's root directory:
REGISTRY=<your-registry> DOCKER_TAG=<some-tag> make -C sentryflow image push
kubectl apply -f deployments/sentryflow.yaml
  1. Enable the envoy proxy injection by labelling the namespace in which you'll deploy workload:
kubectl label ns <namespace_name> istio-injection=enabled
  1. Deploy some workload and generate traffic by calling APIs.
  2. Forward sentryflow's gRPC port 8080 to get the logs for called APIs:
kubectl -n sentryflow port-forward svc/sentryflow 8080:8080
  1. Extract and use the following client to get the logs:
    client.tgz
tar xf client.tgz
./sf-client

@anurag-rajawat anurag-rajawat self-assigned this Sep 15, 2024
@anurag-rajawat anurag-rajawat changed the title refactor: Refactor for better maintainability refactor!: Refactor for better maintainability Sep 15, 2024
@anurag-rajawat anurag-rajawat force-pushed the refactor-sentryflow branch 4 times, most recently from 4d99f7c to e0494ca Compare September 17, 2024 05:18
@anurag-rajawat anurag-rajawat force-pushed the refactor-sentryflow branch 7 times, most recently from 4861ce0 to b337b1f Compare September 17, 2024 19:57
@anurag-rajawat anurag-rajawat marked this pull request as ready for review September 18, 2024 06:07
@anurag-rajawat anurag-rajawat force-pushed the refactor-sentryflow branch 4 times, most recently from cfd4482 to ada1f37 Compare September 18, 2024 08:58
@anurag-rajawat anurag-rajawat marked this pull request as draft September 18, 2024 10:31
@anurag-rajawat anurag-rajawat removed the request for review from nam-jaehyun September 23, 2024 04:17
@anurag-rajawat anurag-rajawat marked this pull request as draft September 23, 2024 05:07
@anurag-rajawat anurag-rajawat force-pushed the refactor-sentryflow branch 2 times, most recently from 8dacfa4 to ea6074d Compare September 23, 2024 06:34
@anurag-rajawat anurag-rajawat removed the request for review from JonesJefferson September 23, 2024 06:36
@anurag-rajawat anurag-rajawat marked this pull request as ready for review September 23, 2024 06:51
@nandhued
Copy link

@daemon1024 Can you review?

Copy link
Collaborator

@daemon1024 daemon1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial Review, PTAL

sentryflow/Dockerfile Outdated Show resolved Hide resolved
sentryflow/pkg/receiver/svcmesh/istio/sidecar/sidecar.go Outdated Show resolved Hide resolved
sentryflow/pkg/receiver/svcmesh/istio/sidecar/sidecar.go Outdated Show resolved Hide resolved
sentryflow/pkg/receiver/svcmesh/istio/sidecar/sidecar.go Outdated Show resolved Hide resolved
sentryflow/pkg/receiver/svcmesh/istio/sidecar/sidecar.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@daemon1024 daemon1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nits! Rest LGTM 👌🏽

.github/workflows/ci-test-py.yml Show resolved Hide resolved
deployments/sentryflow.yaml Show resolved Hide resolved
Copy link
Collaborator

@daemon1024 daemon1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have added the Labels in WASM and Envoy Filter as well, Nice 👌🏽

LGTM 🙌🏽

Copy link
Member

@PrimalPimmy PrimalPimmy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except for a comment.

sentryflow/config/default.yaml Show resolved Hide resolved
@daemon1024 daemon1024 merged commit 889c6ba into 5GSEC:main Oct 4, 2024
3 checks passed
@anurag-rajawat anurag-rajawat deleted the refactor-sentryflow branch October 4, 2024 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

4 participants