Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[COST-5773] Handle multiple resource for OCP on GCP managed flow #5427

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

myersCody
Copy link
Contributor

@myersCody myersCody commented Dec 16, 2024

Jira Ticket

COST-5573

Description

This change will:

  1. Add multiple clusters, add a wait, and trigger ocp ingest before cloud ingest to prevent ocp on cloud parquet file error.
  2. Restructure the managed flow to use a dataclass to pass in build the sql parameters in the db accessors.
  3. Restructure the GCP flow to handle row deduplication
  • Create two temporary tables
  • Table One: Grab all the possible rows from the GCP table and assign a unique identifier to them
  • Table Two: Read from table one to insert all of the OCP resource & tag matched rows
  • When we insert into the final managed table make we make sure the uuid only shows up once preventing duplication

Testing

  • Set the unleash flag to True for ocp on cloud managed processing
  1. using this branch Load test customer data for:
  • AWS
  • Azure
  • GCP
  1. The test data populated for OCP on GCP will end with a verification successful return. Whereas if you were to try on master you will get a failed message.

Release Notes

  • proposed release note
* [COST-####](https://issues.redhat.com/browse/COST-####) Fix some things

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant