Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: pipeline detail show create schama status #918

Merged
merged 3 commits into from
Mar 4, 2024

Conversation

tyyzqmf
Copy link
Contributor

@tyyzqmf tyyzqmf commented Mar 3, 2024


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Summary

(describe what this merge request does)

Implementation highlights

(describe how the merge request does for feature changes, share the RFC link if it has)

Test checklist

  • add new test cases
  • all code changes are covered by unit tests
  • end-to-end tests
    • deploy web console with CloudFront + S3 + API gateway
    • deploy web console within VPC
    • deploy ingestion server
      • with MSK sink
      • with KDS sink
      • with S3 sink
    • deploy data processing
    • deploy data modeling
      • new Redshift Serverless
      • provisioned Redshift
      • Athena
    • deploy with reporting
    • streaming ingestion
      • with Redshift Serverless
      • with provisioned Redshift

Is it a breaking change

  • add parameters without default value in stack
  • introduce new service permission in stack
  • introduce new top level stack module

Miscellaneous

  • introduce new symbol link source file(s) to be shared among infra code, web console frontend, and web console backend

@tyyzqmf tyyzqmf requested a review from zxkane March 3, 2024 03:54
@tyyzqmf tyyzqmf self-assigned this Mar 3, 2024
@tyyzqmf tyyzqmf force-pushed the chore-fetch-schema-status branch from 5e15f49 to 14c2db0 Compare March 3, 2024 03:56
Copy link

github-actions bot commented Mar 3, 2024

Streaming ingestion flink application's test coverage

Overall Project 94.57% 🍏

There is no coverage information present for the Files changed

1 similar comment
Copy link

github-actions bot commented Mar 3, 2024

Streaming ingestion flink application's test coverage

Overall Project 94.57% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 3, 2024

Data Pipeline spark application's test coverage

Overall Project 96.53% 🍏

There is no coverage information present for the Files changed

1 similar comment
Copy link

github-actions bot commented Mar 3, 2024

Data Pipeline spark application's test coverage

Overall Project 96.53% 🍏

There is no coverage information present for the Files changed

@tyyzqmf tyyzqmf force-pushed the chore-fetch-schema-status branch from 14c2db0 to 8c36a40 Compare March 3, 2024 05:23
Copy link

github-actions bot commented Mar 3, 2024

Streaming ingestion flink application's test coverage

Overall Project 94.57% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 3, 2024

Data Pipeline spark application's test coverage

Overall Project 96.53% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 4, 2024

Streaming ingestion flink application's test coverage

Overall Project 94.57% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 4, 2024

Data Pipeline spark application's test coverage

Overall Project 96.53% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 4, 2024

Streaming ingestion flink application's test coverage

Overall Project 94.57% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 4, 2024

Data Pipeline spark application's test coverage

Overall Project 96.53% 🍏

There is no coverage information present for the Files changed

Copy link

github-actions bot commented Mar 4, 2024

Code Coverage

Package Line Rate Branch Rate Health
src 100% 83%
src.analytics 100% 100%
src.analytics.lambdas 95% 81%
src.analytics.lambdas.clear-expired-events-workflow 100% 100%
src.analytics.lambdas.custom-resource 96% 87%
src.analytics.lambdas.load-data-workflow 99% 90%
src.analytics.lambdas.scan-metadata-workflow 98% 90%
src.analytics.lambdas.sql-execution-sfn 93% 60%
src.analytics.private 100% 92%
src.analytics.private.athena 100% 100%
src.base-lib.src 100% 100%
src.base-lib.src.constant 100% 100%
src.common 98% 76%
src.common.lambda 97% 86%
src.control-plane 97% 72%
src.control-plane.auth 90% 48%
src.control-plane.backend 100% 88%
src.control-plane.backend.lambda.api 97% 33%
src.control-plane.backend.lambda.api.common 93% 82%
src.control-plane.backend.lambda.api.middle-ware 89% 83%
src.control-plane.backend.lambda.api.model 95% 59%
src.control-plane.backend.lambda.api.router 99% 100%
src.control-plane.backend.lambda.api.service 85% 71%
src.control-plane.backend.lambda.api.service.quicksight 90% 85%
src.control-plane.backend.lambda.api.store.aws 86% 53%
src.control-plane.backend.lambda.api.store.dynamodb 90% 79%
src.control-plane.backend.lambda.api.test.api 98% 80%
src.control-plane.backend.lambda.batch-insert-ddb 94% 83%
src.control-plane.backend.lambda.listen-stack-status 83% 50%
src.control-plane.backend.lambda.listen-state-status 89% 40%
src.control-plane.backend.lambda.sfn-action 84% 47%
src.control-plane.backend.lambda.sfn-workflow 69% 50%
src.control-plane.backend.layer.lambda-web-adapter 100% 20%
src.control-plane.private 100% 100%
src.data-pipeline 100% 100%
src.data-pipeline.lambda.copy-assets 97% 90%
src.data-pipeline.lambda.emr-job-state-listener 96% 90%
src.data-pipeline.lambda.emr-job-submitter 94% 78%
src.data-pipeline.lambda.emr-serverless-app 100% 90%
src.data-pipeline.lambda.partition-syncer 98% 89%
src.data-pipeline.tables 100% 100%
src.data-pipeline.utils 100% 97%
src.ingestion-server.custom-resource 100% 100%
src.ingestion-server.custom-resource.delete-ecs-cluster 98% 90%
src.ingestion-server.custom-resource.update-alb-rules 100% 78%
src.ingestion-server.kafka-s3-connector 100% 100%
src.ingestion-server.kafka-s3-connector.custom-resource.kafka-s3-sink-connector 94% 81%
src.ingestion-server.kinesis-data-stream 100% 100%
src.ingestion-server.kinesis-data-stream.kinesis-to-s3-lambda 100% 100%
src.ingestion-server.kinesis-data-stream.private 100% 100%
src.ingestion-server.server 98% 89%
src.ingestion-server.server-v2 100% 79%
src.ingestion-server.server-v2.private 100% 82%
src.ingestion-server.server.private 100% 89%
src.metrics 100% 100%
src.metrics.custom-resource.add-sns-subscription 95% 86%
src.metrics.custom-resource.get-interval 99% 96%
src.metrics.custom-resource.put-dashboard 97% 84%
src.metrics.custom-resource.set-metrics-widgets 98% 93%
src.private 97% 67%
src.reporting 100% 71%
src.reporting.lambda.custom-resource.quicksight 95% 82%
src.reporting.private 91% 51%
src.streaming-ingestion 100% 100%
src.streaming-ingestion.lambdas.custom-resource 86% 63%
src.streaming-ingestion.private 99% 89%
src.streaming-ingestion.redshift 100% 100%
test 96% 83%
test.common 93% 100%
test.control-plane 97% 64%
test.ingestion-server.server 97% 88%
Summary 95% (59814 / 62844) 76% (5590 / 7331)

Minimum allowed line rate is 60%

@zxkane zxkane enabled auto-merge March 4, 2024 06:16
@zxkane zxkane added this pull request to the merge queue Mar 4, 2024
Merged via the queue into main with commit b22fa31 Mar 4, 2024
16 checks passed
@zxkane zxkane deleted the chore-fetch-schema-status branch March 4, 2024 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants