Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make packages baked into images by Amigo visible to the service catalogue #1333

Merged
merged 6 commits into from
Dec 4, 2024

Conversation

kelvin-chappell
Copy link
Member

@kelvin-chappell kelvin-chappell commented Nov 22, 2024

Purpose

This PR makes a new table available to the service catalogue, amigo_bake_packages. This table will tell us at a glance what packages are installed on which Amigo bakes, and in conjunction with other tables in the catalogue will tell us what packages are installed on which EC2 instances and in which AWS accounts.

Implementation

We add a new cloudformed task definition to ECS to run a CloudQuery job. The CQ job fetches data from the Amigo Dynamo DB tables and Amigo packagelist S3 objects and stores them in the new service catalogue table on a daily schedule.

The CQ job makes use of our new custom image package source plugin.

Testing

Tested on Code environment where the new task has run successfully and created the new table populated correctly with Code Amigo data.

@kelvin-chappell kelvin-chappell force-pushed the kc/package-table branch 3 times, most recently from 16eaedf to 22cb7f1 Compare December 2, 2024 11:59
@kelvin-chappell kelvin-chappell changed the title Add CDK config for image package source Make packages baked into images by Amigo visible to the service catalogue Dec 2, 2024
@kelvin-chappell kelvin-chappell marked this pull request as ready for review December 2, 2024 14:15
@kelvin-chappell kelvin-chappell requested review from a team as code owners December 2, 2024 14:15
@tjsilver
Copy link
Contributor

tjsilver commented Dec 2, 2024

Should we also add the amigo_bake_packages to the Prisma schema (with @@ignored) for now? See #1338.

@kelvin-chappell
Copy link
Member Author

Should we also add the amigo_bake_packages to the Prisma schema (with @@ignored) for now? See #1338.

👍 Done in bfa14f7

Comment on lines +597 to +604
const packagesBucketName = new GuStringParameter(
scope,
'packagesBucketNameParam',
{
fromSSM: true,
default: `/${stage}/deploy/amigo/amigo.data.bucket`,
},
).valueAsString;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who creates/owns this SSM Parameter? Could it be created by AMIgo, as it creates the bucket?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's created by Amigo as it pre-exists this integration.

Copy link
Contributor

@tjsilver tjsilver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@kelvin-chappell kelvin-chappell merged commit 4603ab8 into main Dec 4, 2024
7 checks passed
@kelvin-chappell kelvin-chappell deleted the kc/package-table branch December 4, 2024 09:13
@kelvin-chappell
Copy link
Member Author

Ran the import process manually and it appears to have worked. Prod Table has data and no errors in log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants