Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate go modules for bridged providers #1238

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Conversation

blampe
Copy link
Contributor

@blampe blampe commented Dec 19, 2024

This introduces a migration to automatically consolidate ./{provider,examples,tests}/go.mod under a single ./go.mod. The SDK module isn't modified so users are NOT impacted.

Examples:

(Try it yourself by running go run github.com/pulumi/ci-mgmt/provider-ci@8953abeef73baacb2c592e96158afe9c29e94255 generate in a bridged provider.)

This gives us significantly more flexibility with regard to how we choose to organize our unit and integration tests. In particular it becomes straightforward to run integration tests under ./examples and ./providers simultaneously because they now belong to the same module. (We currently run one and then wait for it to complete before starting the other, but this sacrifices parallelism while waiting for long polls to complete.)

This is essentially an alternative to the integrationTestProvider option with a number of add benefits, including better performance and an improved overall developer experience.

#1039 (removing arbitrary injection hooks) will be further unblocked after this:

  • Pre-test hooks for things like credentials (example) can be handled with some logic in TestMain.
  • The "Run provider tests" step (example) can be removed because executing go test -tags=${{ matrix.language }} ... from the repository root will now capture integration tests under ./examples as well as ./provider.

This only benefits bridged providers, but we are very close to being able to put native providers on the same (or very similar) workflows as bridged providers -- at which point the same migration can be applied.

Related:

@blampe
Copy link
Contributor Author

blampe commented Dec 19, 2024

@ringods wanted to put this on your radar. This would move us towards a more idiomatic layout for these projects but I don't want it to be a surprise for pulumiverse.

Copy link
Member

@ringods ringods left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blampe may I ask for one addition to this PR?

Users working with VSCode to manage a provider (me included), add a go.work and go.work.sum file at the root of the repo currently. This is required for VSCode to have proper code completion in the different folders which currently have a go.mod file.

In this PR, you consolidate these go.mod files into a single one at the root. Can you either add go.work and go.work.sum to getDeletedFiles or make it part of your migration?

@blampe
Copy link
Contributor Author

blampe commented Dec 19, 2024

@blampe may I ask for one addition to this PR?

Users working with VSCode to manage a provider (me included), add a go.work and go.work.sum file at the root of the repo currently. This is required for VSCode to have proper code completion in the different folders which currently have a go.mod file.

In this PR, you consolidate these go.mod files into a single one at the root. Can you either add go.work and go.work.sum to getDeletedFiles or make it part of your migration?

@ringods done! If there is a go.work file it's now edited to reflect the new module structure. Technically we don't want to delete it, because it's possible for it to include other modules like pu/pu.

@blampe blampe requested a review from ringods December 19, 2024 18:27
blampe added a commit to pulumi/pulumi-kubernetes that referenced this pull request Dec 20, 2024
The `./tests` modules currently depends on the `./provider` module for
some OpenAPI helpers, but `./provider` also depends on `./tests` for
Gomega matchers.

Since `./tests` isn't versioned like a proper module this puts us in a
situation where Renovate will continuously try to update it
#3384

We can teach Renovate to ignore this bump, or we can break the
dependency by moving some code around. I've opted for the latter, moving
`tests/gomega` to `provider/pkg/gomega`. (This makes it easier to
consume from other projects if we want to.)

Of course it's worth mentioning this is a self-inflicted problem due to
our module structure. Related
pulumi/ci-mgmt#1238.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants