Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OAS: Fix discriminator field definition for Artifact #22

Merged
merged 2 commits into from
Mar 1, 2024

Conversation

isinyaaa
Copy link
Contributor

@isinyaaa isinyaaa commented Feb 26, 2024

This fixes an improper polymorphic type definition by moving its discriminator to the composing types.

There's also a small fix for ignoring coverage.html, that is generated when running make test-cover.

Description

This problem was spotted while using the Kiota generator, and can be traced back to the following pattern,
which is incompatible with JSONSchema.

How Has This Been Tested?

Following the same steps laid out on the last PR referencing OAS #17:

make FORCE_SERVER_GENERATION=true clean build
make test

then,

  1. start an MLMD server
  2. run make proxy
  3. run robot test/robot/ && TEST_MODE=python robot test/robot/MRandLogicalModel.robot

Merge criteria:

  • The commits and have meaningful messages; the author will squash them after approval or will ask to merge with squash.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

Signed-off-by: Isabella Basso do Amaral <[email protected]>
@dhirajsb
Copy link
Contributor

@isinyaaa I noticed this PR is moving the discriminator to derived types. How does that help, or what is the difference?

Copy link
Member

@lampajr lampajr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @isinyaaa , thanks a lot for this PR.

Overall the change looks good to me, I added a comment whether we should also add the artifactProperty i nthe default obj (outside the allOf) which seems making the WARN disappear on openapi generator side (but I don't know on Kiota side)

internal/server/openapi/type_asserts.go Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still not sure if that is really enough or not because even with this change (which makes totally sense to me) I see:

[main] WARN  o.o.codegen.DefaultCodegen - 'Artifact' defines discriminator 'artifactType', but the referenced schema 'ModelArtifact' is incorrect. artifactType is missing from the schema, define it as required and type string
[main] WARN  o.o.codegen.DefaultCodegen - 'Artifact' defines discriminator 'artifactType', but the referenced schema 'DocArtifact' is incorrect. artifactType is missing from the schema, define it as required and type string

from the openapi generator logs, which seems to be fixed when adding artifactType as property before the allOf keyword:

    ModelArtifact:
      description: An ML model artifact.
      type: object
      required:
        - artifactType
      properties:
        artifactType:
          type: string
          default: "model-artifact"
      allOf:
        - $ref: "#/components/schemas/BaseArtifact"
        - $ref: "#/components/schemas/ModelArtifactCreate"
        - type: object
          properties:
            artifactType:
              type: string
              default: "model-artifact"
...
    DocArtifact:
      description: A document.
      type: object
      required:
        - artifactType
      properties:
        artifactType:
          type: string
          default: "doc-artifact"
      allOf:
        - $ref: "#/components/schemas/BaseArtifact"
        - type: object
          properties:
            artifactType:
              type: string
              default: "doc-artifact"

TBH I am not that sure that having artifactType defined in multiple places is correct but just thinking loud here.

Copy link
Member

@lampajr lampajr Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some discussion with Peruffo we agreed that the following specs are equivalent each other, but keeping the artifactType to the leaf obj should make the WARN disappear on server side as well.

The spec defined in this PR where we still have the WARN on server side model generation (actually the spec is fine, we just have the WARN)

    ModelArtifact:
      description: An ML model artifact.
      type: object
      required:
        - artifactType
      allOf:
        - $ref: "#/components/schemas/BaseArtifact"
        - $ref: "#/components/schemas/ModelArtifactCreate"
        - type: object
          properties:
            artifactType:
              type: string
              default: "model-artifact"

And the proposed one (here the actual final result is the same but the warning disappears)

    ModelArtifact:
      description: An ML model artifact.
      type: object
      required:
        - artifactType
      properties:
        artifactType:
          type: string
          default: "model-artifact"
      allOf:
        - $ref: "#/components/schemas/BaseArtifact"
        - $ref: "#/components/schemas/ModelArtifactCreate"

To make the change I am proposing clearer I created a PR against this PR: isinyaaa#1

So I think it is up to us how to proceed here, if the WARN on server side is acceptable I think we can move ahead with this PR as it is.. otherwise I think the proposed change should solve the issue without changing the final code generation.

@isinyaaa can you check if this change works well with Kiota as well?
[edit: I tried using Kiota and there is no error like Schema Artifact must contain property specified in the discriminator artifactType in the required field list so I think that even with this change the issue should be fixed on kiota side]

wdyt @isinyaaa @tarilabs @dhirajsb ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for following up on this OpenAPI topic, if the "proposing clearer" PR is acceptable sounds to me the best setup consolidating contributions from all

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's interesting, thanks for the investigation! updated now.

Signed-off-by: Isabella Basso do Amaral <[email protected]>
@isinyaaa
Copy link
Contributor Author

isinyaaa commented Feb 27, 2024

@isinyaaa I noticed this PR is moving the discriminator to derived types. How does that help, or what is the difference?

IIUC, having the discriminator field defined on Artifact doesn't imply that it's present on the composing types, as there's no such thing as "required properties" that have been inherited. Thus, artifactType should be required on all the composing classes.

Just updated the broken link on the description which I think also highlights the point.

Copy link
Member

@lampajr lampajr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @isinyaaa , lgtm!

@tarilabs
Copy link
Member

tarilabs commented Mar 1, 2024

/lgtm

Thank you so much, this is great complementary work to make our OpenAPI contract more robust to be consumed by clients

Copy link
Member

@tarilabs tarilabs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: isinyaaa, lampajr, tarilabs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit be3b275 into kubeflow:main Mar 1, 2024
11 checks passed
openshift-merge-bot bot referenced this pull request in opendatahub-io/model-registry Mar 4, 2024
* kubeflow: fix go module and odh. debranding (#15)

* kubeflow: change go module name and references

Signed-off-by: Andrea Lamparelli <[email protected]>

* kubeflow: rename odh. into kfmr.

Signed-off-by: tarilabs <[email protected]>

* kubeflow: py: pyproject description

Signed-off-by: tarilabs <[email protected]>

* kubeflow: nit picks in text documents

Signed-off-by: tarilabs <[email protected]>

---------

Signed-off-by: Andrea Lamparelli <[email protected]>
Signed-off-by: tarilabs <[email protected]>
Co-authored-by: tarilabs <[email protected]>

* Fix metadata OpenAPI spec (#17)

* fix: OpenAPI metadata discriminator

Signed-off-by: tarilabs <[email protected]>

* wiring factories and default values missed in codegen

Signed-off-by: tarilabs <[email protected]>

* introduce openapi defaults

Signed-off-by: tarilabs <[email protected]>

* fix TestMetadataValue*

Signed-off-by: Andrea Lamparelli <[email protected]>

* fix: type assert generation

Signed-off-by: Andrea Lamparelli <[email protected]>

* upgrade openapi spec version to v1alpha2

Signed-off-by: Andrea Lamparelli <[email protected]>

---------

Signed-off-by: tarilabs <[email protected]>
Signed-off-by: Andrea Lamparelli <[email protected]>
Co-authored-by: tarilabs <[email protected]>

* kubeflow: make MLMD type names (and prefix) pluggable (#19)

Signed-off-by: Matteo Mortari <[email protected]>

* build(deps): bump google.golang.org/grpc from 1.61.1 to 1.62.0 (#20)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.61.1 to 1.62.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.61.1...v1.62.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* OAS: Fix discriminator field definition for Artifact (#22)

* gitignore: ignore all coverage files

Signed-off-by: Isabella Basso do Amaral <[email protected]>

* OAS: fix discriminator field for Artifact

Signed-off-by: Isabella Basso do Amaral <[email protected]>

---------

Signed-off-by: Isabella Basso do Amaral <[email protected]>

---------

Signed-off-by: Andrea Lamparelli <[email protected]>
Signed-off-by: tarilabs <[email protected]>
Signed-off-by: Matteo Mortari <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Isabella Basso do Amaral <[email protected]>
Co-authored-by: Andrea Lamparelli <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Isabella Basso <[email protected]>
@isinyaaa isinyaaa deleted the fix-art-oas branch March 5, 2024 15:54
tarilabs pushed a commit to tarilabs/model-registry that referenced this pull request Mar 18, 2024
[pull] main from kubeflow:main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants