Skip to content

Commit

Permalink
kubeflow: fix go module and odh. debranding (#15)
Browse files Browse the repository at this point in the history
* kubeflow: change go module name and references

Signed-off-by: Andrea Lamparelli <[email protected]>

* kubeflow: rename odh. into kfmr.

Signed-off-by: tarilabs <[email protected]>

* kubeflow: py: pyproject description

Signed-off-by: tarilabs <[email protected]>

* kubeflow: nit picks in text documents

Signed-off-by: tarilabs <[email protected]>

---------

Signed-off-by: Andrea Lamparelli <[email protected]>
Signed-off-by: tarilabs <[email protected]>
Co-authored-by: tarilabs <[email protected]>
  • Loading branch information
lampajr and tarilabs authored Feb 22, 2024
1 parent 6c29dcd commit edefcc4
Show file tree
Hide file tree
Showing 41 changed files with 133 additions and 134 deletions.
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -37,19 +37,19 @@ api/grpc/ml_metadata/proto/metadata_source.proto:
mkdir -p api/grpc/ml_metadata/proto/
cd api/grpc/ml_metadata/proto/ && \
curl -LO "https://raw.githubusercontent.com/google/ml-metadata/v${MLMD_VERSION}/ml_metadata/proto/metadata_source.proto" && \
sed -i 's#syntax = "proto[23]";#&\noption go_package = "github.com/opendatahub-io/model-registry/internal/ml_metadata/proto";#' metadata_source.proto
sed -i 's#syntax = "proto[23]";#&\noption go_package = "github.com/kubeflow/model-registry/internal/ml_metadata/proto";#' metadata_source.proto

api/grpc/ml_metadata/proto/metadata_store.proto:
mkdir -p api/grpc/ml_metadata/proto/
cd api/grpc/ml_metadata/proto/ && \
curl -LO "https://raw.githubusercontent.com/google/ml-metadata/v${MLMD_VERSION}/ml_metadata/proto/metadata_store.proto" && \
sed -i 's#syntax = "proto[23]";#&\noption go_package = "github.com/opendatahub-io/model-registry/internal/ml_metadata/proto";#' metadata_store.proto
sed -i 's#syntax = "proto[23]";#&\noption go_package = "github.com/kubeflow/model-registry/internal/ml_metadata/proto";#' metadata_store.proto

api/grpc/ml_metadata/proto/metadata_store_service.proto:
mkdir -p api/grpc/ml_metadata/proto/
cd api/grpc/ml_metadata/proto/ && \
curl -LO "https://raw.githubusercontent.com/google/ml-metadata/v${MLMD_VERSION}/ml_metadata/proto/metadata_store_service.proto" && \
sed -i 's#syntax = "proto[23]";#&\noption go_package = "github.com/opendatahub-io/model-registry/internal/ml_metadata/proto";#' metadata_store_service.proto
sed -i 's#syntax = "proto[23]";#&\noption go_package = "github.com/kubeflow/model-registry/internal/ml_metadata/proto";#' metadata_store_service.proto

internal/ml_metadata/proto/%.pb.go: api/grpc/ml_metadata/proto/%.proto
protoc -I./api/grpc --go_out=./internal --go_opt=paths=source_relative \
Expand All @@ -59,7 +59,7 @@ internal/ml_metadata/proto/%.pb.go: api/grpc/ml_metadata/proto/%.proto
gen/grpc: internal/ml_metadata/proto/metadata_store.pb.go internal/ml_metadata/proto/metadata_store_service.pb.go

internal/converter/generated/converter.go: internal/converter/*.go
${GOVERTER} gen github.com/opendatahub-io/model-registry/internal/converter/
${GOVERTER} gen github.com/kubeflow/model-registry/internal/converter/

.PHONY: gen/converter
gen/converter: gen/grpc internal/converter/generated/converter.go
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Model registry provides a central repository for model developers to store and m
- Red Hat drives the project's development through Open Source principles, ensuring transparency, sustainability, and community ownership.
- Red Hat values the Kubeflow community and commits to providing a minimum of 12 months' notice before ending project maintenance after the initial release.

![build checks status](https://github.com/opendatahub-io/model-registry/actions/workflows/build.yml/badge.svg?branch=main)
[![codecov](https://codecov.io/github/opendatahub-io/model-registry/graph/badge.svg?token=61URLQA3VS)](https://codecov.io/github/opendatahub-io/model-registry)
![build checks status](https://github.com/kubeflow/model-registry/actions/workflows/build.yml/badge.svg?branch=main)
[![codecov](https://codecov.io/github/kubeflow/model-registry/graph/badge.svg?token=61URLQA3VS)](https://codecov.io/github/kubeflow/model-registry)

## Pre-requisites:
- go >= 1.19
Expand All @@ -19,14 +19,14 @@ Model registry provides a central repository for model developers to store and m

The model registry proxy server implementation follows a contract-first approach, where the contract is identified by [model-registry.yaml](api/openapi/model-registry.yaml) OpenAPI specification.

You can also easily display the latest OpenAPI contract for model-registry in a Swagger-like editor directly from this repository; for example, [here](https://editor.swagger.io/?url=https://raw.githubusercontent.com/opendatahub-io/model-registry/main/api/openapi/model-registry.yaml).
You can also easily display the latest OpenAPI contract for model-registry in a Swagger-like editor directly from this repository; for example, [here](https://editor.swagger.io/?url=https://raw.githubusercontent.com/kubeflow/model-registry/main/api/openapi/model-registry.yaml).
### Starting the OpenAPI Proxy Server
Run the following command to start the OpenAPI proxy server from source:

```shell
make run/proxy
```
The proxy service implements the OpenAPI defined in [model-registry.yaml](api/openapi/model-registry.yaml) to create an Open Data Hub specific REST API on top of the existing ml-metadata server.
The proxy service implements the OpenAPI defined in [model-registry.yaml](api/openapi/model-registry.yaml) to create a Model Registry specific REST API on top of the existing ml-metadata server.

> **NOTE** The ml-metadata server must be running and accessible from the environment where model-registry starts up.
Expand Down
2 changes: 1 addition & 1 deletion api/grpc/ml_metadata/proto/metadata_store.proto
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
syntax = "proto2";
option go_package = "github.com/opendatahub-io/model-registry/internal/ml_metadata/proto";
option go_package = "github.com/kubeflow/model-registry/internal/ml_metadata/proto";

package ml_metadata;

Expand Down
2 changes: 1 addition & 1 deletion api/grpc/ml_metadata/proto/metadata_store_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ limitations under the License.
==============================================================================*/

syntax = "proto2";
option go_package = "github.com/opendatahub-io/model-registry/internal/ml_metadata/proto";
option go_package = "github.com/kubeflow/model-registry/internal/ml_metadata/proto";

package ml_metadata;

Expand Down
4 changes: 2 additions & 2 deletions clients/python/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Model Registry Python Client

[![Python](https://img.shields.io/badge/python%20-3.9%7C3.10-blue)](https://github.com/opendatahub-io/model-registry)
[![Python](https://img.shields.io/badge/python%20-3.9%7C3.10-blue)](https://github.com/kubeflow/model-registry)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](../../../LICENSE)

This library provides a high level interface for interacting with a model registry server.
Expand Down Expand Up @@ -85,7 +85,7 @@ Use `nox -l` to list sessions and execute them using `nox -s [session]`.

### Running Locally on Mac M1 or M2 (arm64 architecture)

If you want run tests locally you will need to set up a colima develeopment environment using the instructions [here](https://github.com/opendatahub-io/model-registry/blob/main/CONTRIBUTING.md#colima)
If you want run tests locally you will need to set up a colima develeopment environment using the instructions [here](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#colima)

You will also have to change the package source to one compatible with ARM64 architecture. This can be actioned by uncommenting lines 14 or 15 in the pyproject.toml file. Run the following command after you have uncommented the line.

Expand Down
6 changes: 3 additions & 3 deletions clients/python/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
[tool.poetry]
name = "model-registry"
version = "0.1.0"
description = "Client for Red Hat OpenDataHub Model Registry"
description = "Client for Kubeflow Model Registry"
authors = ["Isabella Basso do Amaral <[email protected]>"]
license = "Apache-2.0"
readme = "README.md"
homepage = "https://github.com/opendatahub-io/model-registry"
homepage = "https://github.com/kubeflow/model-registry"

[tool.poetry.urls]
"Issues" = "https://github.com/opendatahub-io/model-registry/issues"
"Issues" = "https://github.com/kubeflow/model-registry/issues"

[tool.poetry.dependencies]
python = ">= 3.9, < 3.11"
Expand Down
4 changes: 2 additions & 2 deletions clients/python/src/model_registry/types/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ def get_proto_type_name(cls) -> str:
"""Name of the proto type.
Returns:
Name of the class prefixed with `odh.`
Name of the class prefixed with `kfmr.`
"""
return f"odh.{cls.__name__}"
return f"kfmr.{cls.__name__}"

@property
@abstractmethod
Expand Down
6 changes: 3 additions & 3 deletions cmd/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ import (
"time"

"github.com/golang/glog"
"github.com/opendatahub-io/model-registry/internal/mlmdtypes"
"github.com/opendatahub-io/model-registry/internal/server/openapi"
"github.com/opendatahub-io/model-registry/pkg/core"
"github.com/kubeflow/model-registry/internal/mlmdtypes"
"github.com/kubeflow/model-registry/internal/server/openapi"
"github.com/kubeflow/model-registry/pkg/core"
"github.com/spf13/cobra"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
Expand Down
4 changes: 2 additions & 2 deletions docs/mr_go_library.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The Model Registry Service go library provides a convenient interface for managi

The recommended way is using `go get`, from your custom project run:
```bash
go get github.com/opendatahub-io/model-registry
go get github.com/kubeflow/model-registry
```

## Getting Started
Expand Down Expand Up @@ -50,7 +50,7 @@ Once the gRPC connection is setup, let's create the `ModelRegistryService`:
```go
import (
"fmt"
"github.com/opendatahub-io/model-registry/pkg/core"
"github.com/kubeflow/model-registry/pkg/core"
)

service, err := core.NewModelRegistryService(conn)
Expand Down
10 changes: 5 additions & 5 deletions docs/remote_only_packaging_of_MLMD_Python_lib.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ whilst also making it [architecture specific](https://pypi.org/project/ml-metada

The [Model Registry project](https://docs.google.com/document/d/1G-pjdGaS2kLELsB5kYk_D4AmH-fTfnCnJOhJ8xENjx0/edit?usp=sharing) (MR) is built on top of MLMD.
The Go implementation interfaces with the MLMD server via gRPC, typically available as a Docker container.
The [MR Python client](https://github.com/opendatahub-io/model-registry/tree/main/clients/python#readme) wraps the MLMD client.
The [MR Python client](https://github.com/kubeflow/model-registry/tree/main/clients/python#readme) wraps the MLMD client.

As the MLMD client is architecture specific, so is the MR Python client, which **severely limits the targets it can run on**, as it only supports x86-64.
This **poses many challenges to contributors** using other CPU architectures, specially ARM, as that's become more prevalent in recent years.
Expand Down Expand Up @@ -97,7 +97,7 @@ Cons:

### Packaging Option2: mix resulting artifact inside Model Registry repo

This delivery option considers placing the resulting artifact by executing the exercise from the alternative selected above and placing it directly inside the Model Registry repo, with the python client source [location](https://github.com/opendatahub-io/model-registry/tree/main/clients/python). (for analogy, this is similar to “shading”/”uberjar” in Java world for those familiar with the concept)
This delivery option considers placing the resulting artifact by executing the exercise from the alternative selected above and placing it directly inside the Model Registry repo, with the python client source [location](https://github.com/kubeflow/model-registry/tree/main/clients/python). (for analogy, this is similar to “shading”/”uberjar” in Java world for those familiar with the concept)

Pros:

Expand All @@ -117,10 +117,10 @@ Based on analysis of the alternatives provided, we decided to further pursue:
- the repackaging by **Alternative B** because makes it actually easier to demonstrate the steps and modifications required using as baseline the upstream repo.
- the distribution by **Packaging Option1** because it will make it easier to "revert" to the upstream `ml-metadata` dependency if upstream will publish for all architectures, OSes, etc. and as the pros outweight considered cons.

MR python client [tests](https://github.com/opendatahub-io/model-registry/blob/259b39320953bf05942dcec1fb5ec74f7eb5d4a7/clients/python/tests/conftest.py#L19) should be rewritten using Testcontainers, and not leveraging the embedded server (already done with [this PR](https://github.com/opendatahub-io/model-registry/pull/225)).
MR python client [tests](https://github.com/kubeflow/model-registry/blob/259b39320953bf05942dcec1fb5ec74f7eb5d4a7/clients/python/tests/conftest.py#L19) should be rewritten using Testcontainers, and not leveraging the embedded server (already done with [this PR](https://github.com/kubeflow/model-registry/pull/225)).

The group concur this is a sensible approach ([recorded here](https://redhat-internal.slack.com/archives/C05LGBNUK9C/p1700763823505259?thread_ts=1700427888.670999&cid=C05LGBNUK9C)).
The group concur this is a sensible approach.

This change would also better align the test approach used for the MR python client, by aligning with the same strategy of the MR core Go layer test framework, which already makes use of Testcontainers for Go ([reference](https://github.com/opendatahub-io/model-registry/blob/259b39320953bf05942dcec1fb5ec74f7eb5d4a7/internal/testutils/test_container_utils.go#L59)).
This change would also better align the test approach used for the MR python client, by aligning with the same strategy of the MR core Go layer test framework, which already makes use of Testcontainers for Go ([reference](https://github.com/kubeflow/model-registry/blob/259b39320953bf05942dcec1fb5ec74f7eb5d4a7/internal/testutils/test_container_utils.go#L59)).

This would allow to update the constraint on the version for the `attrs` dependency as part of this activity.
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module github.com/opendatahub-io/model-registry
module github.com/kubeflow/model-registry

go 1.19

Expand Down
8 changes: 4 additions & 4 deletions internal/apiutils/api_utils.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
package apiutils

import (
"github.com/opendatahub-io/model-registry/internal/converter"
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/api"
model "github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/converter"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/api"
model "github.com/kubeflow/model-registry/pkg/openapi"
)

func BuildListOperationOptions(listOptions api.ListOptions) (*proto.ListOperationOptions, error) {
Expand Down
14 changes: 7 additions & 7 deletions internal/constants/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ package constants

// MLMD type names
const (
RegisteredModelTypeName = "odh.RegisteredModel"
ModelVersionTypeName = "odh.ModelVersion"
ModelArtifactTypeName = "odh.ModelArtifact"
DocArtifactTypeName = "odh.DocArtifact"
ServingEnvironmentTypeName = "odh.ServingEnvironment"
InferenceServiceTypeName = "odh.InferenceService"
ServeModelTypeName = "odh.ServeModel"
RegisteredModelTypeName = "kfmr.RegisteredModel"
ModelVersionTypeName = "kfmr.ModelVersion"
ModelArtifactTypeName = "kfmr.ModelArtifact"
DocArtifactTypeName = "kfmr.DocArtifact"
ServingEnvironmentTypeName = "kfmr.ServingEnvironment"
InferenceServiceTypeName = "kfmr.InferenceService"
ServeModelTypeName = "kfmr.ServeModel"
)
6 changes: 3 additions & 3 deletions internal/converter/generated/mlmd_openapi_converter.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions internal/converter/generated/openapi_converter.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions internal/converter/generated/openapi_mlmd_converter.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions internal/converter/mlmd_converter_util_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import (
"strings"
"testing"

"github.com/opendatahub-io/model-registry/internal/constants"
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/constants"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/openapi"
"github.com/stretchr/testify/assert"
"golang.org/x/exp/maps"
)
Expand Down
4 changes: 2 additions & 2 deletions internal/converter/mlmd_openapi_converter.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package converter

import (
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/openapi"
)

// goverter:converter
Expand Down
6 changes: 3 additions & 3 deletions internal/converter/mlmd_openapi_converter_util.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import (
"fmt"
"strings"

"github.com/opendatahub-io/model-registry/internal/constants"
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/constants"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/openapi"
)

// MapMLMDCustomProperties maps MLMD custom properties model to OpenAPI one
Expand Down
2 changes: 1 addition & 1 deletion internal/converter/openapi_converter.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package converter

import "github.com/opendatahub-io/model-registry/pkg/openapi"
import "github.com/kubeflow/model-registry/pkg/openapi"

// NOTE: methods must follow these patterns, otherwise tests could not find possible issues:
// Converters createEntity to entity: Convert<ENTITY>Create
Expand Down
2 changes: 1 addition & 1 deletion internal/converter/openapi_converter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
"strings"
"testing"

"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/pkg/openapi"
)

// visitor
Expand Down
2 changes: 1 addition & 1 deletion internal/converter/openapi_converter_util.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package converter

import "github.com/opendatahub-io/model-registry/pkg/openapi"
import "github.com/kubeflow/model-registry/pkg/openapi"

type OpenAPIModel interface {
openapi.RegisteredModel |
Expand Down
4 changes: 2 additions & 2 deletions internal/converter/openapi_mlmd_converter.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package converter

import (
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/openapi"
)

type OpenAPIModelWrapper[
Expand Down
6 changes: 3 additions & 3 deletions internal/converter/openapi_mlmd_converter_util.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ import (
"strconv"

"github.com/google/uuid"
"github.com/opendatahub-io/model-registry/internal/constants"
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/constants"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/openapi"
"google.golang.org/protobuf/types/known/structpb"
)

Expand Down
10 changes: 5 additions & 5 deletions internal/mapper/mapper.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ package mapper
import (
"fmt"

"github.com/opendatahub-io/model-registry/internal/constants"
"github.com/opendatahub-io/model-registry/internal/converter"
"github.com/opendatahub-io/model-registry/internal/converter/generated"
"github.com/opendatahub-io/model-registry/internal/ml_metadata/proto"
"github.com/opendatahub-io/model-registry/pkg/openapi"
"github.com/kubeflow/model-registry/internal/constants"
"github.com/kubeflow/model-registry/internal/converter"
"github.com/kubeflow/model-registry/internal/converter/generated"
"github.com/kubeflow/model-registry/internal/ml_metadata/proto"
"github.com/kubeflow/model-registry/pkg/openapi"
)

type Mapper struct {
Expand Down
Loading

0 comments on commit edefcc4

Please sign in to comment.