Skip to content

Commit

Permalink
Add Dockerfile for registry, part of kubeflow#15, fixes kubeflow#40
Browse files Browse the repository at this point in the history
  • Loading branch information
dhirajsb committed Oct 2, 2023
1 parent e1e7f08 commit d8277f5
Show file tree
Hide file tree
Showing 3 changed files with 76 additions and 5 deletions.
46 changes: 45 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1 +1,45 @@
FROM
# Build the model-registry binary
FROM registry.access.redhat.com/ubi8/go-toolset:1.19 as builder

WORKDIR /workspace
# Copy the Go Modules manifests
COPY ["go.mod", "go.sum", "./"]
# cache deps before building and copying source so that we don't need to re-download as much
# and so that source changes don't invalidate our downloaded layer
RUN go mod download

# Copy the go source
USER root
COPY ["Makefile", "main.go", "gqlgen.yml", "./"]

# Download protoc compiler v24.3
RUN wget -q https://github.com/protocolbuffers/protobuf/releases/download/v24.3/protoc-24.3-linux-x86_64.zip -O protoc.zip && \
unzip -q protoc.zip && \
bin/protoc --version && \
rm protoc.zip

# Download tools
RUN make deps

# Copy rest of the source
COPY bin/ bin/
COPY cmd/ cmd/
COPY api/ api/
COPY config/ config/
COPY internal/ internal/

# Build
USER root
RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 make clean model-registry

# Use distroless as minimal base image to package the model-registry binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.6
WORKDIR /
# copy the metadata library
COPY --from=builder /workspace/config ./config
# copy the registry binary
COPY --from=builder /workspace/model-registry .
USER 65532:65532

ENTRYPOINT ["/model-registry"]
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ gen/grpc: internal/ml_metadata/proto/metadata_store.pb.go internal/ml_metadata/p
gen/graph: internal/model/graph/models_gen.go

internal/model/graph/models_gen.go: api/graphql/*.graphqls gqlgen.yml
go run github.com/99designs/gqlgen generate
gqlgen generate

.PHONY: vet
vet:
Expand Down
33 changes: 30 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ The generated binary uses spf13 cmdline args. More information on using the serv
```
./model-registry --help
```
## Creating/Migrating Server DB
## Server
### Creating/Migrating Server DB
The server uses a local SQLite DB file (`metadata.sqlite.db` by default), which can be configured using the `-d` cmdline option.
The following command creates the DB:
```
Expand All @@ -29,11 +30,12 @@ Note that currently no duplicate detection is done as the implementation is a WI
Running this command multiple times will create duplicate metadata types.
To clear the DB simply delete the SQLite DB file `metadata.sqlite.db`.

### Running Server
### Starting the Server
Run the following command to start the server:
```
make run/server &
```
## Clients
### Running Python ml-metadata test client
Before running the test client, install the required Python libraries (using a python venv, if using one)
using the command:
Expand All @@ -52,7 +54,32 @@ http://localhost:8080/
```
Where, 8080 is the default port that the server listens on. This port can be changed with the `-p` option.
### Clean
Run the following command to clean the DB file, generated gRPC and GraphQL models, etc.:
Run the following command to clean the server binary, generated gRPC and GraphQL models, etc.:
```
make clean
```
## Docker Image
### Building the Docker Image
The following command builds a docker image for the server with the tag `model-registry``:
```shell
docker build -t model-registry .
```
Note that the first build will be longer as it downloads the build tool dependencies.
Subsequent builds will re-use the cached tools layer.
### Creating/Migrating Server DB
The following command migrates or creates a DB for the server:
```shell
docker run -it --user <uid>:<gid> -v <host-path>:/var/db model-registry migrate -d /var/db/metadata.sqlite.db -m /config/metadata-library
```
Where, `<uid>` and `<gid>` are local user and group ids on the host machine to allow volume mapping for the DB files.
And, `<host-path>` is the path on the local directory writable by the `<uid>:<gid>` user.
### Running the Server
The following command starts the server:
```shell
docker run -d -p <hostname>:<port>:8080 --user <uid>:<gid> -v <host-path>:/var/db --name server model-registry serve -n 0.0.0.0 -d /var/db/metadata.sqlite.db
```
Where, `<uid>`, `<gid>`, and `<host-path>` are the same as in the migrate command above.
And `<hostname>` and `<port>` are the local ip and port to use to expose the container's default `8080` listening port.
The server listens on `localhost` by default, hence the `-n 0.0.0.0` option allows the server port to be exposed.

Once the server has started, test clients and playground can be used as described in the above sections.

0 comments on commit d8277f5

Please sign in to comment.