[Serve] Java Serve improvement #42
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
edoakes
reviewed
Sep 1, 2023
There was a problem hiding this comment.
This looks like a great step forward. A few high-level comments:
- As I indicated below, we are moving away from the "InputNode" and "DAGDriver" concepts (they're unnecessarily complicated). For Java, let's prioritize matching the
.bind()API for multiple deployments and using the handles that are substituted in the constructor. - The recommended way for users to deploy in production these days is via config file (e.g., embedded in a KubeRay CR). Can we include in the proposal adding support to deploy Java this way as well?
- [P1] As part of this plan, it would be great to have a path forward to unify the handle implementations (e.g., shared C++ core).
edoakes
reviewed
Sep 22, 2023
|
@liuyang-my looks great now, just need to add the config section! |
sihanwang41
reviewed
Sep 22, 2023
added 6 commits
September 28, 2023 19:30
Signed-off-by: chuhan.ly <[email protected]>
Signed-off-by: chuhan.ly <[email protected]>
Signed-off-by: chuhan.ly <[email protected]>
Signed-off-by: chuhan.ly <[email protected]>
…mentResponse demo. 3. Adjusting the way Python deploys deployment for other languages. Signed-off-by: chuhan.ly <[email protected]>
Signed-off-by: chuhan.ly <[email protected]>
liuyang-my
force-pushed
the
java-serve-improvement
branch
from
eb2a07a
to
c22d959
Compare
edoakes
reviewed
Sep 28, 2023
Signed-off-by: chuhan.ly <[email protected]>
edoakes
reviewed
Sep 29, 2023
edoakes
reviewed
Sep 29, 2023
| grpc_servicer_functions: [] | ||
|
|
||
| applications: | ||
| - name: app |
There was a problem hiding this comment.
Note that this can take args: parameter as well (will be passed in the same way that it is via command line)
There was a problem hiding this comment.
Great, the example of adding args in the config file has been supplemented.
edoakes
approved these changes
Sep 29, 2023
Signed-off-by: chuhan.ly <[email protected]>
8 tasks
edoakes
pushed a commit
to ray-project/ray
that referenced
this pull request
Oct 10, 2023
This is about Serve Java Improvement. The goal of this PR is to make the Java API consistent with Python. The design of the user API is discussed in this proposal: ray-project/enhancements#42. This PR only covers the Java programming API part, including Deployment.bind, Serve.run, DeploymentHandle, and so on, ensuring that these APIs can be used properly. The alignment of some internal core logic, documentation enhancements, and support for the config file will be submitted in subsequent PRs. Signed-off-by: chuhan.ly <[email protected]> Co-authored-by: chuhan.ly <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.