Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ai proxy #1506

Conversation

floreks
Copy link
Member

@floreks floreks commented Oct 24, 2024

  • Added go/ai-proxy go module
  • Added ghcr.io/pluralsh/ai-proxy container
  • Added basic chart to deploy it

It does on-the-fly request/response modification to match the configured --provider API. The response format will always match the ollama API format. Provider token when needed can be provided via PLRL_PROVIDER_TOKEN env var or --provider-token flag.

Test Plan

  • Locally
  • Test on dev

Checklist

  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have added a meaningful title and summary to convey the impact of this PR to a user.

Copy link

linear bot commented Oct 24, 2024

@floreks floreks marked this pull request as draft October 24, 2024 15:16
@floreks floreks self-assigned this Oct 24, 2024
@floreks floreks added the enhancement New feature or request label Oct 24, 2024
Copy link
Member

@michaeljguarino michaeljguarino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should write at least one basic mock test against the endpoint to ensure regressions are covered and make sure there's a gh action to run it

@floreks
Copy link
Member Author

floreks commented Oct 24, 2024

we should write at least one basic mock test against the endpoint to ensure regressions are covered and make sure there's a gh action to run it

Yeah, I also wanted to add some basic tests and gh actions. Should be mostly ready tomorrow.

@floreks floreks marked this pull request as ready for review October 28, 2024 17:51
@floreks floreks changed the title wip: feat: add ai proxy feat: add ai proxy Oct 28, 2024
@michaeljguarino michaeljguarino merged commit 7401ede into master Oct 29, 2024
12 checks passed
@michaeljguarino michaeljguarino deleted the sebastian/prod-2766-openai-proxy-for-autoconfiguring-ai-for-cloud-console-users branch October 29, 2024 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants