Update model fields immediately on save

[dlqqq]

Description

• Closes api_version in GUI not considered, need to set OPENAI_API_VERSION #807.
• Closes Missing Request Schema for SageMaker Endpoint in jupyter-ai Extension #1006.
• Closes profile_name field ignored for Amazon Bedrock Chat Models in jupyter-ai Extension #1007.
• Closes Can't pass Base API URL from UI #1088.
• Closes Provider text fields values saved from Chat UI are only applied after Jupyter Lab is restarted. #1118.

Fixes usage of model fields by respecting the updated values immediately on save (without requiring a restart).

Model fields are used to specify provider-specific keyword arguments directly from Jupyter AI's Settings UI. These include Base API URL for OpenAI, Region name for Amazon Bedrock, etc.

Demo

Screen.Recording.2024-11-27.at.10.33.12.AM.mov

Details for contributors

• Fixes a bug introduced by Setting default model providers #421: BaseProvider.get_model_parameters() would return the fields set by the existing config.json on init, instead of the user-specified overrides in the AiExtension.model_parameters trait.

  • This was due to the ConfigManager accidentally merging the existing config into self.settings["model_parameters"] instead of a deep copy of that dictionary.

• Fixes a bug introduced by Distinguish between completion and chat models #711: Model fields were not being returned at all from the dictionary returned by ConfigManager.lm_provider_params.

  • The fields specified by config.json on init still got passed to the LLM due to the bug introduced by Setting default model providers #421. Essentially, the bug introduced by Setting default model providers #421 "covered up" the bug introduced by Distinguish between completion and chat models #711. This explains why the issue was difficult to reproduce and why a restart fixed it.

• Makes the allowed_providers, blocked_providers, allowed_models, blocked_models arguments to ConfigManager optional and default to None, as indicated by their type signatures.

  • This change will not have any impact on users, as Jupyter AI always provides these arguments on extension init. This is purely for the convenience of contributors authoring tests.

• Adds unit test coverage for both aforementioned bugs.

Testing instructions

• Make the following changes in config.py:
  • Change deepcopy(self._defaults.get(config_key)) to self._defaults.get(config_key) near line 240.
  • Remove the line **fields, near line 475.
• Run pytest, and verify that the 2 added tests fail. This asserts that the tests actually capture the bugs introduced by Setting default model providers #421 and Distinguish between completion and chat models #711.
• Revert the changes and re-run pytest, and verify the 2 added tests now pass.
• Run Jupyter AI from this branch, and verify that model fields are updated immediately on save.

[srdas]

1. Tested the deepcopy change by removing the code for it and failing pytest. Reinstating it works as intended.
2. Tested that the changes to model fields are saved and there is no need to restart Jupyter AI. The system logs report the change as intended. Resolves Issue Provider text fields values saved from Chat UI are only applied after Jupyter Lab is restarted. #1118 .
3. Issue profile_name field ignored for Amazon Bedrock Chat Models in jupyter-ai Extension #1007 : used blank profile_name and new region and the saved model works well.
4. Issue Missing Request Schema for SageMaker Endpoint in jupyter-ai Extension #1006 (SageMaker endpoint request schema error). This still needs to be tried though there may not be JAI users who are calling SageMaker for the models.
5. Issue api_version in GUI not considered, need to set OPENAI_API_VERSION #807 to be tested but no access to Azure. Hopefully @OliverKleinBST can test this.

[krassowski]

I think this should go in, but I suspect that completions_fields does not work correctly.

I do not have a way to test it (no access to Amazon models nor Azure which are the ones that are parametrized). Maybe it would be good to expose some model parameters (temperature) as fields so that this could be tested by folks easily?

[krassowski]

I think this should go in, but I suspect that completions_fields does not work correctly.

To clarify, I think this is not a result of this PR but since #711. Yet, it might be that fixing these together could be easier than separately.

[srdas]

All LGTM.

[dlqqq]

@krassowski Thank you for the callout. I agree it would be good to also include a fix for completion fields before the next patch release. However, I think it should be done in a separate PR since it can be done independently.

I can help with these changes. It shouldn't be too much work; the _provider_params() method needs to read from config.completion_fields instead of config.fields when getting params for a completion model. Let me know if you'd like to help, and I'll open an issue and assign it to you. Otherwise I'll open a PR for this before the next patch release.

Proceeding to merge. Thank you @krassowski and @srdas for the testing & feedback!

[krassowski]

Let me know if you'd like to help, and I'll open an issue and assign it to you. Otherwise I'll open a PR for this before the next patch release.

I'm unlikely to find time this week, if you see how to fix it easily feel free to go ahead. Also, agree a separate PR would be better.

[dlqqq]

@krassowski No worries! Take care.

[dlqqq]

@meeseeksdev please backport to v3-dev

[OliverKleinBST]

Confirm that it solves also the problem reported in #807