diff --git a/docs/source/_static/bedrock-chat-basemodel-arn.png b/docs/source/_static/bedrock-chat-basemodel-arn.png index fbd0457f5..2cd59e05e 100644 Binary files a/docs/source/_static/bedrock-chat-basemodel-arn.png and b/docs/source/_static/bedrock-chat-basemodel-arn.png differ diff --git a/docs/source/_static/bedrock-chat-basemodel-modelid.png b/docs/source/_static/bedrock-chat-basemodel-modelid.png index e930edb3c..eeadaddae 100644 Binary files a/docs/source/_static/bedrock-chat-basemodel-modelid.png and b/docs/source/_static/bedrock-chat-basemodel-modelid.png differ diff --git a/docs/source/_static/bedrock-chat-custom-model-arn.png b/docs/source/_static/bedrock-chat-custom-model-arn.png index 111fc7cce..cdf7949a8 100644 Binary files a/docs/source/_static/bedrock-chat-custom-model-arn.png and b/docs/source/_static/bedrock-chat-custom-model-arn.png differ diff --git a/docs/source/_static/bedrock-cross-region-inference.png b/docs/source/_static/bedrock-cross-region-inference.png new file mode 100644 index 000000000..15d017dbf Binary files /dev/null and b/docs/source/_static/bedrock-cross-region-inference.png differ diff --git a/docs/source/_static/bedrock-model-select.png b/docs/source/_static/bedrock-model-select.png index a321c52cf..93fc52f94 100644 Binary files a/docs/source/_static/bedrock-model-select.png and b/docs/source/_static/bedrock-model-select.png differ diff --git a/docs/source/users/bedrock.md b/docs/source/users/bedrock.md index 558bf93c9..8bcce8a57 100644 --- a/docs/source/users/bedrock.md +++ b/docs/source/users/bedrock.md @@ -1,8 +1,10 @@ # Using Amazon Bedrock with Jupyter AI -[(Return to Chat Interface page for Bedrock)](index.md#amazon-bedrock-usage) +[(Return to the Chat Interface page)](index.md#amazon-bedrock-usage) -Bedrock supports many language model providers such as AI21 Labs, Amazon, Anthropic, Cohere, Meta, and Mistral AI. To use the base models from any supported provider make sure to enable them in Amazon Bedrock by using the AWS console. Go to Amazon Bedrock and select `Model Access` as shown here: +Bedrock supports many language model providers such as AI21 Labs, Amazon, Anthropic, Cohere, Meta, and Mistral AI. To use the base models from any supported provider make sure to enable them in Amazon Bedrock by using the AWS console. You should also select embedding models in Bedrock in addition to language completion models if you intend to use retrieval augmented generation (RAG) on your documents. + +Go to Amazon Bedrock and select `Model Access` as shown here: Screenshot of the Jupyter AI chat panel where the base language model and embedding model is selected. -Bedrock also allows custom models to be trained from scratch or fine-tuned from a base model. Jupyter AI enables a custom model to be called in the chat panel using its `arn` (Amazon Resource Name). As with custom models, you can also call a base model by its `model id` or its `arn`. An example of using a base model with its `model id` through the custom model interface is shown below: +If your provider requires an API key, please enter it in the box that will show for that provider. Make sure to click on `Save Changes` to ensure that the inputs have been saved. + +Bedrock also allows custom models to be trained from scratch or fine-tuned from a base model. Jupyter AI enables a custom model to be called in the chat panel using its `arn` (Amazon Resource Name). A fine-tuned model will have your 12-digit customer number in the ARN: + +Screenshot of the Jupyter AI chat panel where the custom model is selected using model arn. + +As with custom models, you can also call a base model by its `model id` or its `arn`. An example of using a base model with its `model id` through the custom model interface is shown below: Screenshot of the Jupyter AI chat panel where the base model is selected using its ARN. +## Fine-tuning in Bedrock + To train a custom model in Amazon Bedrock, select `Custom models` in the Bedrock console as shown below, and then you may customize a base model by fine-tuning it or continuing to pre-train it: Screenshot of Bedrock cross-region inference usage. + +## Summary + +1. Bedrock Base models: All available models will already be available in the drop down model list. The above interface also allows use of base model IDs or ARNs, though this is unnecessary as they are in the dropdown list. +2. Bedrock Custom models: If you have fine-tuned a Bedrock base model you may use the ARN for this custom model. Make sure to enter the correct provider information, such as `amazon`, `anthropic`, `cohere`, `meta`, `mistral` (always in lower case). +3. Provisioned Models: These are models that run on dedicated endpoints. Users can purchase Provisioned Throughput Model Units to get faster throughput. These may be base or custom models. Enter the ARN for these models in the Model ID field. +4. Cross-region Inference: Use the Inference profile ID for the cross-region model instead of the ARN. + +[(Return to the Chat Interface page)](index.md#amazon-bedrock-usage) diff --git a/docs/source/users/index.md b/docs/source/users/index.md index f4505805a..f0bd975ba 100644 --- a/docs/source/users/index.md +++ b/docs/source/users/index.md @@ -334,29 +334,9 @@ The chat backend remembers the last two exchanges in your conversation and passe ### Amazon Bedrock Usage -Jupyter AI enables use of language models hosted on [Amazon Bedrock](https://aws.amazon.com/bedrock/) on AWS. First, ensure that you have authentication to use AWS using the `boto3` SDK with credentials stored in the `default` profile. Guidance on how to do this can be found in the [`boto3` documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). +Jupyter AI enables use of language models hosted on [Amazon Bedrock](https://aws.amazon.com/bedrock/) on AWS. Ensure that you have authentication to use AWS using the `boto3` SDK with credentials stored in the `default` profile. Guidance on how to do this can be found in the [`boto3` documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html). -For more detailed workflows, see [Using Amazon Bedrock with Jupter AI](bedrock.md). - -Bedrock supports many language model providers such as AI21 Labs, Amazon, Anthropic, Cohere, Meta, and Mistral AI. To use the base models from any supported provider make sure to enable them in Amazon Bedrock by using the AWS console. You should also select embedding models in Bedrock in addition to language completion models if you intend to use retrieval augmented generation (RAG) on your documents. - -You may now select a chosen Bedrock model from the drop-down menu box title `Completion model` in the chat interface. If RAG is going to be used then pick an embedding model that you chose from the Bedrock models as well. An example of these selections is shown below: - -Screenshot of the Jupyter AI chat panel where the base language model and embedding model is selected. - -If your provider requires an API key, please enter it in the box that will show for that provider. Make sure to click on `Save Changes` to ensure that the inputs have been saved. - -Bedrock also allows custom models to be trained from scratch or fine-tuned from a base model. Jupyter AI enables a custom model to be called in the chat panel using its `arn` (Amazon Resource Name). The interface is shown below: - -Screenshot of the Jupyter AI chat panel where the custom model is selected using model arn. - -For detailed workflows, see [Using Amazon Bedrock with Jupter AI](bedrock.md). +For details on enabling model access in your AWS account, using cross-region inference, or invoking custom/provisioned models, please see our dedicated documentation page on [using Amazon Bedrock in Jupyter AI](bedrock.md). ### SageMaker endpoints usage diff --git a/packages/jupyter-ai-magics/jupyter_ai_magics/partner_providers/aws.py b/packages/jupyter-ai-magics/jupyter_ai_magics/partner_providers/aws.py index 4baff07a6..9feb39441 100644 --- a/packages/jupyter-ai-magics/jupyter_ai_magics/partner_providers/aws.py +++ b/packages/jupyter-ai-magics/jupyter_ai_magics/partner_providers/aws.py @@ -124,8 +124,9 @@ class BedrockCustomProvider(BaseProvider, ChatBedrock): ), ] help = ( - "Specify the ARN (Amazon Resource Name) of the custom/provisioned model as the model ID. For more information, see the [Amazon Bedrock model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html).\n\n" - "The model provider must also be specified below. This is the provider of your foundation model *in lowercase*, e.g. `amazon`, `anthropic`, `meta`, or `mistral`." + "- For Cross-Region Inference use the appropriate `Inference profile ID` (Model ID with a region prefix, e.g., `us.meta.llama3-2-11b-instruct-v1:0`). See the [inference profiles documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html). \n" + "- For custom/provisioned models, specify the model ARN (Amazon Resource Name) as the model ID. For more information, see the [Amazon Bedrock model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html).\n\n" + "The model provider must also be specified below. This is the provider of your foundation model *in lowercase*, e.g., `amazon`, `anthropic`, `cohere`, `meta`, or `mistral`." ) registry = True