diff --git a/model-serving/1.1/chapter2/_images/openshift_ai_operator_search.png b/model-serving/1.1/chapter2/_images/openshift_ai_operator_search.png new file mode 100644 index 0000000..dbcafeb Binary files /dev/null and b/model-serving/1.1/chapter2/_images/openshift_ai_operator_search.png differ diff --git a/model-serving/1.1/chapter2/index.html b/model-serving/1.1/chapter2/index.html index 36a2dfc..a6ca908 100644 --- a/model-serving/1.1/chapter2/index.html +++ b/model-serving/1.1/chapter2/index.html @@ -169,7 +169,7 @@

Product Documentation for Red Hat OpenShift AI Self-Managed 2.8.

+For information about OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see Product Documentation for Red Hat OpenShift AI Self-Managed 2.10.

@@ -191,49 +191,28 @@

<

In addition to the Red Hat OpenShift AI Operator there are additional operators that you may need to install depending on which features and components of Red Hat OpenShift AI you want to utilize.

-
- - - - - -
- - -
-

To support the KServe component, which is used by the single-model serving platform to serve large models, install the Operators for Red Hat OpenShift Serverless and Red Hat OpenShift Service Mesh.

-
-
-
-
OpenShift Serveless Operator
+
Red Hat OpenShift Serverless Operator
-

The OpenShift Serveless Operator is a prerequisite for the Single Model Serving Platform.

+

The Red Hat OpenShift Serverless operator provides a collection of APIs that enables containers, microservices and functions to run "serverless". The Red Hat OpenShift Serverless Operator is required if you want to install the Single-model serving platform component.

-
OpenShift Service Mesh Operator
+
Red Hat OpenShift Service Mesh Operator
-

The OpenShift Service Mesh Operator is a prerequisite for the Single Model Serving Platform.

+

Red Hat OpenShift Service Mesh operator provides an easy way to create a network of deployed services that provides discovery, load balancing, service-to-service authentication, failure recovery, metrics, and monitoring. The Red Hat OpenShift Serverless Operator is required if you want to install the Single-model serving platform component.

-
Red Hat OpenShift Pipelines Operator
+
Red Hat Authorino (technical preview) Operator
-

The Red Hat OpenShift Pipelines Operator is a prerequisite for the Single Model Serving Platform.

+

Red Hat Authorino is an open source, Kubernetes-native external authorization service to protect APIs. The Red Hat Authorino Operator is required to support enforcing authentication policies in Red Hat OpenShift AI.

-
- - - - - -
- - +
+

The following Operators are required to support the use of Nvidia GPUs (accelerators) with OpenShift AI:

-
+
diff --git a/model-serving/1.1/chapter2/section1.html b/model-serving/1.1/chapter2/section1.html index 105691a..2009366 100644 --- a/model-serving/1.1/chapter2/section1.html +++ b/model-serving/1.1/chapter2/section1.html @@ -161,13 +161,6 @@

Installing Red Hat OpenShift AI Using the Web Console

Red Hat OpenShift AI is available as an operator via the OpenShift Operator Hub. You will install the Red Hat OpenShift AI operator and dependencies using the OpenShift web console in this section.

-
-
- -
-
@@ -207,16 +200,13 @@

diff --git a/model-serving/1.1/chapter3/section1.html b/model-serving/1.1/chapter3/section1.html index 0ebe893..4760429 100644 --- a/model-serving/1.1/chapter3/section1.html +++ b/model-serving/1.1/chapter3/section1.html @@ -171,23 +171,21 @@

Creating OpenShift AI Resources - 1

Model Serving Runtimes

-

A model-serving runtime provides integration with a specified model server and the model frameworks that it supports. By default, Red Hat OpenShift AI includes the following Model RunTimes:

+

A model-serving runtime provides integration with a specified model server and the model frameworks that it supports. By default, Red Hat OpenShift AI includes the following model serving runTimes:

+
+
+
+
Multi-model
+* OpenVINO Model Server - Multi-model
+Single-model
+* OpenVINO Model Server
+* Caikit TGIS for KServe
+* TGIS Standalone for KServe
+* vLLM For KServe
-
-
    -
  • -

    OpenVINO Model Server runtime.

    -
    • -
  • -

    Caikit TGIS for KServe

    -
    • -
  • -

    TGIS Standalone for KServe

    -
    • -
-

However, if these runtime do not meet your needs (if they don’t support a particular model framework, for example), you might want to add your own custom runtimes.

+

However, if these runtimes do not meet your needs (if they don’t support a particular model framework, for example), you might want to add your own custom runtimes.

As an administrator, you can use the OpenShift AI interface to add and enable custom model-serving runtimes. You can then choose from your enabled runtimes when you create a new model server.

@@ -247,7 +245,7 @@

Serving an LLM using OpenShift AI

This program was designed to guide you through the process of installing an OpenShift AI Platform using the OpenShift Container Platform Web Console UI. We get hands-on experience in each component needed to enable a RHOAI Platform using an Openshift Container Platform Cluster.

-

Once we have an operational OpenShift AI Platform, we will login and begin the configuration of: Model Runtimes, Data Science Projects, Data connections, & finally use a jupyter notebook to infer the answers to easy questions.

+

Once we have an operational OpenShift AI Platform, we will login and begin the configuration of: model runtimes, data science projects, data connections, & finally use a jupyter notebook to infer the answers to easy questions.

There will be some challenges along the way, all designed to teach us about a component, or give us the knowledge needed to utilize OpenShift AI and host a Large Language Model.

@@ -237,13 +237,6 @@

-
-
- -
-

When ordering this catalog item in RHDP:

@@ -267,7 +260,7 @@

-

For Red Hat partners who do not have access to RHDP, provision an environment using the Red Hat Hybrid Cloud Console. Unfortunately, the labs will NOT work on the trial sandbox environment. You need to provision an OpenShift AI cluster on-premises, or in the supported cloud environments by following the product documentation at Product Documentation for Red Hat OpenShift AI 2024.

+

For Red Hat partners who do not have access to RHDP, provision an environment using the Red Hat Hybrid Cloud Console. Unfortunately, the labs will NOT work on the trial sandbox environment. You need to provision an OpenShift AI cluster on-premises, or in the supported cloud environments by following the product documentation at Product Documentation for installing Red Hat OpenShift AI 2.10.

@@ -313,7 +306,7 @@

Objectives

Import (from git repositories), interact with LLM model via Jupyter Notebooks

  • -

    Experiment with the Mistral LLM

    +

    Experiment with the Mistral LLM and Llama3 large language models