Skip to content

Commit

Permalink
Merge branch 'main' into Technical-Edits
Browse files Browse the repository at this point in the history
  • Loading branch information
kknoxrht authored Jun 12, 2024
2 parents f391cba + 06edc31 commit 8ed76e8
Show file tree
Hide file tree
Showing 8 changed files with 76 additions and 73 deletions.
1 change: 1 addition & 0 deletions modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
= Serving LLM Models on OpenShift AI
:navtitle: Home


video::intro_v4.mp4[width=800,start=60,opts=autoplay]

Welcome to this quick course on _Serving an LLM using OpenShift AI_.
Expand Down
8 changes: 5 additions & 3 deletions modules/chapter2/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
OpenShift AI is supported in two configurations:

* A managed cloud service add-on for *Red Hat OpenShift Dedicated* (with a Customer Cloud Subscription for AWS or GCP) or for Red Hat OpenShift Service on Amazon Web Services (ROSA).
For information about OpenShift AI on a Red Hat managed environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_cloud_service/1[Product Documentation for Red Hat OpenShift AI Cloud Service 1]
For information about OpenShift AI on a Red Hat managed environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_cloud_service/1[Product Documentation for Red Hat OpenShift AI Cloud Service 1].

* Self-managed software that you can install on-premise or on the public cloud in a self-managed environment, such as *OpenShift Container Platform*.
For information about OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.8[Product Documentation for Red Hat OpenShift AI Self-Managed 2.8]
For information about OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.8[Product Documentation for Red Hat OpenShift AI Self-Managed 2.8].

In this course we cover installation of *Red Hat OpenShift AI self-managed* using the OpenShift Web Console.

Expand All @@ -28,9 +28,11 @@ To support the KServe component, which is used by the single-model serving platf
====

https://docs.openshift.com/container-platform/latest/hardware_enablement/psap-node-feature-discovery-operator.html[OpenShift Serveless Operator]::
// Is this the correct link for OpenShift Serveless Operator?
The *OpenShift Serveless Operator* is a prerequisite for the *Single Model Serving Platform*.

https://docs.openshift.com/container-platform/latest/hardware_enablement/psap-node-feature-discovery-operator.html[OpenShift Service Mesh Operator]::
// Is this the correct link for OpenShift Service Mesh Operator?
The *OpenShift Service Mesh Operator* is a prerequisite for the *Single Model Serving Platform*.

https://www.redhat.com/en/technologies/cloud-computing/openshift/pipelines[Red{nbsp}Hat OpenShift Pipelines Operator]::
Expand All @@ -40,7 +42,7 @@ The *Red{nbsp}Hat OpenShift Pipelines Operator* is a prerequisite for the *Singl

[NOTE]
====
The following Operators are required to support the use of Nvidia GPUs (accelerators) with OpenShift AI
The following Operators are required to support the use of Nvidia GPUs (accelerators) with OpenShift AI:
====

https://docs.openshift.com/container-platform/latest/hardware_enablement/psap-node-feature-discovery-operator.html[Node Feature Discovery Operator]::
Expand Down
4 changes: 2 additions & 2 deletions modules/chapter2/pages/section1.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This exercise uses the Red Hat Demo Platform; specifically the OpenShift Contain

. Login to the Red Hat OpenShift using a user which has the _cluster-admin_ role assigned.

. Navigate to **Operators** -> **OperatorHub** and search for each of the following Operators individually. Click on the button or tile for each. In the pop up window that opens, ensure you select the latest version in the *stable* channel and click on **Install** to open the operator's installation view. For this lab you can skip the installation of the optional operators
. Navigate to **Operators** -> **OperatorHub** and search for each of the following Operators individually. Click on the button or tile for each. In the pop up window that opens, ensure you select the latest version in the *stable* channel and click on **Install** to open the operator's installation view. For this lab you can skip the installation of the optional operators.

[*] You do not have to wait for the previous Operator to complete before installing the next. For this lab you can skip the installation of the optional operators as there is no GPU.

Expand Down Expand Up @@ -44,4 +44,4 @@ image::openshiftai_operator.png[width=640]
. The operator Installation progress window will pop up. The installation may take a couple of minutes.


WARNING: Do proceed with the installation past this point. In order to access the LLM remotely; There will be some modifcations to the Data Science Cluster YAML file prior to completing the installation of Red Hat OpenShift AI.
WARNING: Do not proceed with the installation past this point. In order to access the LLM remotely; you will need to make some modifcations to the Data Science Cluster YAML file prior to completing the installation of Red Hat OpenShift AI.
14 changes: 7 additions & 7 deletions modules/chapter2/pages/section2.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@

An SSL/TLS certificate is a digital object that allows systems to verify the identity & subsequently establish an encrypted network connection to another system using the Secure Sockets Layer/Transport Layer Security (SSL/TLS) protocol.

By default, the Single Model Serving Platform in Openshift AI uses a self-signed certificate generated at installation for the endpoints that are created when deploying a Model server.
By default, the Single Model Serving Platform in Openshift AI uses a self-signed certificate generated during installation for the endpoints that are created when deploying a Model server.

This can be counter-intuitive because the OCP Cluster already has certificates configured which will be used by default for endpoints like Routes.
This can be counterintuitive because the OCP Cluster already has certificates configured which will be used by default for endpoints like Routes.

This following procedure explains how to use the same certificate from the OpenShift Container cluster for OpenShift AI.

== Use OpenShift Certificates for Ingress Routes

[NOTE]
Most customers will not use the self-signed certificates, opting instead to use certificates generated by their own authority. Therefore this step of adding secrets to OpenShift & OpenShift AI is common process during installation.
Most customers will not use the self-signed certificates, opting instead to use certificates generated by their own authority. Therefore, this step of adding secrets to OpenShift & OpenShift AI is common process during installation.

=== Navigate to the OpenShift Container Cluster Dashboard

Expand All @@ -29,7 +29,7 @@ image::openshiftingress_project.png[width=640]

. Locate the file named *ingress-certs-(XX-XX-2024)*, type should be *Opaque*
. Click on the filename to open the secret, Select the *YAML Tab*
. Copy all the text from the window, insure you scroll down. (CTL-A should work).
. Copy all the text from the window, and ensure that you scroll down. (CTL-A should work).

*Clean & Deploy the Secret YAML Text:*

Expand Down Expand Up @@ -92,11 +92,11 @@ Once you have made those changes to the YAML file, *Click Create* to Deploy the

image::createDSC.png[width=640]

Single Model Serve Platform will now be deployed / expose ingress connections with the same certificate as OpenShift Routes. Endpoints will be accessible using TLS without having to ignore error messages or create special configurations.
Single Model Serve Platform will now be deployed to expose ingress connections with the same certificate as OpenShift Routes. Endpoints will be accessible using TLS without having to ignore error messages or create special configurations.

== OpenShift AI install summary

Congradulations, you have successful completed the installation of OpenShift AI on an OpenShift Container Cluster. OpenShift AI is now running as new Dashboard!
Congratulations, you have successfully completed the installation of OpenShift AI on an OpenShift Container Cluster. OpenShift AI is now running on a new Dashboard!


* We Installed the required OpenShift AI Operators
Expand All @@ -106,4 +106,4 @@ Congradulations, you have successful completed the installation of OpenShift AI

Additionally, we took this installation a step further by sharing TLS certificates from the OpenShift Cluster with OpenShift AI.

We pick up working OpenShift AI UI in the next Chapter.
We will pick up working with the OpenShift AI UI in the next Chapter.
6 changes: 3 additions & 3 deletions modules/chapter3/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

This chapter begins with running & configured OpenShift AI environment, if you don't already have your environment running, head over to Chapter 2.

Lots to cover in section 1, we add the Ollama custom Runtime, Create a Data Science Project, Setup Storage, Create a Workbench, and finally serving the Ollama Framework, utilizing the Single Model Serving Platform to deliver our model to our Notebook Application.
There's a lot to cover in section 1, we add the Ollama custom Runtime, create a data science project, setup storage, create a workbench, and finally serve the Ollama Framework, utilizing the Single Model Serving Platform to deliver our model to our Notebook Application.


In section 2 we will explore using the Jupyter Notebook from our workbench, infere data from the Mistral 7B LLM. While less technical than previous section of this hands on course, there are some steps download the Mistral Model, updating our notebook with inference endpoint, and evaluating our Models performance.
In section 2 we will explore using the Jupyter Notebook from our workbench to infere data from the Mistral 7B LLM. While less technical than previous section of this hands-on course, there are some steps to download the Mistral Model, update our notebook with inference endpoint, and evaluate our Models performance.

Let's get started ---
Let's get started!
Loading

0 comments on commit 8ed76e8

Please sign in to comment.