Skip to content

Commit

Permalink
Content edits suggested for this training
Browse files Browse the repository at this point in the history
  • Loading branch information
Rutuja-desh committed Jun 10, 2024
1 parent f804878 commit 58f5e11
Show file tree
Hide file tree
Showing 9 changed files with 90 additions and 87 deletions.
6 changes: 3 additions & 3 deletions modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Serving LLM Models on OpenShift AI
:navtitle: Home

Welcome to this Quick course on _Deploying an LLM using OpenShift AI_. This is the first of a set of advanced courses about Red Hat OpenShift AI:
Welcome to this Quick course on _Deploying an LLM using OpenShift AI_. This is the first of a set of advanced courses about Red Hat OpenShift AI.

IMPORTANT: The hands-on labs in this course were created and tested with RHOAI v2.9.1. Labs should mostly work without any changes in minor dot release upgrades of the product. Please open issues in this repository if you face any issue.

Expand Down Expand Up @@ -30,7 +30,7 @@ When ordering this catalog item in RHDP:

* Enter Learning RHOAI in the Salesforce ID field

* Scroll to the bottom, check the box to confirm acceptance of terms and conditions
* Scroll to the bottom, and check the box to confirm acceptance of terms and conditions

* Click order

Expand All @@ -48,7 +48,7 @@ The overall objectives of this introductory course include:

* Familiarize utilizing Red Hat OpenShift AI to Serve & Interact with an LLM.

* Installing Red Hat OpenShift AI Operator & Dependencies
* Install Red Hat OpenShift AI Operator & Dependencies

* Add a custom Model Serving Runtime

Expand Down
23 changes: 12 additions & 11 deletions modules/chapter1/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,37 +6,38 @@ This segment of the course provides context to know & analogies to guide us to c

=== Why this technical course ?

Previously, read a post on LinkenIn and felt it summed up the why quite nicely.
Previously, I read a post on LinkedIn and felt it summed up the
'why' quite nicely.

It described the basic idea that a Formula One Driver doesn't need to know the how to build an engine to be an F1 champion. However, she/he needs to have a *mechanical sympathy* which is understanding of car's mechanics to drive it effectively and get the best out it.
It described the basic idea that a Formula One Driver doesn't need to know the how to build an engine to be an F1 champion. However, they needs to have a *mechanical sympathy*, which is understanding of car's mechanics to drive it effectively and get the best out it.

The same applies to AI, we don't need to be AI experts to harness the power of large language models but we to develop a certain level of "mechanical sympathy" with how these Models are Selected, Operationized, Served, Infered from, and kept up to date, to work with AI in harmony. Not just as users, but as collaborators who understand the underlying mechanics to communicate with clients, partners, and co-workers effectively.
The same applies to AI : we don't need to be AI experts to harness the power of large language models but we do need to develop a certain level of "mechanical sympathy" with how these Models are selected, operationized, served, infered from, and kept up to date, to work with AI in harmony. Not just as users, but as collaborators who understand the underlying mechanics to communicate with clients, partners, and co-workers effectively.

It's not just about the Model itself, it's about the platform that empowers us to create trushtworthy AI applications and guides us in making informed choices.
It's not just about the Model itself; it's about the platform that empowers us to create trustworthy AI applications and guides us in making informed choices.

The true power lies in the platform that enables us to harness a diverse range of AI models, tools, infrastructure and operationalize our ML projects.

That platform, *OpenShift AI* is what we learn to create, configure, and utilize to Serve LLM Models in this quick course.
That platform, *OpenShift AI*, is what we learn to create, configure, and utilize to serve LLM Models in this quick course.


=== The Ollama Model Framework

LLMs - Large Language Models (LLMs) can generate new stories, summarize texts, and even performing advanced tasks like reasoning and problem solving, which is not only impressive but also remarkable due to their accessibility and easy integration into applications.
LLMs - Large Language Models (LLMs) can generate new stories, summarize texts, and even perform advanced tasks like reasoning and problem-solving, which is not only impressive but also remarkable due to their accessibility and easy integration into applications.

There are a lot of popular LLMs, Nonetheless, their operation remains the same: users provide instructions or tasks in natural language, and the LLM generates a response based on what the model "thinks" could be the continuation of the prompt.
There are a lot of popular LLMs. Nonetheless, their operation remains the same: users provide instructions or tasks in natural language, and the LLM generates a response based on what the model "thinks" could be the continuation of the prompt.

Ollama is not an LLM Model - Ollama is a relatively new but powerful open-source framework designed for serving machine learning models. It's designed to be efficient, scalable, and easy to use, making it an attractive option for developers and organizations looking to deploy their AI models into production.
Ollama is not an LLM model. Ollama is a relatively new but powerful open-source framework designed for serving machine learning models. It's designed to be efficient, scalable, and easy to use, making it an attractive option for developers and organizations looking to deploy their AI models into production.

==== How does Ollama work?


*At its core, Ollama simplifies the process of downloading, installing, and interacting with a wide range of LLMs, empowering users to explore their capabilities without the need for extensive technical expertise or reliance on cloud-based platforms.

In this course, we will focus on single LLM, Mistral. However, with the understanding of the Ollama Framework, we will be able to work with a variety of large language models utilizing the exact same configuration.
In this course, we will focus on a single LLM, Mistral. However, with an understanding of the Ollama Framework, we will be able to work with a variety of large language models using the exact same configuration.

You be able to switch models in minutes, all running on the same platform. This will enable you test, compare, and evalute multiple models with the skills gained in the course.
You will be able to switch models in minutes, all running on the same platform. This will enable you test, compare, and evaluate multiple models with the skills gained in the course.

*Experimentation and Learning*

Ollama provides a powerful platform for experimentation and learning, allowing users to explore the capabilities and limitations of different LLMs, understand their strengths and weaknesses, and develop skills in prompt engineering and LLM interaction. This hands-on approach fosters a deeper understanding of AI technology and empowers users to push the boundaries of what’s possible.*
Ollama provides a powerful platform for experimentation and learning, allowing users to explore the capabilities and limitations of different LLMs, understand their strengths and weaknesses, and develop skills in prompt engineering and LLM interaction. This hands-on approach fosters a deeper understanding of AI technology and empowers users to push the boundaries of what’s possible.

8 changes: 5 additions & 3 deletions modules/chapter2/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
OpenShift AI is supported in two configurations:

* A managed cloud service add-on for *Red Hat OpenShift Dedicated* (with a Customer Cloud Subscription for AWS or GCP) or for Red Hat OpenShift Service on Amazon Web Services (ROSA).
For information about OpenShift AI on a Red Hat managed environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_cloud_service/1[Product Documentation for Red Hat OpenShift AI Cloud Service 1]
For information about OpenShift AI on a Red Hat managed environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_cloud_service/1[Product Documentation for Red Hat OpenShift AI Cloud Service 1].

* Self-managed software that you can install on-premise or on the public cloud in a self-managed environment, such as *OpenShift Container Platform*.
For information about OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.8[Product Documentation for Red Hat OpenShift AI Self-Managed 2.8]
For information about OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.8[Product Documentation for Red Hat OpenShift AI Self-Managed 2.8].

In this course we cover installation of *Red Hat OpenShift AI self-managed* using the OpenShift Web Console.

Expand All @@ -32,15 +32,17 @@ To support the KServe component, which is used by the single-model serving platf
====

https://docs.openshift.com/container-platform/latest/hardware_enablement/psap-node-feature-discovery-operator.html[OpenShift Serveless Operator]::
// Is this the correct link for OpenShift Serveless Operator?
The *OpenShift Serveless Operator* is a prerequisite for the *Single Model Serving Platform*.

https://docs.openshift.com/container-platform/latest/hardware_enablement/psap-node-feature-discovery-operator.html[OpenShift Service Mesh Operator]::
// Is this the correct link for OpenShift Service Mesh Operator?
The *OpenShift Service Mesh Operator* is a prerequisite for the *Single Model Serving Platform*.


[NOTE]
====
The following Operators are required to support the use of Nvidia GPUs (accelerators) with OpenShift AI
The following Operators are required to support the use of Nvidia GPUs (accelerators) with OpenShift AI:
====

https://docs.openshift.com/container-platform/latest/hardware_enablement/psap-node-feature-discovery-operator.html[Node Feature Discovery Operator]::
Expand Down
4 changes: 2 additions & 2 deletions modules/chapter2/pages/section1.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ IMPORTANT: The installation requires a user with the _cluster-admin_ role

. Login to the Red Hat OpenShift using a user which has the _cluster-admin_ role assigned.

. Navigate to **Operators** -> **OperatorHub** and search for each of the following Operators individually. Click on the button or tile for each. In the pop up window that opens, ensure you select the latest version in the *stable* channel and click on **Install** to open the operator's installation view. For this lab you can skip the installation of the optional operators
. Navigate to **Operators** -> **OperatorHub** and search for each of the following Operators individually. Click on the button or tile for each. In the pop up window that opens, ensure you select the latest version in the *stable* channel and click on **Install** to open the operator's installation view. For this lab you can skip the installation of the optional operators.

[*] You do not have to wait for the previous Operator to complete before installing the next. For this lab you can skip the installation of the optional operators as there is no GPU.

Expand Down Expand Up @@ -40,4 +40,4 @@ IMPORTANT: The installation requires a user with the _cluster-admin_ role
. The operator Installation progress window will pop up. The installation may take a couple of minutes.


WARNING: Do proceed with the installation past this point. In order to access the LLM remotely; There will be some modifcations to the Data Science Cluster YAML file prior to completing the installation of Red Hat OpenShift AI.
WARNING: Do not proceed with the installation past this point. In order to access the LLM remotely; you will need to make some modifcations to the Data Science Cluster YAML file prior to completing the installation of Red Hat OpenShift AI.
14 changes: 7 additions & 7 deletions modules/chapter2/pages/section2.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@

An SSL/TLS certificate is a digital object that allows systems to verify the identity & subsequently establish an encrypted network connection to another system using the Secure Sockets Layer/Transport Layer Security (SSL/TLS) protocol.

By default, the Single Model Serving Platform in Openshift AI uses a self-signed certificate generated at installation for the endpoints that are created when deploying a Model server.
By default, the Single Model Serving Platform in Openshift AI uses a self-signed certificate generated during installation for the endpoints that are created when deploying a Model server.

This can be counter-intuitive because the OCP Cluster already has certificates configured which will be used by default for endpoints like Routes.
This can be counterintuitive because the OCP Cluster already has certificates configured which will be used by default for endpoints like Routes.

This following procedure explains how to use the same certificate from the OpenShift Container cluster for OpenShift AI.

== Use OpenShift Certificates for Ingress Routes

[NOTE]
Most customers will not use the self-signed certificates, opting instead to use certificates generated by their own authority. Therefore this step of adding secrets to OpenShift & OpenShift AI is common process during installation.
Most customers will not use the self-signed certificates, opting instead to use certificates generated by their own authority. Therefore, this step of adding secrets to OpenShift & OpenShift AI is common process during installation.

=== Navigate to the OpenShift Container Cluster Dashboard

Expand All @@ -26,7 +26,7 @@ The content of the Secret (data) should contain two items, *tls.cert* and *tls.k
. Select the *openshift-ingress* project from the list.
. Locate the file named *ingress-certs-(XX-XX-2024)*, type should be *Opaque*
. Click on the filename to open the secret, Select the *YAML Tab*
. Copy all the text from the window, insure you scroll down. (CTL-A should work).
. Copy all the text from the window, and ensure that you scroll down. (CTL-A should work).

*Clean & Deploy the Secret YAML Text:*

Expand Down Expand Up @@ -84,11 +84,11 @@ serving:

Once you have made those changes to the YAML file, *Click Create* to Deploy the Data Science Cluster.

Single Model Serve Platform will now be deployed / expose ingress connections with the same certificate as OpenShift Routes. Endpoints will be accessible using TLS without having to ignore error messages or create special configurations.
Single Model Serve Platform will now be deployed to expose ingress connections with the same certificate as OpenShift Routes. Endpoints will be accessible using TLS without having to ignore error messages or create special configurations.

== Epilogue

Congradulations, you have successful completed the installation of OpenShift AI on an OpenShift Container Cluster. OpenShift AI is now running as new Dashboard!
Congratulations, you have successfully completed the installation of OpenShift AI on an OpenShift Container Cluster. OpenShift AI is now running on a new Dashboard!


* We Installed the required OpenShift AI Operators
Expand All @@ -98,4 +98,4 @@ Congradulations, you have successful completed the installation of OpenShift AI

Additionally, we took this installation a step further by sharing TLS certificates from the OpenShift Cluster with OpenShift AI.

We pick up working OpenShift AI UI in the next Chapter.
We will pick up working with the OpenShift AI UI in the next Chapter.
6 changes: 3 additions & 3 deletions modules/chapter3/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

This chapter begins with running & configured OpenShift AI environment, if you don't already have your environment running, head over to Chapter 2.

Lots to cover in section 1, we add the Ollama custom Runtime, Create a Data Science Project, Setup Storage, Create a Workbench, and finally serving the Ollama Framework, utilizing the Single Model Serving Platform to deliver our model to our Notebook Application.
There's a lot to cover in section 1, we add the Ollama custom Runtime, create a data science project, setup storage, create a workbench, and finally serve the Ollama Framework, utilizing the Single Model Serving Platform to deliver our model to our Notebook Application.


In section 2 we will explore using the Jupyter Notebook from our workbench, infere data from the Mistral 7B LLM. While less technical than previous section of this hands on course, there are some steps download the Mistral Model, updating our notebook with inference endpoint, and evaluating our Models performance.
In section 2 we will explore using the Jupyter Notebook from our workbench to infere data from the Mistral 7B LLM. While less technical than previous section of this hands-on course, there are some steps to download the Mistral Model, update our notebook with inference endpoint, and evaluate our Models performance.

Let's get started ---
Let's get started!
Loading

0 comments on commit 58f5e11

Please sign in to comment.