diff --git a/modules/chapter3/images/curl_command.png b/modules/chapter3/images/curl_command.png new file mode 100644 index 0000000..8aca6c2 Binary files /dev/null and b/modules/chapter3/images/curl_command.png differ diff --git a/modules/chapter3/images/experiment.png b/modules/chapter3/images/experiment.png new file mode 100644 index 0000000..ecd1633 Binary files /dev/null and b/modules/chapter3/images/experiment.png differ diff --git a/modules/chapter3/images/london.png b/modules/chapter3/images/london.png new file mode 100644 index 0000000..81d2a10 Binary files /dev/null and b/modules/chapter3/images/london.png differ diff --git a/modules/chapter3/images/mistral_config.png b/modules/chapter3/images/mistral_config.png new file mode 100644 index 0000000..c126263 Binary files /dev/null and b/modules/chapter3/images/mistral_config.png differ diff --git a/modules/chapter3/images/paris.png b/modules/chapter3/images/paris.png new file mode 100644 index 0000000..07d9aed Binary files /dev/null and b/modules/chapter3/images/paris.png differ diff --git a/modules/chapter3/images/serverurl.png b/modules/chapter3/images/serverurl.png new file mode 100644 index 0000000..f00548f Binary files /dev/null and b/modules/chapter3/images/serverurl.png differ diff --git a/modules/chapter3/pages/section2.adoc b/modules/chapter3/pages/section2.adoc index 4c2d8dd..e58c07b 100644 --- a/modules/chapter3/pages/section2.adoc +++ b/modules/chapter3/pages/section2.adoc @@ -30,6 +30,8 @@ Return the Jupyter Notebook Environment: . Paste the inference endpoint into the Cell labeled interfence_server_url = *"replace with your own inference address"* +image::serverurl.png[width=800] + . We can now start executing the code in the cells, starting with the set the inference server URL cell. . Next we run the second cell: !pip install -q langchain==0.1.14 ; there is a notice to update pip, just ignore and continue. @@ -45,6 +47,8 @@ Before we continue, we need to perform the following additional step. As mention We will need to obtain the endpoint from the OpenShift AI model serving console. I usually just paste the text below into a cell in the Jupyter Notebook and paste the url in the code block from there. +image::mistral_config.png[width=640] + [source, yaml] ---- curl https://your-endpoint/api/pull \ @@ -61,6 +65,8 @@ curl https://your-endpoint/api/pull \ The message: *status: pulling manifest* should appear. This begins the model downloading process. +image::curl_command.png[width=800] + Once the download completes, the *status: success:* message appears. We can now return to the Jupyter Notebook Tab in the browser and proceed. === Create the Prompt @@ -81,12 +87,16 @@ In green text is the window, there is the setup message that is sent along with It takes approximately 12 seconds for the model to respond with the first word of the reply, and the final word is printed to the screen approximately 30 seconds after the request was started. +image::paris.png[width=800] + The responce answered the question in a well-considered and informated paragraph that is less than 100 words in length. === Second Input Notice that the Second input - "Is there a River" - does not specify where the location is that might have a River. Because the conversation history is passed with the second input, there is not need to specify any additional informaiton. +image::london.png[width=800] + The total time to first word took approximately 14 seconds this time, just a bit longer due the orginal information being sent. The time for the entire reponse to be printed to the screen just took over 4 seoncds. Overall our Model is performing well without a GPU and in a container limited to 4 cpus & 10Gb of memory. @@ -101,11 +111,13 @@ There is no change to memory setting, but go ahead and evalute where the second Add a few new cells to the Notebook. +image::experiment.png[width=800] + Experiment with clearing the memory statement, then asking the river question again. Or perhaps copy one of the input statements and add your own question for the model. Try not clearing the memory and asking a few questions. -You have successfully deployed a Large Language Model, now test the information that it has available and find out what is doesn't know. +**You have successfully deployed a Large Language Model, now test the information that it has available and find out what is doesn't know.** == Delete the Environment