Skip to content

Commit

Permalink
remove azure embedding option
Browse files Browse the repository at this point in the history
  • Loading branch information
JamesHWade committed Jan 21, 2024
1 parent f337b32 commit ea30c57
Show file tree
Hide file tree
Showing 9 changed files with 354 additions and 9 deletions.
12 changes: 3 additions & 9 deletions R/harvest-docs.R
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ recursive_hyperlinks <- function(local_domain,
#' @param pkg_version Package version number
#' @param pkg_name Package name
#' @param service The service to use for scraping. Default is "openai". Options
#' are "openai", "local", and "azure".
#' are "openai" and "local".
#'
#' @return NULL. The resulting tibble is saved into a parquet file.
#'
Expand All @@ -129,7 +129,7 @@ crawl <- function(url,
pkg_version = NULL,
pkg_name = NULL,
service = "openai") {
rlang::arg_match(service, c("openai", "local", "azure"))
rlang::arg_match(service, c("openai", "local"))

Check warning on line 132 in R/harvest-docs.R

View check run for this annotation

Codecov / codecov/patch

R/harvest-docs.R#L132

Added line #L132 was not covered by tests
parsed_url <- urltools::url_parse(url)
local_domain <- parsed_url$domain
url_path <- parsed_url$path
Expand Down Expand Up @@ -197,13 +197,7 @@ crawl <- function(url,
sink = scraped_text_file
)
if (index_create) {
if (service == "azure") {
create_index_azure(local_domain_name,
overwrite = overwrite,
pkg_version = pkg_version,
pkg_name = pkg_name
)
} else if (service == "openai") {
if (service == "openai") {
create_index(local_domain_name,

Check warning on line 201 in R/harvest-docs.R

View check run for this annotation

Codecov / codecov/patch

R/harvest-docs.R#L200-L201

Added lines #L200 - L201 were not covered by tests
overwrite = overwrite,
pkg_version = pkg_version,
Expand Down
30 changes: 30 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,33 @@ template:
bootswatch: litera
development:
mode: auto

navbar:
structure:
left: [addins, services, reference, news]
right: [search, github]
components:
# addins:
# text: Addins
# menu:
# - text: Chat
# href: articles/chat.html
# - text: Chat in Source
# href: articles/chat-in-source.html
services:
text: Services
menu:
- text: OpenAI
href: articles/openai.html
- text: Local Models with Ollama
href: articles/ollama.html
- text: HuggingFace
href: articles/huggingface.html
- text: Anthropic
href: articles/anthropic.html
- text: Azure OpenAI
href: articles/azure.html
- text: Google AI Studio
href: articles/google.html
- text: Perplexity
href: articles/perplexity.html
45 changes: 45 additions & 0 deletions vignettes/anthropic.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: "Anthropic API Service"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Anthropic API Service}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

### Creating an Anthropic Account

- Go to the [Anthropic website](https://www.anthropic.com/) and sign up.
- Verify your account as instructed.

### Creating an Anthropic API Key

- Log into your Anthropic account and navigate to the API section.
- Create an API key following their guidelines. Check Anthropic's API documentation for more details.

### Setting the Anthropic API Key in .Renviron

To modify the `.Renviron` file:

```{r}
#| eval: false
require(usethis)
edit_r_environ()
```

For a persistent setting, add the following line to `.Renviron`, replacing `"<APIKEY>"` with your actual Anthropic API key:

```bash
ANTHROPIC_API_KEY="<APIKEY>"
```

Save the file and restart your R session for the changes to take effect.

**Caution:** Ensure `.Renviron` is not exposed if using version control systems.
47 changes: 47 additions & 0 deletions vignettes/azure.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: "Azure OpenAI API Service"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Azure OpenAI API Service}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

## Azure OpenAI Service

To configure gptstudio to work using Azure OpenAI service, you need to provide some configuration details in your .Renviron file. Specifically, gptstudio looks for five environment variables:

- AZURE_OPENAI_TASK
- AZURE_OPENAI_ENDPOINT
- AZURE_OPENAI_DEPLOYMENT_NAME
- AZURE_OPENAI_KEY
- AZURE_OPENAI_API_VERSION

Here's how you can add these details to your .Renviron file:

1. Locate your .Renviron file with `usethis::edit_r_environ()`.
2. Add environment variable details: Add a new line for each variable you need to set in the following format: VARIABLE_NAME="YOUR_VALUE". Replace VARIABLE_NAME with the name of the environment variable and YOUR_VALUE with the actual value that you want to set. For example, to set the API key you would have a line like this:

```bash
AZURE_OPENAI_KEY="your_actual_key_goes_here"
```
You need to do this for each of the environment variables expected by the function. Your .Renviron file should look something like this:

```bash
AZURE_OPENAI_TASK="your_task_code"
AZURE_OPENAI_ENDPOINT="your_endpoint_url"
AZURE_OPENAI_DEPLOYMENT_NAME="your_deployment_name"
AZURE_OPENAI_KEY="your_api_key"
AZURE_OPENAI_API_VERSION="your_api_version"
```

3. Save and Close .Renviron: After adding your environment variables, save your .Renviron file and close it. You will need to restart your R session to make sure the new environment variables are loaded properly.

Remember to replace your_task_code, your_endpoint_url, your_deployment_name, your_api_key, and your_api_version with your actual Azure OpenAI details. You can retrieve these details from your Azure OpenAI service account. For more information about Azure OpenAI configuration, refer to the [Microsoft quickstart guide](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quickstart?tabs=command-line&pivots=rest-api).
45 changes: 45 additions & 0 deletions vignettes/google.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: "Google's AI Studio API Service"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Google's AI Studio API Service}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

### Creating an Account in Google's AI Studio

- Visit [Google's AI Studio website](https://makersuite.google.com) and sign up.
- Complete the verification process.

### Accessing Google PALM API

- In your Google Cloud Console, enable the PALM API.
- Create an API key as per the instructions in [Google's API documentation](https://ai.google.dev/docs).

### Setting the Google AI Studio API Key in .Renviron

To modify the `.Renviron` file:

```{r}
#| eval: false
require(usethis)
edit_r_environ()
```

For a persistent setting, add the following line to `.Renviron`, replacing `"<APIKEY>"` with your actual Google PALM API key:

```bash
PALM_API_KEY="<APIKEY>"
```

Save the file and restart your R session for the changes to take effect.

**Caution:** Be careful not to expose `.Renviron` in public repositories or version control systems to protect your API key.
45 changes: 45 additions & 0 deletions vignettes/huggingface.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: "HuggingFace API Service"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{HuggingFace API Service}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

### Creating a HuggingFace Account

- Visit [HuggingFace's website](https://huggingface.co/) and sign up for an account.
- Complete the account verification process.

### Creating a HuggingFace API Key

- After logging in, go to your account settings.
- Find the section for API keys and create a new one. Detailed guidance is available in HuggingFace's API documentation.

### Setting the HuggingFace API Key in .Renviron

To modify the `.Renviron` file:

```{r}
#| eval: false
require(usethis)
edit_r_environ()
```

For a persistent setting, add the following line to `.Renviron`, replacing `"<APIKEY>"` with your actual HuggingFace API key:

```bash
HF_API_KEY="<APIKEY>"
```

Save the file and restart your R session for the changes to take effect.

**Caution:** Remember to include `.Renviron` in your `.gitignore` file to prevent exposing your API key, especially if using version control systems like GitHub or GitLab.
46 changes: 46 additions & 0 deletions vignettes/no-build/ollama.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: "Local Models with Ollama"
output: rmarkdown::html_vignette
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

The [ollama](<https://github.com/jmorganca/ollama>) service allows you to run open source LLMs locally, providing a command line interface and an API. By wrapping the later, we can use it within our chat app.

You can run ollama in any platform as a docker container. The following code runs the CPU-only version:

```bash
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
```

This code:

1. pulls the latest ollama image from the ollama hub (`ollama/ollama`)
2. exposes the ollama API in `http://localhost:11434` (`-p 11434:11434`)
3. sets up the ollama volume, to be used in the "/root/.ollama" path inside the container. this will allow you to update the container later without losing your already downloaded models. (`-v ollama:/root/.ollama`)
4. assigns the name "ollama" to the container (`--name ollama`)
5. runs the container in detached mode (`docker run -d`)

You can see more docker options in the [official blog post](https://ollama.ai/blog/ollama-is-now-available-as-an-official-docker-image).

Before using the service, you need to pull a model. Run the following code inside your container to pull llama2:

```bash
ollama pull llama2
```

Check the [ollama library](https://ollama.ai/library) to see more models. For more advanced install options, check the [official documentation](https://github.com/jmorganca/ollama).

By default, the chat addin will use `http://localhost:11434` to locate the ollama API. You can customize this by setting up the `OLLAMA_HOST` environmental variable with `usethis::edit_r_environ()`.


## An Example with Ollama

Here is a short video showing you how to get started with ollama. It assumes that you have already installed docker. See the docker [installation guide](https://docs.docker.com/get-docker/) for more information.

![](https://raw.githubusercontent.com/MichelNivard/gptstudio/main/media/gptstudio-ollama-example.gif){width=700px}
48 changes: 48 additions & 0 deletions vignettes/openai.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: "OpenAI API Service"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{OpenAI API Services}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```


## Creating an OpenAI Account

- Visit [OpenAI's website](https://openai.com/) and sign up for an account.
- Follow the instructions to verify your account.

## Creating an OpenAI API Key

- Once logged in, navigate to the API section in your account settings.
- Follow the instructions to create a new API key. More detailed steps can be found in OpenAI's [API documentation](https://beta.openai.com/docs/).

## Setting the OpenAI API Key in .Renviron

To modify the `.Renviron` file:

```{r}
#| eval: false
require(usethis)
edit_r_environ()
```

For a persistent setting, add the following line to `.Renviron`, replacing `"<APIKEY>"` with your actual API key:

```bash
OPENAI_API_KEY="<APIKEY>"
```

Save the file and restart your R session for the changes to take effect.

**Caution:** Ensure `.Renviron` is included in your `.gitignore` file to avoid exposing your API key with version control systems like GitHub or GitLab.

**Important Note:** OpenAI API requires valid payment details in your OpenAI account to function. This is a restriction imposed by OpenAI and is unrelated to this package.
45 changes: 45 additions & 0 deletions vignettes/perplexity.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: "Perplexity"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Perplexity}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

### Creating an Perplexity Account

- Go to the [Perplexity website](https://perplexity.ai/) and sign up.
- Verify your account as instructed.

### Creating an Perplexity API Key

- Log into your Perplexity account and navigate to the [API documentation](https://docs.perplexity.ai/).
- Create an API key following their guidelines. Check Perplexity's API documentation for more details.

### Setting the Perplexity API Key in .Renviron

To modify the `.Renviron` file:

```{r}
#| eval: false
require(usethis)
edit_r_environ()
```

For a persistent setting, add the following line to `.Renviron`, replacing `"<APIKEY>"` with your actual Perplexity API key:

```bash
PERPLEXITY_API_KEY="<APIKEY>"
```

Save the file and restart your R session for the changes to take effect.

**Caution:** Ensure `.Renviron` is not exposed if using version control systems.

0 comments on commit ea30c57

Please sign in to comment.