Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Llama v2 Model to Textbase with Replicate API, created example bot using Llama, and updated Docs accordingly #86

Open
wants to merge 44 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
6e3f2d3
Added support for google palm API
jahanvir Sep 2, 2023
3072aea
added palm API key in .env.example
jahanvir Sep 2, 2023
9797c82
added Google Palm AI example
jahanvir Sep 2, 2023
b7debab
Added Meta's Llama 2 API to textbase, updated docs accordingly
Ayan-Banerjee-918 Sep 3, 2023
dd8e110
Updated Readme and docs
Ayan-Banerjee-918 Sep 3, 2023
428b3f6
Removed replicate API key
Ayan-Banerjee-918 Sep 3, 2023
bc28bcf
Updated docs
Ayan-Banerjee-918 Sep 3, 2023
3e2a688
Make path absolute instead of cwd
sammyCofactory Sep 3, 2023
f6ba210
Merge pull request #88 from cofactoryai/make-path-absolute
sammyCofactory Sep 3, 2023
cb565ae
Merge branch 'main' of https://github.com/cofactoryai/textbase into c…
jahanvir Sep 3, 2023
6ddaec7
merged changes
jahanvir Sep 3, 2023
1ff289d
Merge branch 'cofactoryai-main'
jahanvir Sep 3, 2023
f6d73d9
add mac zip software to docs (#96)
kaustubh-cf Sep 3, 2023
a88e18e
added script to zip the requirement and main file for deploy.
jahanvir Sep 3, 2023
5cb2eb4
Delete zip.py
jahanvir Sep 3, 2023
056edb1
added zip function in textbase cli tool
jahanvir Sep 3, 2023
877fd15
Update llama-bot.md
Ayan-Banerjee-918 Sep 3, 2023
9e431ae
Update main.py
Ayan-Banerjee-918 Sep 3, 2023
2caa2a0
Custom port functionality (#107)
thenakulgupta-backup Sep 4, 2023
8d80bdc
Update README: Fix Broken Link (#142)
Haleshot Sep 5, 2023
658bf00
Fix typo in README.md (#141)
eltociear Sep 5, 2023
52506a9
Error handling (#104)
thenakulgupta-backup Sep 6, 2023
854102c
Added file check such that deploy.zip won't be created if either of m…
jahanvir Sep 6, 2023
c6e4b58
Merge branch 'main' of https://github.com/cofactoryai/textbase into p…
jahanvir Sep 6, 2023
995812c
resolved merge conflicts
jahanvir Sep 6, 2023
b0bf5fb
removed unused code.
jahanvir Sep 6, 2023
b0e0323
Merge branch 'main' of https://github.com/cofactoryai/textbase
jahanvir Sep 6, 2023
28316fa
added documentation for paLM AI
jahanvir Sep 6, 2023
891bdb6
Replaced the extract_user_content_values with extract_content_values
jahanvir Sep 6, 2023
c6c7d73
Merge pull request #70 from jahanvir/main
sammyCofactory Sep 7, 2023
e54e0ad
modified compress command to take directory path, check if main.py an…
jahanvir Sep 7, 2023
ffaab83
added the condition to avoid deploy.zip being created inside deploy.zip
jahanvir Sep 7, 2023
58fa7f3
piping out logs in the cli (#143)
kaustubh-cf Sep 7, 2023
f15c0a5
Merge branch 'main' into package-zip
sammyCofactory Sep 8, 2023
ba237aa
Merge pull request #102 from jahanvir/package-zip
sammyCofactory Sep 8, 2023
2ac3f35
Updated repo to latest build
Ayan-Banerjee-918 Sep 8, 2023
a021720
Added Meta's Llama 2 API to textbase, updated docs accordingly
Ayan-Banerjee-918 Sep 3, 2023
d80de50
Updated Readme and docs
Ayan-Banerjee-918 Sep 3, 2023
71171fc
Removed replicate API key
Ayan-Banerjee-918 Sep 3, 2023
03fda2c
Updated docs
Ayan-Banerjee-918 Sep 3, 2023
1d54f5b
Update llama-bot.md
Ayan-Banerjee-918 Sep 3, 2023
8de7761
Update main.py
Ayan-Banerjee-918 Sep 3, 2023
e1d3ae9
Updated repo to latest build
Ayan-Banerjee-918 Sep 8, 2023
b290921
Merge branch 'main' of https://github.com/Ayan-Banerjee-918/textbase
Ayan-Banerjee-918 Sep 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added Meta's Llama 2 API to textbase, updated docs accordingly
  • Loading branch information
Ayan-Banerjee-918 committed Sep 3, 2023
commit b7debabc06d9de2f1fbab84136180c5ea5ea9b0a
51 changes: 51 additions & 0 deletions docs/docs/examples/llama-bot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
sidebar_position: 3
---

# HuggingFace bot

This bot makes an API call to Meta's LLama 2 API and processes the user input. It uses Llama-2-7B model.

```py
from textbase import bot, Message
from textbase.models import Llama
from typing import List

# Load your Replicate API key
Llama.replicate_api_key = "r8_8xsgb2mpX8GB9Iz9BduNR3vwc0vs7Ei1zFi3u"
Ayan-Banerjee-918 marked this conversation as resolved.
Show resolved Hide resolved

# Default Prompt for Llama7b. States how the AI Model is supposed to act like
SYSTEM_PROMPT = """\
You are a helpful assistant"""

@bot()
def on_message(message_history: List[Message], state: dict = None):

# Generate Llama7b responses
bot_response = Llama.generate(
system_prompt=SYSTEM_PROMPT,
message_history=message_history, # Assuming history is the list of user messages
)

response = {
"data": {
"messages": [
{
"data_type": "STRING",
"value": bot_response
}
],
"state": state
},
"errors": [
{
"message": ""
}
]
}

return {
"status_code": 200,
"response": response
}
```
41 changes: 41 additions & 0 deletions examples/llama-bot/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
from textbase import bot, Message
from textbase.models import Llama
from typing import List

# Load your Replicate API key
Llama.replicate_api_key = "r8_8xsgb2mpX8GB9Iz9BduNR3vwc0vs7Ei1zFi3u"
Ayan-Banerjee-918 marked this conversation as resolved.
Show resolved Hide resolved

# Prompt for Llama7b. Llama gets offensive with larger complicated system prompts. This works just fine
SYSTEM_PROMPT = """\
You are a helpful assistant"""

@bot()
def on_message(message_history: List[Message], state: dict = None):

# Generate Llama7b response by default
bot_response = Llama.generate(
system_prompt=SYSTEM_PROMPT,
message_history=message_history, # Assuming history is the list of user messages
)

response = {
"data": {
"messages": [
{
"data_type": "STRING",
"value": bot_response
}
],
"state": state
},
"errors": [
{
"message": ""
}
]
}

return {
"status_code": 200,
"response": response
}
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ tabulate = "^0.9.0"
functions-framework = "^3.4.0"
yaspin = "^3.0.0"
pydantic = "^2.3.0"
replicate = "^0.11.0"

[build-system]
requires = ["poetry-core"]
Expand Down
38 changes: 37 additions & 1 deletion textbase/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import time
import typing
import traceback
import replicate

from textbase import Message

Expand Down Expand Up @@ -143,4 +144,39 @@ def generate(
data = json.loads(response.text) # parse the JSON data into a dictionary
message = data['message']

return message
return message

class Llama:

@classmethod
def generate(
cls,
system_prompt: str,
message_history: list[Message],
model = "a16z-infra/llama7b-v2-chat:4f0a4744c7295c024a1de15e1a63c880d3da035fa1f49bfd344fe076074c8eea",
temperature = 0.81, #seems to give the best responses
top_p = 0.95,
max_length = 3008 #same for this
):
try:
assert cls.replicate_api_key is not None, "Replicate API key is not set."
client=replicate.Client(api_token=cls.replicate_api_key) #creating the client for the model

past_conversation=""

for message in message_history:
if message["role"] == "user":
past_conversation+="[INST] ".join(extract_content_values(message))+" [/INST]\n" #Llama2 paper mentions tokenization of user messages with INST delimiter, otherwise the responses get ugly with time.
else:
past_conversation+=" ".join(extract_content_values(message))+"\n" #response messages dont need any delimiters

response=client.run(model,input={"prompt": past_conversation,"system_prompt": system_prompt,"temperature":temperature,"top_p":top_p,"max_length":max_length,"repetition_penalty":1})

resp=""
for word in response:
resp+=word

return resp

except Exception:
print(f"An exception occured while using this model, please try using another model.\nException: {traceback.format_exc()}.")