Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to OpenAI v1.0 #3

Merged
merged 69 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
920219c
Remove empty evadb.db file.
xzdandy Oct 4, 2023
2b5ba6a
Move github test into long intergration test so they are run on the
xzdandy Oct 4, 2023
45af91e
Clean up test_relational_api.py
xzdandy Oct 4, 2023
9d2227e
Update link in the github data source documentation
xzdandy Oct 4, 2023
c322ada
Fix doc links
xzdandy Oct 4, 2023
89e4888
Bump Version to v0.3.9+dev (#1304)
github-actions[bot] Oct 19, 2023
e19f13d
Fix current issues with forecasting (#1283)
americast Oct 19, 2023
4640d8f
Configuration Manager Redesign (#1272)
gaurav274 Oct 19, 2023
a6fdd6a
Fix: minor typo (#1307)
gaurav274 Oct 19, 2023
c637a71
SnowFlake Integration for EvaDB (#1289)
kslohith Oct 20, 2023
0f2d1d4
add documentation
Chitti-Ankith Oct 22, 2023
e026460
Starting the changes for XGBoost classification integration.
Oct 19, 2023
0ab5ff8
Adding doc changes
Oct 19, 2023
3c25029
Added note
Chitti-Ankith Oct 23, 2023
d25b228
Minor Fix
Chitti-Ankith Oct 23, 2023
ce64231
Merge branch 'db_doc_note' of github.com:Chitti-Ankith/eva into Chitt…
jarulraj Oct 24, 2023
81ab7a3
Make the config related changes
Oct 24, 2023
d340548
checkpoint
jarulraj Oct 24, 2023
c6970bd
Skip long running model train test.
xzdandy Oct 24, 2023
d8f4a1b
Linter
xzdandy Oct 24, 2023
3f0d829
Fix typo
xzdandy Oct 24, 2023
703dc94
Starting the changes for XGBoost classification integration. (#1305)
jineetd Oct 24, 2023
6af77d2
Add roadmap page to toc
xzdandy Oct 25, 2023
482097e
Merge branch 'staging' into roadmap
xzdandy Oct 25, 2023
f420faa
Convert nested join in Vector Queries to Pandas Merge. (#1298)
Chitti-Ankith Oct 26, 2023
5857740
checkpoint
xzdandy Oct 26, 2023
3061552
docs: added survey link
jarulraj Oct 26, 2023
af696d6
refactor: code refactor (#1319)
gaurav274 Oct 27, 2023
71b9aca
Add Milvus integration for vector create and search (#1269)
RichardZhangRZ Oct 27, 2023
7c44d98
Adding colab notebook for XGBoost Regression and Classification (#1316)
jineetd Oct 27, 2023
60f4b68
Added docs for SET and SHOW CONFIG query (#1323)
hershd23 Oct 27, 2023
9a206b8
REST API Documentation (#1321)
imnidhi Oct 28, 2023
fae7001
String Helper functions in EvaDB (#1322)
Chitti-Ankith Oct 28, 2023
0513284
Merge branch 'staging' into fix_staging
xzdandy Oct 28, 2023
679b193
Skip 19-employee-classification-prediction.ipynb
xzdandy Oct 28, 2023
dc77d65
Add data sources to README
xzdandy Oct 28, 2023
51bc222
Fix list
xzdandy Oct 28, 2023
fd92bba
Use table for side by side display
xzdandy Oct 28, 2023
ed7809e
Add link to make it looks better
xzdandy Oct 28, 2023
40de531
Add all models supported
xzdandy Oct 28, 2023
f8c9c5f
Add etc
xzdandy Oct 28, 2023
339e7e0
Add link
xzdandy Oct 28, 2023
4ca964e
Add details on optimization
xzdandy Oct 28, 2023
c57e36e
Add automl framework
xzdandy Oct 28, 2023
affa3cf
Move around links
xzdandy Oct 28, 2023
919cd5b
Merge remote-tracking branch 'origin/roadmap' into staging
jarulraj Oct 30, 2023
8dc9283
checkpoint
jarulraj Oct 30, 2023
2ed8448
checkpoint
jarulraj Oct 30, 2023
8eb9e6b
Merge remote-tracking branch 'origin/fix_staging' into staging
jarulraj Oct 30, 2023
1c58afe
docs: Update README.md
jarulraj Oct 30, 2023
52ff444
docs: Update README.md
jarulraj Oct 30, 2023
f409057
Add the validation score and training time for create_function in XGB…
jineetd Nov 2, 2023
21aee56
Add test_eva_db to gitignore (#1336)
LordDarkula Nov 3, 2023
f56ef82
Fix #1333 dependency and CMD in DockerFile (#1335)
kumeagidi Nov 3, 2023
c296318
CREATE INDEX IF NOT EXISTS is broken. (#1337)
Chitti-Ankith Nov 3, 2023
6929103
Support semicolon and escaped strings in lark (#1339)
pchunduri6 Nov 6, 2023
bd344ed
feat: third party app support in EVADB (#1033)
kaushikravichandran Nov 6, 2023
35b772b
[WIP] Improving error handling messages for Custom Functions (#1330)
hershd23 Nov 6, 2023
64219f1
logging an error message for invalid files while loading (#1334)
rohithmulumudy Nov 6, 2023
bb45db4
checkpoint
jarulraj Nov 7, 2023
7a49bec
Verified that issue #1067 is resolved and added documentation for loa…
kslohith Nov 7, 2023
1c78b22
Add train scores for ludwig in the create function handler. (#1342)
jineetd Nov 10, 2023
995f7fe
Make Logic Operators Case Insensitve (#1352)
xzdandy Nov 14, 2023
1fbb74f
Job scheduler implementation (#1308)
dungnmaster Nov 14, 2023
2575f4f
Fix python3.8 failing testcases due to type hint (#1364)
xzdandy Nov 16, 2023
69b39b8
Add feedback for forecasting (#1258)
americast Nov 16, 2023
0c25a44
Adding changes for Flaml Sklearn integration (#1361)
jineetd Nov 17, 2023
5aaa447
Migrate ChatGPT function to openai v1.0 (#1368)
pchunduri6 Nov 17, 2023
8a8fc46
v0.3.9 - new release (#1367)
gaurav274 Nov 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ jobs:
else
pip install ".[dev,pinecone,chromadb]" # ray < 2.5.0 does not work with python 3.11 ray-project/ray#33864
fi
python -c "import yaml;f = open('evadb/evadb.yml', 'r+');config_obj = yaml.load(f, Loader=yaml.FullLoader);config_obj['experimental']['ray'] = True;f.seek(0);f.write(yaml.dump(config_obj));f.truncate();"
python -c "import evadb;cur=evadb.connect().cursor();cur.query('SET ray=True;')"
else
if [ $PY_VERSION != "3.11" ]; then
pip install ".[dev,ludwig,qdrant,pinecone,chromadb]"
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ env.bak/
venv.bak/
env38/
env_eva/
test_eva_db/

# Spyder project settings
.spyderproject
Expand Down
31 changes: 31 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### [Deprecated]
### [Removed]

## [0.3.8] - 2023-10-18

* PR #1303: v0.3.8 - new release
* PR #1302: Reenable batch for release
* PR #1301: Add Documentation for UDF Unit Testing and Mocking
* PR #1232: Starting the change for XGBoost integration into EVADb.
* PR #1294: fix: improve testcase
* PR #1293: fix: make the table/function catalog insert operation atomic
* PR #1295: feat: add support for show databases
* PR #1296: feat: function_metadata supports boolean and float
* PR #1290: fix: text_summarization uses drop udf
* PR #1240: Add stable diffusion integration
* PR #1285: Update custom-ai-function.rst
* PR #1234: Added basic functionalities of REST apis
* PR #1281: Clickhouse integration
* PR #1273: Update custom-ai-function.rst
* PR #1274: Fix Notebook and Ray testcases at staging
* PR #1264: SHOW command for retrieveing configurations
* PR #1270: fix: Catalog init introduces significant overhead
* PR #1267: Improve the error message when there is a typo in the column name in the query.
* PR #1261: Remove dimensions from `TEXT` and `FLOAT`
* PR #1256: Remove table names from column names for `df
* PR #1253: Collection of fixes for the staging branch
* PR #1246: feat: insertion update index
* PR #1245: Documentation on vector stores + vector benchmark
* PR #1244: feat: create index from projection
* PR #1233: GitHub Data Source Integration
* PR #1115: Add support for Neuralforecast
* PR #1241: Bump Version to v0.3.8+dev
* PR #1239: release 0.3.7

## [0.3.7] - 2023-09-30

* PR #1239: release 0.3.7
Expand Down
158 changes: 140 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,17 +42,6 @@
</a>
</p>

<p align="center"><b>Share EvaDB</b></p>

<p align="center">

<a href="https://twitter.com/intent/tweet?text=Check%20out%20EvaDB!%20Bring%20AI%20inside%20your%20database%20system%20and%20build%20AI-powered%20apps.%20&url=https://github.com/georgia-tech-db/evadb&hashtags=EvaDB,AI,Database" target="blank">
<img src="https://img.shields.io/twitter/follow/_superAGI?label=Twitter&style=social" alt="Follow _superAGI"/></a>
<a href="https://t.me/share/url?text=Check%20out%20EvaDB!%20Bring%20AI%20inside%20your%20database%20and%20build%20AI-powered%20apps.%20&url=https://github.com/georgia-tech-db/evadb" target="_blank"><img src="https://img.shields.io/twitter/url?label=Telegram&logo=Telegram&style=social&url=https://github.com/georgia-tech-db/evadb" alt="Share on Telegram"/></a>
<a href="https://api.whatsapp.com/send?text=Check%20out%20EvaDB!%20Bring%20AI%20inside%20your%20database%20and%20build%20AI-powered%20apps.%20https://github.com/georgia-tech-db/evadb"><img src="https://img.shields.io/twitter/url?label=whatsapp&logo=whatsapp&style=social&url=https://github.com/georgia-tech-db/evadb" /></a> <a href="https://www.reddit.com/submit?url=https://github.com/georgia-tech-db/evadb&title=Check%20out%20EvaDB!%20Bring%20AI%20inside%20your%20database%20and%20build%20AI-powered%20apps." target="blank">
<img src="https://img.shields.io/twitter/url?label=Reddit&logo=Reddit&style=social&url=https://github.com/georgia-tech-db/evadb" alt="Share on Reddit"/>
</p>

<p align="center">
<a href="https://colab.research.google.com/github/georgia-tech-db/evadb/blob/master/tutorials/03-emotion-analysis.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Launch EvaDB on Colab"/>
Expand All @@ -67,15 +56,144 @@
<img alt="Downloads" src="https://static.pepy.tech/badge/evadb"/>
</a> -->
<br />
<a href="https://gitpod.io/#https://github.com/georgia-tech-db/evadb" target="_blank"><img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Open in Gitpod"></a>
</p>

EvaDB enables software developers to build AI apps in a few lines of code. Its powerful SQL API simplifies AI app development for both structured and unstructured data. EvaDB's benefits include:
- 🔮 Easy to connect EvaDB with your SQL database system and build AI-powered apps with SQL queries
- 🤝 Query your data with a pre-trained AI model from Hugging Face, OpenAI, YOLO, PyTorch, and other AI frameworks
- ⚡️ Faster queries thanks to AI-centric query optimization
- 💰 Save money spent on running models by efficient CPU/GPU use
- 🔧 Fine-tune your AI models to achieve better results
<details>
<summary> 🔮 Easy to <a href="https://evadb.readthedocs.io/en/latest/source/overview/connect-to-data-sources.html">connect the EvaDB query engine with your data sources</a>, such as PostgreSQL or S3 buckets, and build AI-powered apps with SQL queries. </summary>
<br/>
<table>
<tr>
<th>Structured Data Sources</th>
<th>Unstructured Data Sources</th>
<th>Application Data Sources</th>
</tr>
<tr>
<td>

- PostgreSQL
- SQLite
- MySQL
- MariaDB
- Clickhouse
- Snowflake

</td>
<td>

- Local filesystem
- AWS S3 bucket

</td>
<td>

- Github

</td>
</tr>
</table>

More details on the supported data sources is [available here](https://evadb.readthedocs.io/en/latest/source/reference/databases/index.html).

</details>

<details>
<summary> 🤝 <a href="https://evadb.readthedocs.io/en/latest/source/overview/ai-queries.html">Query your connected data with a pre-trained AI model</a> from Hugging Face, OpenAI, YOLO, Stable Diffusion, etc. </summary>
<br/>
<table>
<tr>
<th>Hugging Face</th>
<th>OpenAI</th>
<th>YOLO</th>
</tr>
<tr>
<td>

- Audio Classification
- Automatic Speech Recognition
- Text Classification
- Summarization
- Text2Text Generation
- Text Generation
- Image Classification
- Image Segmentation
- Image-to-Text
- Object Detection
- Depth Estimation

</td>
<td>

- gpt-4
- gpt-4-0314
- gpt-4-32k
- gpt-4-32k-0314
- gpt-3.5-turbo
- gpt-3.5-turbo-0301

</td>
<td>

- yolov8n.pt
- yolov8s.pt
- yolov8m.pt
- yolov8l.pt
- yolov8x.pt

</td>
</tr>
</table>

More details on the supported AI models is [available here](https://evadb.readthedocs.io/en/latest/source/reference/ai/index.html)
</details>

<details>
<summary> 🔧 Create or fine-tune AI models for regression, classification, and time series forecasting.</summary>
<br/>
<table>
<tr>
<th>Regression</th>
<th>Classification</th>
<th>Time Series Forecasting</th>
</tr>
<tr>
<td>

- Ludwig
- Sklearn
- Xgboost

</td>
<td>

- Ludwig
- Xboost

</td>
<td>

- Statsforecast
- Neuralforecast

</td>
</tr>
</table>

More details on the supported AutoML frameworks is [available here](https://evadb.readthedocs.io/en/latest/source/reference/ai/index.html).
</details>

<details>
<summary> 💰 Faster AI queries thanks to AI-centric query optimizations such as caching, batching, and parallel processing. </summary>
<br/>

- Function result caching helps reuse results of expensive AI function calls.
- LLM batching reduces token usage and dollars spent on LLM calls.
- Parallel query processing saves money and time spent on running AI models by better utilizing CPUs and/or GPUs.
- Query predicate re-ordering and predicate push-down accelerates queries over both structured and unstructured data.

More details on the optimizations in EvaDB is [available here](https://evadb.readthedocs.io/en/latest/source/reference/optimizations.html).
</details>
<br/>

👋 Hey! If you're excited about our vision of bringing AI inside database systems, show some ❤️ by:
<ul>
Expand All @@ -85,6 +203,8 @@ EvaDB enables software developers to build AI apps in a few lines of code. Its p
<li> 📝 following us on <a href="https://medium.com/evadb-blog">Medium</a>
</ul>

We would love to learn about your AI app. Please complete this 1-minute form: https://v0fbgcue0cm.typeform.com/to/BZHZWeZm

## Quick Links

- [Quick Links](#quick-links)
Expand Down Expand Up @@ -116,7 +236,7 @@ Our target audience is software developers who may not necessarily have a backgr

<details>
<ul>
<li>Connect EvaDB to your SQL and vector database systems with the <a href="https://evadb.readthedocs.io/en/stable/source/reference/databases/postgres.html">`CREATE DATABASE`</a> and <a href="https://evadb.readthedocs.io/en/stable/source/reference/evaql/create.html#create-index">`CREATE INDEX`</a> statements.</li>
<li>Connect EvaDB to your SQL and vector database systems with the <a href="https://evadb.readthedocs.io/en/stable/source/reference/databases/postgres.html">`CREATE DATABASE`</a> and <a href="https://evadb.readthedocs.io/en/stable/source/reference/evaql/create_index.html">`CREATE INDEX`</a> statements.</li>
<li>Write SQL queries with AI functions to get inference results:</li>
<ul>
<li>Pick a pre-trained AI model from Hugging Face, Open AI, Ultralytics, PyTorch, and built-in AI frameworks for generative AI, NLP, and vision applications;</li>
Expand Down Expand Up @@ -210,6 +330,8 @@ EvaDB's AI-centric query optimizer takes a query as input and generates a query

## Community and Support

We would love to learn about your AI app. Please complete this 1-minute form: https://v0fbgcue0cm.typeform.com/to/BZHZWeZm

<!--<p>
<a href="https://evadb.ai/community">
<img width="70%" src="https://raw.githubusercontent.com/georgia-tech-db/evadb/master/docs/images/evadb/evadb-slack.png" alt="EvaDB Slack Channel">
Expand Down
4 changes: 2 additions & 2 deletions apps/pandas_qa/pandas_qa.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ def receive_user_input() -> Dict:

# get OpenAI key if needed
try:
api_key = os.environ["OPENAI_KEY"]
api_key = os.environ["OPENAI_API_KEY"]
except KeyError:
api_key = str(input("🔑 Enter your OpenAI key: "))
os.environ["OPENAI_KEY"] = api_key
os.environ["OPENAI_API_KEY"] = api_key

return user_input

Expand Down
4 changes: 2 additions & 2 deletions apps/youtube_qa/youtube_qa.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,10 @@ def receive_user_input() -> Dict:

# get OpenAI key if needed
try:
api_key = os.environ["OPENAI_KEY"]
api_key = os.environ["OPENAI_API_KEY"]
except KeyError:
api_key = str(input("🔑 Enter your OpenAI key: "))
os.environ["OPENAI_KEY"] = api_key
os.environ["OPENAI_API_KEY"] = api_key

return user_input

Expand Down
Loading