Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use database native index scan to accelerate table scan during similarity search #1222

Open
1 of 2 tasks
jiashenC opened this issue Sep 26, 2023 · 2 comments
Open
1 of 2 tasks

use database native index scan to accelerate table scan during similarity search #1222

jiashenC opened this issue Sep 26, 2023 · 2 comments
Labels
Optimizations Features/Bugs related to optimizations

Comments

@jiashenC
Copy link
Member

Search before asking

  • I have searched the EvaDB issues and found no similar feature requests.

Description

Currently, the table scan takes long time. If we switch from table scan to index scan, it can greatly accelerate the query speed.

Use case

No response

Are you willing to submit a PR?

  • Yes I'd like to help by submitting a PR!
@jiashenC jiashenC added the Optimizations Features/Bugs related to optimizations label Sep 26, 2023
@xzdandy xzdandy moved this to Ideation in EVA Public Roadmap ⚡🚀 Sep 26, 2023
@Chitti-Ankith
Copy link
Contributor

faiss

Attaching the profiled output for FAISS Vector Scan. Almost 80% of the time is spent in the post-processing after getting the results from the Vector Store.

@Chitti-Ankith
Copy link
Contributor

I have added a small optimization to convert the nested join to Pandas Merge here #1298. We can expand upon it further as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Optimizations Features/Bugs related to optimizations
Projects
EVA Public Roadmap ⚡🚀
Status: Ideation
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jiashenC @Chitti-Ankith