Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Index.add_documents compatible with other list-like types #1

Open
AlexAltea opened this issue Feb 11, 2023 · 3 comments
Open

Make Index.add_documents compatible with other list-like types #1

AlexAltea opened this issue Feb 11, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@AlexAltea
Copy link
Owner

AlexAltea commented Feb 11, 2023

E.g. tuple, dict_values.

Otherwise there might be performance loss during to needless conversions.

@AlexAltea AlexAltea added the enhancement New feature or request label Feb 11, 2023
@mardix
Copy link

mardix commented Feb 11, 2023

Couldn't the tuple or the other list types be converted to list ([{...}, ...]) before adding the documents to Index.add_documents?

@AlexAltea
Copy link
Owner Author

Yes, you can always do that, but it might have some performance impact. Imagine you want to index a million documents which happen to be a tuple, or dict_values.

Doing list(...) causes everything to be copied from the original tuple into a new list, and then processed by Milli.

What I'm trying to do is have Milli's DocumentBatchBuilder receive the documents straight from the tuple/dict_values.

@mardix
Copy link

mardix commented Feb 11, 2023

ok. make sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants