Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a QueuedFile object #434

Merged
merged 2 commits into from
Dec 10, 2024
Merged

Add a QueuedFile object #434

merged 2 commits into from
Dec 10, 2024

Conversation

msm-cert
Copy link
Member

Your checklist for this pull request

  • I've read the contributing guideline.
  • I've tested my changes by building and running mquery, and testing changed functionality (if applicable)
  • I've added automated tests for my change (if applicable, optional)
  • I've updated documentation to reflect my change (if applicable)

A smaller version of #432.

This PR creates a "core" (database object) of the index queue that will be managed by ursadb. This will let us queue files for indexing, and index them with available plugins.

This PR doesn't add the "indexer" worker yet, since it's nontrivial and needs some careful design.

@msm-cert msm-cert requested a review from michalkrzem October 30, 2024 22:00
@msm-cert
Copy link
Member Author

msm-cert commented Oct 30, 2024

On the other hand, this idea needs some design. It's possible that the feature is too powerful and I should just special-case indexing s3 and local disk (because that's what everyone uses).

Finishing #432 correctly will probably require ursadb changes.

@msm-cert msm-cert force-pushed the feat/add-queuedfile branch from 6f76c74 to de5d8eb Compare November 18, 2024 22:23
@msm-cert msm-cert merged commit d34a882 into master Dec 10, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants