Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Which UI element allows searching for Audio Features? #167

Open
Lanceeeelot opened this issue Jun 18, 2024 · 1 comment
Open

Which UI element allows searching for Audio Features? #167

Lanceeeelot opened this issue Jun 18, 2024 · 1 comment

Comments

@Lanceeeelot
Copy link

How can you search for a video scene with a specific sound through the interface if this sound is described in a prompt, such as "explosion" or "traffic noise"?

@sauterl
Copy link
Contributor

sauterl commented Jun 26, 2024

If you use the default setup using Cineast as your retrieval engine (assumption based on your previous issue), then I am not aware of such a feature already implemented.
Do you have a specific model / your own model in mind?

Generally speaking, I'd suggest to introduce a new textual input field, similar to text-embedding / OCR / ASR:
You would add another category name - display label tuple:

https://github.com/vitrivr/vitrivr-ng/blob/master/src/app/shared/model/config/config.model.ts#L107-L109

would end up in something similar to:

 text: {
        categories: [['visualtextcoembedding', 'Description (VTE)'], ['ocr', 'OCR'],['text-to-audio-category', 'Sound']]
      },

with text-to-audio-category being a registered category to cineast.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants