Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ML Annotations #679

Open
dlangenk opened this issue Oct 16, 2023 · 3 comments
Open

ML Annotations #679

dlangenk opened this issue Oct 16, 2023 · 3 comments

Comments

@dlangenk
Copy link
Member

We discussed to potentially add a confidence and flag for ML annotations in the backend. In the frontend we could display it like in one of the mockups below.

mockupa
mockupb

@mzur mzur moved this to Medium Priority in BIIGLE Roadmap Oct 16, 2023
@mzur
Copy link
Member

mzur commented Oct 16, 2023

A good start! The confidence values are too highlighted IMO. Maybe they can just be displayed as muted numbers on the right. In contrast to that, the "ML" label should be highlighted somehow (like in your mockups). But maybe just put the label in the same font size before the user name.

Alternative suggestions for the text: "AI" (I know...), "SCRIPT" or "API". ML is the most correct but AI will be most intuitive for the users. SCRIPT would be correct, too (2nd place after ML) but is probably too long. API is very generic.

@mzur
Copy link
Member

mzur commented Oct 16, 2023

Technical details for the implementation:

The image_annotations and video_annotations DB tables can get a new nullable ai_confidence column. When a new annotation is created, the ai_confidence attribute can optionally be specified. If the attribute is null, the annotation is not made by ML. If the attribute is anything else, it is made by ML. If the ML method does not support confidence values, ai_confidence can be set to 0. In this case the "ML" (or similar) label will be displayed in the UI but not the confidence number.

The existing confidence column will not be touched by the implementation. We need to decide in #182 what to do with the confidence. If we keep it, we can display it in the same way than the ML confidence just without the "ML" label.

@mzur
Copy link
Member

mzur commented Apr 19, 2024

The ai_confidence also needs to be added to the relevant reports (e.g. iFDO) and to the biigle/sync export/import (maybe together with #1021).

Also there should be a new option in #709 to specify if imported annotations were made by an AI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Medium Priority
Development

No branches or pull requests

2 participants