Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plot_annotation by labels, instead of optimally separating segments #57

Open
janaab11 opened this issue Jun 9, 2021 · 2 comments
Open

Comments

@janaab11
Copy link

janaab11 commented Jun 9, 2021

The default implementation seems to separate overlapping segments into different lines (vertically), by parsing the start and end times of all segments, linearly from left to right. [source]

While this is pretty useful visually, there is another way to handle overlaps: plot differently-labelled segments on separate lines. In particular, this is useful if the different labels represent different audio environments and you want to visualize differences in your model's performance across environments/labels.

The actual implementation of this is pretty simple, and adds only three new lines of code:

  def plot_annotation(self, annotation: Annotation, ax=None, time=True, legend=True, separate_by="optimal"):

      if not self.crop:
          self.crop = annotation.get_timeline(copy=False).extent()

      cropped = annotation.crop(self.crop, mode='intersection')
      labels = cropped.labels()
      labels_dict = {label:i for i, label in enumerate(labels)}
      segments = [s for s, _ in cropped.itertracks()]

      ax = self.setup(ax=ax, time=time)

      for (segment, track, label), y in zip(
              cropped.itertracks(yield_label=True),
              self.get_y(segments)):
          if separate_by == "labels":
              y = 1. - 1. / (len(labels) + 1) * (1 + labels_dict.get(label))
          self.draw_segment(ax, segment, y, label=label)

Let me know if this works and i can make a PR!

An example of this implementation at work. The model clearly underperforms in the SYS label, by comparison:
human-vad
chonker-vad

@hbredin
Copy link
Member

hbredin commented Jun 14, 2021

The visualisation part of pyannote.core definitely needs some love!
I'd likely merge a PR adding this kind of feature. Thanks!

Not sure how to name the option though.
What about arrangement="pack" (current behavior) vs arrangement="stack" (what you propose)?

@janaab11
Copy link
Author

The names you suggest sound apt for this option. I will put through a PR with these changes, this weekend - sorry about the delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants