Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Passing VideoReader to video models #383

Merged
merged 5 commits into from
Jan 3, 2020
Merged

Passing VideoReader to video models #383

merged 5 commits into from
Jan 3, 2020

Conversation

brimoor
Copy link
Contributor

@brimoor brimoor commented Jan 1, 2020

This PR updates the eta.core.learning interface for models that previously took paths to videos on disk to instead take VideoReader instances that can read the frames.

This is preferable because it separates the abstraction of loading the video from applying the model.

The impetus for this was the need to write modules that apply video models to either a video path/frames sequence, which can be read by eta.core.video.FFmpegVideoReader or a video frames directory containing not necessarily sequential frames, which can be read by eta.core.video.SampledFramesVideoReader. Much easier to do now!

Note that this is safe to do now because there are not yet any concrete instances of VideoModel, VideoClassifier, or VideoObjectDetector in develop.

@brimoor brimoor added the feature Work on a feature request label Jan 1, 2020
@brimoor brimoor self-assigned this Jan 1, 2020
@jasoncorso
Copy link
Contributor

This is the right thing to do but two comments/questions:

  1. The pipeline system sees a video path and knows to create a video reader? What if, previously, certain other module parameters would impact how the video was read, such as ranges and rates?
  2. VideoReader will need to support seeking formally in order for this to work as modules may need to go back and forth or at the very least "reopen" a video.

@brimoor
Copy link
Contributor Author

brimoor commented Jan 2, 2020

  1. This module shows how one would configure the VideoReader as desired (any sources, any parameters, etc): https://github.com/voxel51/eta/pull/383/files#diff-73bfe1f6c2c139a1be5f0b3b743ccacd

  2. Yes good point. I'll leave the work of upgrading VideoReader to expressly support seeking/re-reading videos to the first person who needs that functionality. Shouldn't be too hard. VideoReaders need to be able to seek to specific frames #373 is updated to reflect this ask

@@ -472,11 +473,12 @@ class VideoClassifier(Classifier):
that featurizes the frames of the input video.
'''

def predict(self, video_path):
def predict(self, video_reader):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there some code in the concrete subclasses that needs to be modified accordingly? Or are there currently no concrete subclasses of VideoClassifier?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no concrete subclasses.

Copy link
Contributor

@benjaminpkane benjaminpkane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LTGM!

@brimoor brimoor merged commit 05f26db into develop Jan 3, 2020
@brimoor brimoor deleted the use-video-reader branch January 3, 2020 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Work on a feature request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants