MediaCapabilities API extended to WebRTC

[drkron]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: MediaCapabilities API extended to WebRTC
• Specification or proposal URL: https://w3c.github.io/media-capabilities/
• Caniuse.com URL (optional):
• Bugzilla URL (optional):
• Mozillians who can provide input (optional): @jan-ivar

Other information

This is an extension to the existing MediaCapabilities API to support the type webrtc for queries to both decodingInfo and encodingInfo.
This was first proposed at the WebRTC WG Virtual Interim that took place on December 2, 2020.

[jan-ivar]

I believe we're generally in favor? @padenot, @martinthomson, @jesup, @karlt, @bwc

I believe this is mostly a code move (w3c/webrtc-extensions#95) from today's RTCRtpSender.getCapabilities() (bug 1531460) and RTCRtpReceiver.getCapabilities (bug 1531460) APIs to MediaCapabilities.

The most sensitive part is probably w3c/webrtc-extensions#54, which I think was the motivator for moving this to MediaCapabilities, since that API already exposes similar info, to consolidate where this information is available.

Not sure who else on the security team to rope in on that.

[martinthomson]

Yes, but ... This describes an API that exposes a fairly large amount of fingerprinting information. It depends this on several grounds.

1. The information is already available. This is generally not a good reason if the information was simply available through other means, but I think that the assessment is somewhat more complex than that. If we want to continue to allow permission-less media playback, then exposing this information might be unavoidable. A clearer analysis of that already of this is probably needed in the spec.
2. The fingerprinting entropy is much less than it seems. This is claimed, but no analysis (empirical or otherwise) is offered in support of this.

It would be good to understand the general trends here. Is the fingerprinting surface getting smaller or larger? What are browsers able to do to influence this? When we expose this information, what do users gain?

All of this probably leads to us getting supportive still, but some aspects of the design (HDR in particular right now, but what is special might change over time) seem to generate very strong fingerprinting signals and I would like to understand how we might manage that problem better.

[drkron]

There are different ways the fingerprinting entropy can be reduced. The Chromium implementation uses buckets to reduce the number of valid input configurations (resolution, framerate). In particular the code that responds to WebRTC queries is very restrictive and quantizes the input configuration into 1 out of 8 buckets per codec profile. The amount of information that is exposed will be tracked but I don't have any data at the moment.

Some of the fields such as HDR is not applicable to WebRTC at the moment. I will prepare a PR to clarify this, this is tracked in this issue w3c/media-capabilities#187.

[jan-ivar]

See also #184