Request Limit

[mingyc]

Open questions from last meeting:

• How to limit: document-wide limit, tab-wide limit, limit per top-level origin, limit per reporting origin?
• Total size limit: 64K for each reporting origins, overall quota for 640K for entire tab?
• Permissions policy: default on or off?

Some concerns:

1. Total limit should not be easily consumed/abused by a single origin
2. Origin-specific info should not be leaked via quota consumption
3. A single request can easily take up 50k (e.g. Boomerang)

Relevant Discussions: w3c/beacon#38 (comment)

@noamr @annevk @nicjansma @yoavweiss

[fergald]

How about taking time into consideration? I'm not going to try to spec precisely here but at the high level:

• a page that has been open for just a few seconds should have a small quota
• a page that has been open for a longer time should have a larger quota
• iframe quotas could grow more slowly than top-level frames
• a permission-policy could "upgrade" an iframe to use the top-level quota

The gaols I'm trying to reach with this are:

• recognize that if we don't make this usable, pages will just send eagerly with fetch() and/or use keep-alive
• enabled by default for iframes to at least some extent
• ability for the top-level frame to "promote" iframes to same status

[noamr]

How about taking time into consideration? I'm not going to try to spec precisely here but at the high level:

• a page that has been open for just a few seconds should have a small quota
• a page that has been open for a longer time should have a larger quota
• iframe quotas could grow more slowly than top-level frames
• a permission-policy could "upgrade" an iframe to use the top-level quota

The gaols I'm trying to reach with this are:

• recognize that if we don't make this usable, pages will just send eagerly with fetch() and/or use keep-alive
• enabled by default for iframes to at least some extent
• ability for the top-level frame to "promote" iframes to same status

Maybe start with shipping something simpler and extend later?

[mingyc]

Pinging @yoavweiss @noamr

As the OT is started, we might want to continue this discussion.

[yoavweiss]

I'd love to hear @annevk's opinion on the options in the OP.

Let me try to clarify (for my own sake) the threat model this quota is protecting against.
Both top-level frames and iframes can use up network resources without any limitations while they are alive. IIRC, the claim was that users can notice this (unclear how - maybe by seeing that they are network constrained elsewhere) and close these tabs to stop that use, while they won't be able to do the same with fetchLater calls.

Hence, we're trying to protect against network abuse by any individual party on a particular site.

If our quotas were to span across frames, they are likely to end up leaking cross-origin state or enable interference with cross-origin activity, and may cause more harm than their presumed protection.

But if our quotas don't span across frames, abusers can easily circumvent them by creating more frames that send more data.

Given the above, my preference would be:

• 64KB limit per reporting origin. 10 reporting origins per frame.
  • Given that the median mobile page is now at 2.2MB, ~30% of that can't possibly be considered abuse.
  • I think that should be enough, but we should be ready and willing to bump those numbers up if they end up unrealistic for real-world deployments.
• If UAs detect abusers of this (e.g. multiple cross-origin reporting iframes that go to the same "first party set", for whatever definition of a "first party set"), they MAY intervene against these abusers and lower quotas for these abusers' origins (up to and including 0KB).

[annevk]

I haven't really changed my thinking on this. We'd give a top-level origin/site a quota and it can share that quota using Permissions Policy. That still seems like the most robust approach. That also puts the blame squarely with the top-level origin/site, which is generally what you want if you were to surface any of this to end users.

[noamr]

I haven't really changed my thinking on this. We'd give a top-level origin/site a quota and it can share that quota using Permissions Policy. That still seems like the most robust approach. That also puts the blame squarely with the top-level origin/site, which is generally what you want if you were to surface any of this to end users.

I also don't see a way around this. It does mean that fetchLater would only work reliably for cross-origin iframes if permitted by permissions-policy.

[mingyc]

I am going to add Permissions Policy into OT implementation, but houw about its default? I think from our side we still want to minimize the friction to make the API available to 3P frames, and making it to 'self' means that the API won't be available in most of the existing sites at all.

[annevk]

That is the correct default though. Otherwise you make the top-level site responsible for something it didn't even know.

[noamr]

That is the correct default though. Otherwise you make the top-level site responsible for something it didn't even know.

I tend to agree with @annevk on this, though perhaps this is not a spec concern. I can't find any place in the spec that defines default values for these policies but perhaps I'm missing something.

I can see how hypothetically a UA can e.g. decide to enable this by default but disable it when there are signs of abuse.

[mingyc]

We'd give a top-level origin/site a quota and it can share that quota using Permissions Policy.

Combing the comment from Yoav

64KB limit per reporting origin. 10 reporting origins per frame

Does that mean a top-level frame has 640KB quota by default, and it allows at most 10 reporting origins within this frame? What happens when 11th origin makes a fetchLater? Does all iframes share the same 640KB or has their own independent quota?

And a quota for origin A in top-level frame is not shared for origin A in other top-level frame right?

[noamr]

We'd give a top-level origin/site a quota and it can share that quota using Permissions Policy.

Combing the comment from Yoav

64KB limit per reporting origin. 10 reporting origins per frame

Does that mean a top-level frame has 640KB quota by default, and it allows at most 10 reporting origins within this frame? What happens when 11th origin makes a fetchLater? Does all iframes share the same 640KB or has their own independent quota?
And a quota for origin A in top-level frame is not shared for origin A in other top-level frame right?

There is no limit to the number of reporting-sink origins. The proposal is:

• The top-level document has 640k quota for deferred fetches
• Each reporting-sink origin nested within that top-level document has a 64k quota. you can have more than 10 of those, but if you reach the total of 640k you're out of luck.
• Deferred fetching is allowed only if the top-level document delegates that quota using permission policy.
• The default permission policy is anyway not something in the spec, but in most/all cases it should be self.

[yoavweiss]

Each reporting-sink origin nested within that top-level document has a 64k quota. you can have more than 10 of those, but if you reach the total of 640k you're out of luck.

This seems fine as long as we think that there won't be more than 10 reporting sink in the "typical" case. If that's the case, this policy would be fine, and won't result in reporters "ensuring their quota" by registering and using it early.

@nicjansma - WDYT?

[noamr]

OK, I'm working on the PR, the main thing I need fleshed-out is how to deal with workers, since they:

• don't have a permissions policy
• A SharedWorker (or service worker) can belong to several documents at the same time.

I see three options here:

• Only support documents for now
• Only support documents + dedicated workers, and have the dedicated worker use its owner's permissions policy
• also support shared workers, but allow the shared worker to use fetchLater only if all its owners allow that in terms of permission, and all of them have the necessary quota (this is necessary because the worker might outlive any of these documents).

thoughts?

[mingyc]

We are fine with supporting only documents for now. There are infra limitation in Chromium such that fetchLater can't be supported in workers.

[mingyc]

Filed w3c/webappsec-permissions-policy#544

[mingyc]

@noamr @yoavweiss

• The top-level document has 640k quota for deferred fetches
• Each reporting-sink origin nested within that top-level document has a 64k quota. you can have more than 10 of those, but if you reach the total of 640k you're out of luck.

Although this is implementation-specific details, I just realized this requires a mechanism to enable cross-process iframes to update each other the size of their pending requests synchrounously (given the example), which is difficult to achieve and may not pass security review.

The above is neccessary, as the steps introduced by whatwg/fetch@8d237d0 assumes that child cross-site iframes can obtain all body sizes of pending fetchLater requests from their parent documents in real time when JS fetchLater() is called (such that QuotaExceededError can be thrown). However, traversing cross-site ancestors in renderer process won't provide anything useful

Is there any alternative approach for it?

[jakeherron-google]

Hi all,

Chiming in from an ads perspective. Reliability is of utmost importance for us.

Regarding origin related permissions:

The usefulness of FetchLater is that it will allow us to send our data in a single beacon at the end of the session, as opposed to many HTTP requests throughout the page life cycle. The problem for us is that we do not control the domain stack we are rendered into because we are often served and rendered with the ad. In most cases, we are rendered into a 3P iframe that we do not control meaning we cannot control the “deferred-fetch” permissions.

If FetchLater is only available in 3P iframes when explicitly permitted with “deferred-fetch” AND we can detect the non-permitted case, then we would have to fallback to the “multiple HTTP requests” solution for that traffic (meaning two event processing stacks would need to exist).

Worse, if FetchLater is only available in 3P iframes when explicitly permitted with “deferred-fetch” AND we CANNOT detect the non-permitted case, we will not be able to adopt this API at all. Knowing that the API is less reliable than stream solutions, would make the API a non-starter for us.

Regarding the size budget:

The beacon size limitation is less of a concern, under the assumption that it would be a constraint in exceedingly rare situations; however, if it turns out it happens often, that would also warrant us to revert to the multi-beacon approach. A size limitation feels like a premature optimization and seems like something we could investigate at a later point in time if there are issues in production.

[noamr]

Hi all,

Chiming in from an ads perspective. Reliability is of utmost importance for us.

Regarding origin related permissions:

The usefulness of FetchLater is that it will allow us to send our data in a single beacon at the end of the session, as opposed to many HTTP requests throughout the page life cycle. The problem for us is that we do not control the domain stack we are rendered into because we are often served and rendered with the ad. In most cases, we are rendered into a 3P iframe that we do not control meaning we cannot control the “deferred-fetch” permissions.

If FetchLater is only available in 3P iframes when explicitly permitted with “deferred-fetch” AND we can detect the non-permitted case, then we would have to fallback to the “multiple HTTP requests” solution for that traffic (meaning two event processing stacks would need to exist).

Worse, if FetchLater is only available in 3P iframes when explicitly permitted with “deferred-fetch” AND we CANNOT detect the non-permitted case, we will not be able to adopt this API at all. Knowing that the API is less reliable than stream solutions, would make the API a non-starter for us.

This will always be detectable using the permissions policy API, or by trying to call fetchLater and catching the appropriate error.

Regarding the size budget:

The beacon size limitation is less of a concern, under the assumption that it would be a constraint in exceedingly rare situations; however, if it turns out it happens often, that would also warrant us to revert to the multi-beacon approach. A size limitation feels like a premature optimization and seems like something we could investigate at a later point in time if there are issues in production.

Thanks for this feedback!

[fergald]

self effectively means we are excluding ads from ever using this API. I'm not sure if there's any other common use case that finds itself embedded in a stack of 3P frames.

This is a major loss and does not seem worth it to prevent some hypothetical quota stealing. If bad-acting 3rd parties turn out to be a real problem, it is easy for the top-level site to control that with a policy.

With regard to making progress and launching with self and reviewing things later and maybe making it *, is that realistic? If we launch with self, sites that care about controlling this will do nothing and if we switch to * it will require action from those sites. The switch to * is not something we can do lightly.

[noamr]

@jakeherron-google can you be more specific about "we do not control the domain stack we are rendered into because we are often served and rendered with the ad"? How does this work more in detail?

[annevk]

Note that the Permissions Policy API does not have cross-browser agreement. (I thought the plan was to merge it into the Permissions API but to what extent that has happened is unclear.)

[jakeherron-google]

@jakeherron-google can you be more specific about "we do not control the domain stack we are rendered into because we are often served and rendered with the ad"? How does this work more in detail?

Yes, to be more specific we deploy Javascript alongside the ad that conducts measurement throughout its lifecycle. The actual ad content along with any scripts will be served into a 3P iframe that the publisher webpage has control over. The publisher does not know anything about the ad served into the 3P frame, by design. Likewise, our ad measurement script has no control over this iframe by design. Our use of the fetchLater API would be beholden to the publisher opting into deferred-fetch for our ad’s iframe under the proposed permissions policy.

[noamr]

Summarizing internal conversations about this:
There is a strong use case for this feature to have a small quota available for 3p origins without requiring permissions policy. This use case is important for us (Google/Chromium).

This quota can be small, e.g. 16kb per 3p-origin-in-a-top-level-document. We see the risk of using this "spare change fund" quota for abusing bandwidth after window close as small, and is anyway possible in other means today such as ordinary keepalive requests (which don't have a standard-defined top-level quota) or service workers (which may be terminated when they don't have active clients, but in practice can send/receive quite a bit of data).

So proposing to keep the current spec wording, but to add a per-3p-origin quota to the effect of 16kb that doesn't require permissions policy. Allowing an iframe to use the overall quota would have the effect of increasing it to 64kb.

We'd prefer to have this as a normal part of the spec, but also open to having it implementation-defined or a MAY/SHOULD that allows user-agents to choose their tradeoff between utility and risk of bandwidth abuse. FWIW I think it's a legitimate tradeoff to leave for browsers to differentiate, and is discoverable enough so that it shouldn't create a substantial interop issue.

@annevk thoughts?

[annevk]

We discussed this once more internally.

We continue to be concerned about letting third parties consume end user system resources without explicit buy-in from the first party.

To the extent that is possible today through service workers that will likely be something we will eliminate going forward (the main reason for keeping service workers alive in WebKit has been to make navigation quicker, not for web application shutdown purposes). And thus we would not want to use that as precedent when designing a new API.

[noamr]

We discussed this once more internally.

We continue to be concerned about letting third parties consume end user system resources without explicit buy-in from the first party.

Understood. I suggest that having this as a MAY in the spec is a better way forward than remaining blocked on this, as allowing 3ps to use this feature without 1p buy-in is an important use case for us (with a very small quota).

We'd prefer to tackle hypothetical problems that arise from this, keeping an eye on metrics, rather than constrain this API in advance in a way that would make it a lot less useful. Since this API is anyway built to not always work based on all kinds of constraints, I think having this as a constraint that's up to some implementation-specific discretion is probably the best we can come up with.

To the extent that is possible today through service workers that will likely be something we will eliminate going forward (the main reason for keeping service workers alive in WebKit has been to make navigation quicker, not for web application shutdown purposes). And thus we would not want to use that as precedent when designing a new API.

Apart from service-workers, another precedence is ordinary keep-alive fetches, which can be used by a 3p without 1p buy-in. The quota proposed here is smaller than the quota of a single keep-alive fetch. Is WebKit also going to eliminate keep-alive fetches in 3p iframes?

[noamr]

Updated the PR, to include a 3p default quota of 16kb even if the permissions policy is not present (in a MAY clause). This is currently the only way I see forward given lack of consensus about this detail, which is really a small bit of this API. I think it's better to have one detail with an implementation-defined difference than hold this issue back indefinitely or ship it with a monkey-patch spec. Open to hear other alternatives!