From f75fb243048684769565c38043c5e151494979ea Mon Sep 17 00:00:00 2001 From: Richard Gibson Date: Wed, 1 May 2024 11:51:26 -0400 Subject: [PATCH 01/10] chore: Fix typo (#185) --- aep/general/0160/aep.md.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aep/general/0160/aep.md.j2 b/aep/general/0160/aep.md.j2 index c3d6cff9..71cda869 100644 --- a/aep/general/0160/aep.md.j2 +++ b/aep/general/0160/aep.md.j2 @@ -18,7 +18,7 @@ so, they **should** follow the common specification for filters discussed here. The syntax is formally defined in the [CEL language definition][]. When employing filtering, a request message **should** have exactly one -filtering field, `string filter`. THe value of this field **must** be a Boolean +filtering field, `string filter`. The value of this field **must** be a Boolean CEL expression. For example, given the resources From d8d41c00e8ff6586a7218b6050798e49bbf586fb Mon Sep 17 00:00:00 2001 From: Mike Kistler Date: Fri, 3 May 2024 06:51:05 -0500 Subject: [PATCH 02/10] Create CODE_OF_CONDUCT.md (#160) * Create CODE_OF_CONDUCT.md based on CNCF CoC * Remove CNCF-specifics from Code of Conduct --- CODE_OF_CONDUCT.md | 93 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 CODE_OF_CONDUCT.md diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..406105f4 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,93 @@ +## Community Code of Conduct + +As contributors, maintainers, and participants in our community, and in the +interest of fostering an open and welcoming community, we pledge to respect all +people who participate or contribute through reporting issues, posting feature +requests, updating documentation, submitting pull requests or patches, +attending conferences or events, or engaging in other community or project +activities. + +We are committed to making participation in our community a harassment-free +experience for everyone, regardless of age, body size, caste, disability, +ethnicity, level of experience, family status, gender, gender identity and +expression, marital status, military or veteran status, nationality, personal +appearance, race, religion, sexual orientation, socioeconomic status, tribe, or +any other dimension of diversity. + +## Scope + +This code of conduct applies: + +- within project and community spaces, +- in other spaces when an individual community participant's words or actions + are directed at or are about a project, our community, or another community + participant. + +## Our Standards + +Our Community is open, inclusive and respectful. Every member of our community +has the right to have their identity respected. + +Examples of behavior that contributes to a positive environment include but are +not limited to: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +- Focusing on what is best not just for us as individuals, but for the overall + community +- Using welcoming and inclusive language + +Examples of unacceptable behavior include but are not limited to: + +- The use of sexualized language or imagery +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment in any form +- Publishing others' private information, such as a physical or email address, + without their explicit permission +- Violence, threatening violence, or encouraging others to engage in violent + behavior +- Stalking or following someone without their consent +- Unwelcome physical contact +- Unwelcome sexual or romantic attention or advances +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +The following behaviors are also prohibited: + +- Providing knowingly false or misleading information in connection with a Code + of Conduct investigation or otherwise intentionally tampering with an + investigation. +- Retaliating against a person because they reported an incident or provided + information about an incident as a witness. + +Project maintainers have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other contributions +that are not aligned to this Code of Conduct. By adopting this Code of Conduct, +project maintainers commit themselves to fairly and consistently applying these +principles to every aspect of managing a project. Project maintainers who do +not follow or enforce the Code of Conduct may be temporarily or permanently +removed from the project team. + +## Reporting + +For incidents occurring in our community, please contact any of the individual +maintainers of the GitHub organization or Slack workspace. + +## Enforcement + +Upon review and investigation of a reported incident, the AEP maintainers that +has jurisdiction will determine what action is appropriate based on this Code +of Conduct and its related documentation. + +## Acknowledgements + +This code of conduct is based on and consistent with the [CNCF Community Code +of Conduct v1.3], which is adapted from the Contributor Covenant +(http://contributor-covenant.org), version 2.0 available at +http://contributor-covenant.org/version/2/0/code_of_conduct/ + +[CNCF Community Code of Conduct v1.3]: + https://github.com/cncf/foundation/blob/main/code-of-conduct.md From 8f0330c0ce5a2bbd5dbb279bffa1abddce75e2aa Mon Sep 17 00:00:00 2001 From: Alex Stephen <1325798+rambleraptor@users.noreply.github.com> Date: Mon, 6 May 2024 18:44:40 -0700 Subject: [PATCH 03/10] Adopt AIP-0203: Field Behavior (#170) * adding raw aip * remove google * prettier * Update aep/general/0203/aep.md.j2 Co-authored-by: Yusuke Tsutsumi * Update aep/general/0203/aep.md.j2 Co-authored-by: Yusuke Tsutsumi * PR comments * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Mike Kistler * Update aep/general/0203/aep.md.j2 Co-authored-by: Yusuke Tsutsumi * Update aep.md.j2 Co-authored-by: Mike Kistler * Update aep.md.j2 Co-authored-by: Mike Kistler --------- Co-authored-by: Yusuke Tsutsumi Co-authored-by: Mike Kistler --- aep/general/0203/aep.md.j2 | 254 ++++++++++++++++++++++++++++++++++++- aep/general/0203/aep.yaml | 4 +- 2 files changed, 253 insertions(+), 5 deletions(-) diff --git a/aep/general/0203/aep.md.j2 b/aep/general/0203/aep.md.j2 index 5847ec0b..529f8e9d 100644 --- a/aep/general/0203/aep.md.j2 +++ b/aep/general/0203/aep.md.j2 @@ -1,5 +1,253 @@ # Field behavior documentation -**Note:** This AEP has not yet been adopted. See -[this GitHub issue](https://github.com/aep-dev/aep.dev/issues/29) for more -information. +When defining fields in protocol buffers, it is customary to explain to users +certain aspects of the field's behavior (such as whether it is required or +optional). Additionally, it can be useful for other tools to understand this +behavior (for example, to optimize client library signatures). + +## Guidance + +APIs use the `aep.api.field_behavior` annotation to describe well-understood +field behavior, such as a field being required or immutable. + +```proto +// The audio data to be recognized. +RecognitionAudio audio = 2 [(aep.api.field_behavior) = REQUIRED]; +``` + +- APIs **must** apply the `aep.api.field_behavior` annotation on every field on + a message or sub-message used in a request. +- The annotation **must** include any google.api.FieldBehavior values that + accurately describe the behavior of the field. + - `FIELD_BEHAVIOR_UNSPECIFIED` **must not** be used. +- APIs **must** at minimum use one of `REQUIRED`, `OPTIONAL`, or `OUTPUT_ONLY`. + +**Warning:** Although `field_behavior` does not impact proto-level behavior, +many clients (e.g. CLIs and SDKs) rely on them to generate code. Thoroughly +review and consider which values are relevant when adding a new field. + +**Note:** The vocabulary given in this document is for _descriptive_ purposes +only, and does not itself add any validation. The purpose is to consistently +document this behavior for clients. + +### field behavior of nested messages + +`aep.api.field_behavior` annotations on a nested message are independent of the +annotations of the parent. + +For example, a nested message can have a field behavior of `REQUIRED` while the +parent field can be `OPTIONAL`: + +```proto +message Title { + string text = 1 [(aep.api.field_behavior) = REQUIRED]; +} + +message Slide { + Title title = 1 [(aep.api.field_behavior) = OPTIONAL]; +} +``` + +In the case above, if a `title` is specified, the `text` field is required. + +## Vocabulary + +### Identifier + +The use of `IDENTIFIER` indicates that a field within a resource message is +used to identify the resource. It **must** be attached to the `path` field and +**must not** be attached to any other field (see [fields representing resource +paths]). + +The `IDENTIFIER` value conveys that the field is not accepted as input (i.e. +`OUTPUT_ONLY`) in the context of a create method, while also being considered +`IMMUTABLE` and accepted as input for mutation methods that accept the resource +as the primary input e.g. [Standard Update](./standard-methods). + +This annotation **must not** be applied to references to other resources within +a message. + +### Immutable + +The use of `IMMUTABLE` indicates that a field on a resource cannot be changed +after it's creation. This can apply to either fields that are input or outputs, +required or optional. + +When a service receives an immutable field in an update request (or similar), +even if included in the update mask, the service **should** ignore the field if +the value matches, but **should** error with `INVALID_ARGUMENT` if a change is +requested. + +Potential use cases for immutable fields (this is not an exhaustive list) are: + +- Attributes of resources that are not modifiable for the lifetime of the + application (e.g. a disk type). + +**Note:** Fields which are "conditionally immutable" **must not** be given the +immutable annotation. + +### Input only + +The use of `INPUT_ONLY` indicates that the field is provided in requests and +that the corresponding field will not be included in output. + +Additionally, a field **should** only be described as input only if it is a +field in a resource message or a field of a message included within a resource +message. Notably, fields in request messages (a message which only ever acts as +an argument to an RPC, with a name usually ending in `Request`) **should not** +be described as input only because this is already implied. + +Potential use cases for input only fields (this is not an exhaustive list) are: + +- The `ttl` field as described in AEP-214. + +**Warning:** Input only fields are rare and should be considered carefully +before use. + +### Optional + +The use of `OPTIONAL` indicates that a field is not required. + +A field **may** be described as optional if it is a field on a request message +(a message that is an argument to an RPC, usually ending in `Request`), or a +field on a submessage. + +### Output only + +The use of `OUTPUT_ONLY` indicates that the field is provided in responses, but +that including the field in a message in a request does nothing (the server +**must** clear out any value in this field and **must not** throw an error as a +result of the presence of a value in this field on input). Similarly, services +**must** ignore the presence of output only fields in update field masks (see: +AEP-161). + +Additionally, a field **should** only be described as output only if it is a +field in a resource message, or a field of a message farther down the tree. +Notably, fields in response messages (a message which only ever acts as a +return value to an RPC, usually ending in `Response`) **should not** be +described as output only because this is already implied. + +Output only fields **may** be set to empty values if appropriate to the API. + +Potential use cases for output only fields (this is not an exhaustive list) +are: + +- Create or update timestamps. +- Derived or structured information based on original user input. +- Properties of a resource assigned by the service which can not be altered. + +### Required + +The use of `REQUIRED` indicates that the field **must** be present (and set to +a non-empty value) on the request. + +A field **should** only be described as required if _either_: + +- It is a field on a resource that a user provides somewhere as input. + - When [creating](./standard-methods) the resource, a value **must** be + provided for the field on the create request. + - When [updating](./standard-methods) the resource, the user **may** omit the + field provided that the field is also absent from the field mask, + indicating no change to the field (otherwise it **must** be provided). +- It is a field on a request message (a message that is an argument to an RPC, + with a name usually ending in `Request`). In this case, a value **must** be + provided as part of the request, and failure to do so **must** cause an error + (usually `INVALID_ARGUMENT`). + +Fields **should not** be described as required in order to signify: + +- A field which will always be present in a response. +- A field which is conditionally required in some situations. +- A field on any message (including messages that are resources) which is never + used as user input. + +**Note:** In most cases, empty values (such as `false` for booleans, `0` for +integers, or the unspecified value for enums) are indistinguishable from unset +values, and therefore setting a required field to a falsy value yields an +error. A corollary to this is that a required boolean must be set to `true`. + +### Unordered List + +The use of `UNORDERED_LIST` on a repeated field of a resource indicates that +the service does not guarantee the order of the items in the list. + +A field **should** be described as an unordered list if the service does not +guarantee that the order of the elements in the list will match the order that +the user sent, including a situation where the service will sort the list on +the user's behalf. + +## Backwards compatibility + +Adding or changing `aep.api.field_behavior` values can represent a semantic +change in the API that is perceived as incompatible for existing clients. The +following are examples of backwards incompatible changes with +`aep.api.field_behavior`: + +- Adding `REQUIRED` to an existing field previously considered `OPTIONAL` + (implicitly or otherwise) +- Adding a new field annotated as `REQUIRED` to an existing request message +- Adding `OUTPUT_ONLY` to an existing field previously accepted as input +- Adding `INPUT_ONLY` to an existing field previously emitted as output +- Adding `IMMUTABLE` to an existing field previously considered mutable +- Removing `OUTPUT_ONLY` from an existing field previously ignored as input +- Removing `IDENTIFIER` from an existing field. + +There are some changes that _are_ backwards compatible, which are as follows: + +- Adding `OPTIONAL` to an existing field +- Adding `IDENTIFIER` to an existing `path` field +- Changing from `REQUIRED` to `OPTIONAL` on an existing field +- Changing from `OUTPUT_ONLY` and/or `IMMUTABLE` to `IDENTIFIER` on an existing + field +- Removing `REQUIRED` from an existing field +- Removing `INPUT_ONLY` from an existing field previously excluded in responses +- Removing `IMMUTABLE` from an existing field previously considered immutable + +## Rationale + +### Identifier field behavior + +Resource paths, the primary identifiers for any compliant resource, are never +fully constructed by the user on create. Such fields are typically assigned +`OUTPUT_ONLY` field behavior. They are, however, also often consumed as the +primary identifier in scenarios where the resource itself is the primary +request payload. Such fields could _not_ be considered `OUTPUT_ONLY`. +Furthermore, in mutation requests, like Standard Update, the resource path as +the primary identifier cannot be changed in place. Such fields are typically +assigned `IMMUTABLE` field behavior. These conflicting and context-dependent +field behaviors meant that a new value was necessary to single out and convey +the behavior of the resource path field. + +### Required set of annotations + +A field used in a request message must be either an input or an output. + +In the case of an output, the `OUTPUT_ONLY` annotation is sufficient. + +In the case of an input, a field is either required or optional, and therefore +should have at least the `REQUIRED` or `OPTIONAL` annotation, respectively. +Only providing `INPUT_ONLY` does not convey the necessity of the field, so +specifying either `REQUIRED` or `OPTIONAL` is still necessary. + +### Requiring field behavior + +By including the field behavior annotation for each field, the overall behavior +that the resource exhibits is more clearly defined. Clearly defined field +behavior improves programmatic clients and user understanding. + +Requiring the annotation also forces the API author to explicitly consider the +behavior when initially authoring of the API. + +Modifying field behavior after initial authoring can result in +backwards-incompatible changes in clients. For example, making an optional +field required results in backwards-incompatible changes in the method +signature of an RPC or a resource in a Declarative client. See the +[Backwards compatibility](#backwards-compatibility) section for more detailed +compatibility guidance. + +[fields representing resource paths]: + ./resource-paths#fields-representing-resource-paths + +## Changelog + +- **2024-04-17**: Added initial guidance. diff --git a/aep/general/0203/aep.yaml b/aep/general/0203/aep.yaml index 77451f5e..d2dd9723 100644 --- a/aep/general/0203/aep.yaml +++ b/aep/general/0203/aep.yaml @@ -1,7 +1,7 @@ --- id: 203 -state: reviewing +state: approved slug: field-behavior-documentation -created: 2023-01-22 +created: 2024-04-17 placement: category: fields From 4b91b83bc03fedcf72c14b3a9d0aed06ad1404cd Mon Sep 17 00:00:00 2001 From: Richard Frankel Date: Fri, 10 May 2024 14:39:39 -0400 Subject: [PATCH 04/10] Adopt Google's AIP-157: Partial responses (#174) * Adopt Google's AIP-157: Partial responses * De-Google * Add a warning about FieldMask docs conflicting with guidance. * Fix typo. * Fix link --- aep/general/0157/aep.md.j2 | 92 ++++++++++++++++++++++++++++++++++++-- aep/general/0157/aep.yaml | 2 +- 2 files changed, 90 insertions(+), 4 deletions(-) diff --git a/aep/general/0157/aep.md.j2 b/aep/general/0157/aep.md.j2 index 76763666..4a09d4ea 100644 --- a/aep/general/0157/aep.md.j2 +++ b/aep/general/0157/aep.md.j2 @@ -1,5 +1,91 @@ # Partial responses -**Note:** This AEP has not yet been adopted. See -[this GitHub issue](https://github.com/aep-dev/aep.dev/issues/11) for more -information. +Sometimes, a resource can be either large or expensive to compute, and the API +needs to give the user control over which fields it sends back. + +## Guidance + +APIs **may** support partial responses in one of two ways: + +### Field masks parameter + +Field masks (`google.protobuf.FieldMask`) can be used for granting the user +fine-grained control over what fields are returned. An API **should** support +the mask in a side channel. For example, the parameter can be specified either +using an HTTP query parameter, an HTTP header, or a [gRPC metadata entry][]. + +Field masks **should not** be specified in the +[request](./0157.md#read-masks-as-a-request-field). + +- The value of the field mask parameter **must** be a + `google.protobuf.FieldMask`. +- The field mask parameter **must** be optional: + - An explicit value of `"*"` **should** be supported, and **must** return all + fields. + - If the field mask parameter is omitted, it **must** default to `"*"`, + unless otherwise documented. +- An API **may** allow read masks with non-terminal repeated fields (unlike + update masks), but is not obligated to do so. + + **Warning:** There is a known conflict between this guidance and the + documentation of `FieldMask` itself: + [google/protobuf/field_mask.proto](https://github.com/protocolbuffers/protobuf/blob/5e84a6169cf0f9716c9285c95c860bcb355dbdc1/src/google/protobuf/field_mask.proto#L85-L86) + states that + `A repeated field is not allowed except at the last position of a paths string.` + As such, official libraries (such as `Merge` for FieldMasks) may not support + wildcards without additional work. Track the issue + [here](https://github.com/protocolbuffers/protobuf/issues/8547#issuecomment-2005180068). + Consider using the view enumeration pattern described below instead instead. + +**Note:** Changing the default value of the field mask parameter is a +[breaking change](./backwards-compatibility#semantic-changes). + +### View enumeration + +Alternatively, an API **may** support partial responses with view enums. View +enums are useful for situations where an API only wants to expose a small +number of permutations to the user: + +```proto +enum BookView { + // The default / unset value. + // The API will default to the BASIC view. + BOOK_VIEW_UNSPECIFIED = 0; + + // Include basic metadata about the book, but not the full contents. + // This is the default value (for both ListBooks and GetBook). + BOOK_VIEW_BASIC = 1; + + // Include everything. + BOOK_VIEW_FULL = 2; +} +``` + +- The enum **should** be specified as a `view` field on the request message. +- The enum **should** be named something ending in `-View` +- The enum **should** at minimum have values named `BASIC` and `FULL` (although + it **may** have values other than these). +- The `UNSPECIFIED` value **must** be valid (not an error), and the API + **must** document what the unspecified value will do. + - For List RPCs, the effective default value **should** be `BASIC`. + - For Get RPCs, the effective default value **should** be either `BASIC` or + `FULL`. +- The enum **should** be defined at the top level of the proto file (as it is + likely to be needed in multiple requests, e.g. both `Get` and `List`). See + [enumerations](./enumerations) for more guidance on top-level enumerations. +- APIs **may** add fields to a given view over time. APIs **must not** remove a + field from a given view (this is a breaking change). + + **Note:** If a service requires (or might require) multiple views with + overlapping but distinct values, there is a potential for a namespace + conflict. In this situation, the service **should** nest the view enum within + the individual resource. + +### Read masks as a request field + +**Warning:** Read masks as a single field on the request message, for example: +`google.protobuf.FieldMask read_mask` are **DEPRECATED**. + + +[gRPC metadata entry]: https://grpc.io/docs/what-is-grpc/core-concepts/#metadata + diff --git a/aep/general/0157/aep.yaml b/aep/general/0157/aep.yaml index 27441b32..32e24135 100644 --- a/aep/general/0157/aep.yaml +++ b/aep/general/0157/aep.yaml @@ -1,6 +1,6 @@ --- id: 157 -state: reviewing +state: approved slug: partial-responses created: 2023-01-22 placement: From f15f8e6695ce94fd8d0a035208bc925de4dfa725 Mon Sep 17 00:00:00 2001 From: Richard Frankel Date: Fri, 10 May 2024 16:17:16 -0400 Subject: [PATCH 05/10] Adopt AIP-200: Precedent (#139) * Adopt AIP-200: Precedent * De-Google * Add proto/OAS tabs * Add not-precedent redirect. * Update aep/general/0200/aep.md.j2 Co-authored-by: Yusuke Tsutsumi * Clarify that only must/must not guidance may be violated. --------- Co-authored-by: Yusuke Tsutsumi --- aep/general/0200/aep.md.j2 | 148 ++++++++++++++++++++++++++++++++++++- aep/general/0200/aep.yaml | 5 +- 2 files changed, 148 insertions(+), 5 deletions(-) diff --git a/aep/general/0200/aep.md.j2 b/aep/general/0200/aep.md.j2 index 6ea5134b..22b63384 100644 --- a/aep/general/0200/aep.md.j2 +++ b/aep/general/0200/aep.md.j2 @@ -1,5 +1,147 @@ # Precedent -**Note:** This AEP has not yet been adopted. See -[this GitHub issue](https://github.com/aep-dev/aep.dev/issues/34) for more -information. +Many times, APIs are written in ways that do not match new guidance that is +added to these standards after those APIs have already been released. +Additionally, sometimes it can make sense to intentionally violate standards +for particular reasons, such as maintaining consistency with established +systems, meeting stringent performance requirements, or other practical +concerns. Finally, as carefully as everyone reviews APIs before they are +released, sometimes mistakes can slip through. + +Since it often is not feasible to fix past mistakes or make the standards serve +every use case, APIs may be stuck with these exceptions for quite some time. +Further, since new APIs often base their designs (names, types, structures, +etc) on existing APIs, it is possible that a standards violation in one API +could spill over into other APIs, even if original reason for the exception is +not applicable to the other APIs. + +As a result of this problem, it is important to "stop the bleeding" of these +standards exceptions into new APIs, and additionally document the reasons for +each exception so that historical wisdom is not lost. + +## Guidance + +If an API violates "**should**" or "**should not**" AEP guidance for any +reason, there **must** be an internal comment linking to this document using +its descriptive link ([aep.dev/not-precedent]()) to ensure others do not copy +the violations or cite the errors as precedent of a "previously approved API". + +**Important:** APIs **must not** violate guidance specified with "**must**" or +"**must not**", even with a link to this AEP. Tools such as documentation +generators and client generators **may** assume full compliance with "**must**" +and "**must not**" guidance. + +The comment should also include an explanation of what violates standards and +why it is necessary. For example: + +{% tab proto %} + +```proto +message DailyMaintenanceWindow { + // Time within the maintenance window to start the maintenance operations. + // It must use the format "HH MM", where HH : [00-23] and MM : [00-59] GMT. + // (-- aep.dev/not-precedent: This was designed for consistency with crontab, + // and preceded the AEP standards. + // Ordinarily, this type should be `aep.type.TimeOfDay`. --) + string start_time = 2; + + // Output only. Duration of the time window, automatically chosen to be + // smallest possible in the given scenario. + // (-- aep.dev/not-precedent: This preceded the AEP standards. + // Ordinarily, this type should be `google.protobuf.Duration`. --) + string duration = 3; +} +``` + +{% tab oas %} + +**Note:** OAS example not yet written. + +{% endtabs %} + +**Important:** APIs should only be considered to be precedent-setting if they +are in beta or stable. + +### Local consistency + +If an API violates a standard throughout, it would be jarring and frustrating +to users to break the existing pattern only for the sake of adhering to the +global standard. + +For example, if all of an API's resources use `creation_time` (instead of the +standard field `create_time` described in AEP-142), a new resource in that API +should continue to follow the local pattern. + +However, others who might otherwise copy that API should be made aware that +this is contra-standard and not something to cite as precedent when launching +new APIs. + +{% tab proto %} + +```proto +// ... +message Book { + // (-- aep.dev/not-precedent: This field was present before there was a + // standard field. + // Ordinarily, it should be spelled `create_time`. --) + google.protobuf.Timestamp creation_time = 1; +} + +// ... +message Author { + // (-- aep.dev/not-precedent: `Book` had `creation_time` before there was + // a standard field, so we match that here for consistency. Ordinarily, + // this would be spelled `create_time`. --) + google.protobuf.Timestamp creation_time = 1; +} +``` + +{% tab oas %} + +**Note:** OAS example not yet written. + +{% endtabs %} + +### Pre-existing functionality + +Standards violations are sometimes overlooked before launching, resulting in +APIs that become stable and therefore can not easily be modified. Additionally, +a stable API may pre-date a standards requirement. + +In these scenarios, it is difficult to make the API fit the standard. However, +the API should still cite that the functionality is contra-standard so that +other APIs do not copy the mistake and cite the existing API as a reason why +their design should be approved. + +### Adherence to external spec + +Occasionally, APIs must violate standards because specific requests are +implementations of an external specification (for example, OAuth), and their +specification may be at odds with AEP guidelines. In this case, it is likely to +be appropriate to follow the external specification. + +### Adherence to existing systems + +Similar to the example of an external specification above, it may be proper for +an API to violate AEP guidelines to fit in with an existing system in some way. +This is a fundamentally similar case where it is wise to meet the customer +where they are. A potential example of this might be integration with or +similarity to a partner API. + +### Expediency + +Sometimes there are users who need an API surface by a very hard deadline or +money walks away. Since most APIs serve a business purpose, there will be times +when an API could be better but cannot get it that way and into users' hands +before the deadline. In those cases, API review councils **may** grant +exceptions to ship APIs that violate guidelines due to time and business +constraints. + +### Technical concerns + +Internal systems sometimes have very specific implementation needs (e.g., they +rely on operation transforms that speak UTF-16, not UTF-8) and adhering to AEP +guidelines would require extra work that does not add significant value to API +consumers. Future systems which are likely to expose an API at some point +should bear this in mind to avoid building underlying infrastructure which +makes it difficult to follow AEP guidelines. diff --git a/aep/general/0200/aep.yaml b/aep/general/0200/aep.yaml index aff81f59..07068bbc 100644 --- a/aep/general/0200/aep.yaml +++ b/aep/general/0200/aep.yaml @@ -1,7 +1,8 @@ --- id: 200 -state: reviewing +state: approved slug: precedent -created: 2023-01-22 +created: 2024-03-14 placement: category: best-practices +redirect_from: /not-precedent From dab58a62507694f99f4017e8dc6485ec70a764a7 Mon Sep 17 00:00:00 2001 From: Marsh Gardiner Date: Fri, 10 May 2024 23:00:35 -0700 Subject: [PATCH 06/10] replacing calendar link New calendar in aep.dev workspace replaces the older one. Also, removed the user aspects in the path, as they should be unnecessary. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7bd8c279..7c8298ad 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ following channels of communication are available: - [The #aep channel in the CNCF Slack](https://cloud-native.slack.com/archives/C04TX46UCTV). Sign up at https://slack.cncf.io. -- [The AEP Google Calendar, to view any upcoming meetings](https://calendar.google.com/calendar/u/0?cid=N2UzNWRkM2RmMTk0YTMyZjRmYTdjMDNhMzQ1NGUyNGJhMzY1MWU2ZjU2ODI0OGVmZTFkZGYxZTM0YTdiZWU5ZUBncm91cC5jYWxlbmRhci5nb29nbGUuY29t). +- [The AEP Google Calendar, to view any upcoming meetings](https://calendar.google.com/calendar?cid=N2UzNWRkM2RmMTk0YTMyZjRmYTdjMDNhMzQ1NGUyNGJhMzY1MWU2ZjU2ODI0OGVmZTFkZGYxZTM0YTdiZWU5ZUBncm91cC5jYWxlbmRhci5nb29nbGUuY29t). - A weekly project meeting is held, which anyone interested is welcome to attend! From ad1db8f48687f2aa210b896b041193a6648e5dc8 Mon Sep 17 00:00:00 2001 From: Alex Stephen <1325798+rambleraptor@users.noreply.github.com> Date: Mon, 13 May 2024 22:38:59 -0700 Subject: [PATCH 07/10] AEP-0191: File and Directory Structure (#186) --- aep/general/0191/aep.md.j2 | 73 ++++++++++++++++++++++++++++++++++++-- aep/general/0191/aep.yaml | 6 ++-- 2 files changed, 73 insertions(+), 6 deletions(-) diff --git a/aep/general/0191/aep.md.j2 b/aep/general/0191/aep.md.j2 index 6b5cdd6f..ab7ff4f0 100644 --- a/aep/general/0191/aep.md.j2 +++ b/aep/general/0191/aep.md.j2 @@ -1,5 +1,72 @@ # File and directory structure -**Note:** This AEP has not yet been adopted. See -[this GitHub issue](https://github.com/aep-dev/aep.dev/issues/46) for more -information. +A consistent file and directory structure, while making minimal difference +technically, makes API surface definitions easier for users and reviewers to +read. + +## Syntax + +APIs defined in protocol buffers **must** use `proto3` syntax. + +## Single package + +APIs defined in protocol buffers **must** define each individual API in a +single package, which **must** end in a version component. For example: + +```proto +syntax = "proto3"; + +package example.cloud.translation.v3; +``` + +APIs **must** reside in a directory that matches the protocol buffer +`package` directive. For example, the package above dictates that the directory +be `example/cloud/translation/v3`. + +## File names + +It is often useful to divide API definitions into multiple files. File names +**must** use `snake_case`. + +APIs **should** have an obvious "entry" file, generally named after the API +itself. An API with a small number of discrete services **may** have a separate entry +file per service. + +APIs with only one file **should** use a filename corresponding to the name of +the API. + +API `service` definitions and associated RPC request and response `message` +definitions **should** be defined in the same file. + +Bear in mind that the file names often become module names in client libraries, +and customers use them in `import` or `use` statements. Therefore, choosing a +descriptive and language keyword-free filename does matter. For example, a file +called `import.proto` may be problematic in Python. + +**Note:** The version **must not** be used as a filename, because this creates +bizarre imports in client libraries. Filenames such as `v3.proto` or +`v1beta1.proto` are prohibited. + +## File layout + +Individual files **should** place higher level and more important definitions +before lower level and less important definitions. + +In a proto file, components **should** be in the following order, and each of +these **should** be separated by a blank line: + +- Copyright and license notice (if applicable). +- The proto `syntax` statement. +- The proto `package` statement. +- Any `import` statements, in alphabetical order. +- Any file-level `option` statements. +- Any `service` definitions. + - Methods **should** be grouped by the resource they impact, and standard + methods **should** precede custom methods. +- Resource `message` definitions. A parent resource **must** be defined before + its child resources. +- The RPC request and response `message` definitions, in the same order of the + corresponding methods. Each request message **must** precede its + corresponding response message (if any). +- Any remaining `message` definitions. +- Any top-level `enum` definitions. \ No newline at end of file diff --git a/aep/general/0191/aep.yaml b/aep/general/0191/aep.yaml index 9756f023..633e582a 100644 --- a/aep/general/0191/aep.yaml +++ b/aep/general/0191/aep.yaml @@ -1,7 +1,7 @@ --- id: 191 -state: reviewing +state: approved slug: file-and-directory-structure -created: 2023-01-22 +created: 2024-05-04 placement: - category: best-practices + category: protobuf From 490f5503799d2cbd4f94719a78bd94183104eafa Mon Sep 17 00:00:00 2001 From: Alex Stephen <1325798+rambleraptor@users.noreply.github.com> Date: Mon, 13 May 2024 22:39:06 -0700 Subject: [PATCH 08/10] AEP-192: Documentation (#187) --- aep/general/0192/aep.md.j2 | 135 ++++++++++++++++++++++++++++++++++++- aep/general/0192/aep.yaml | 4 +- 2 files changed, 134 insertions(+), 5 deletions(-) diff --git a/aep/general/0192/aep.md.j2 b/aep/general/0192/aep.md.j2 index 5b927311..de261b7f 100644 --- a/aep/general/0192/aep.md.j2 +++ b/aep/general/0192/aep.md.j2 @@ -1,5 +1,134 @@ # Documentation -**Note:** This AEP has not yet been adopted. See -[this GitHub issue](https://github.com/aep-dev/aep.dev/issues/47) for more -information. +Documentation is one of the most critical aspects of API design. Users of your +API are unable to dig into the implementation to understand the API better; +often, the API surface definition and its corresponding documentation will be +the only things a user has. Therefore, it is important that documentation be as +clear, complete, and unambiguous as possible. + +## Guidance + +In APIs defined in protocol buffers, public comments **must** be included over +every component (service, method, message, field, enum, and enum value) using +the protocol buffers comment format. This is important even in cases where the +comment is terse and uninteresting, as numerous tools read these comments and +use them. + +Services, in particular, **should** have descriptive comments that explain what +the service is and what users are able to do with it. + +**Note:** Many readers will not be native English speakers. Comments **should** +avoid jargon, slang, complex metaphors, pop culture references, or anything +else that will not easily translate. Additionally, many readers will have +different backgrounds and viewpoints; if writing examples involving people, +comments **should** use people who are non-controversial and no longer alive. + +### Style + +Comments **should** be in grammatically correct American English. However, the +first sentence of each comment **should** omit the subject and be in the +third-person present tense: + +```proto +// Creates a book under the given publisher. +rpc CreateBook(CreateBookRequest) returns (Book) { + option (google.api.http) = { + post: "/v1/{parent=publishers/*}/books" + body: "book" + }; +} +``` + +### Descriptions + +Descriptions of messages and fields **should** be brief but complete. Sometimes +comments are necessarily perfunctory because there is little to be said; +however, before jumping to that conclusion, consider whether some of the +following questions are relevant: + +- What is it? +- How do you use it? +- What does it do if it succeeds? What does it do if it fails? +- Is it idempotent? +- What are the units? (Examples: meters, degrees, pixels) +- What are the side effects? +- What are common errors that may break it? + - What is the expected input format? + - What range of values does it accept? (Examples: `[0.0, 1.0)`, `[1, 10]`) + - Is the range inclusive or exclusive? + - For strings, what is the minimum and maximum length, and what characters + are allowed? + - If a value is above the maximum length, do you truncate or send an error? +- Is it always present? (Example: "Container for voting information. Present + only when voting information is recorded.") +- Does it have a default setting? (Example: "If `page_size` is omitted, the + default is 50.") + +### Formatting + +Any formatting in comments **must** be in [CommonMark][]. Headings and tables +**must not** be used, as these cause problems for several tools, and are +unsuitable for client library reference documentation. + +Comments **should** use `code font` for field or method names and for literals +(such as `true`). + +Raw HTML **must not** be used. + +"ASCII art" attempts to present a diagram within the protos **must not** be +used. The Markdown within the protos is consumed by a large number of renderers, +and any ASCII art is very unlikely to be well-presented by all of them. If +a diagram is useful in order to understand the API, include a link to a +documentation page containing the diagram as an image. + +### Cross-references + +Comments **may** "link" to another component (service, method, message, field, +enum, or enum value) by using the fully-qualified name of the element as a +Markdown reference link. For example: `[Book][company.example.v1.Book]` + +### External links + +Comments **may** link to external pages to provide background information +beyond what is described in the public comments themselves. External links +**must** use absolute (rather than relative) URLs, including the protocol +(usually `https`), and **should not** assume the documentation is located on +any particular host. For example: +`[Spanner Documentation](https://cloud.google.com/spanner/docs)` + +### Trademarked names + +When referring to the proper, trademarked names of companies or products in +comments, acronyms **should not** be used, unless the acronym is such dominant +colloquial use that avoiding it would obscure the reference (example: [IBM][]). + +Comments **should** spell and capitalize trademarked names consistent with the +trademark owner's current branding. + +### Deprecations + +To deprecate a component (service, method, message, field, enum, or enum value), +the `deprecated` [option](https://developers.google.com/protocol-buffers/docs/proto#options) +**must** be set to `true`, and the first line of the respective comment +**must** start with `"Deprecated: "` and provide alternative solutions for +developers. If there is no alternative solution, a deprecation reason **must** +be given. + +### Internal comments + + + +Comments **may** be explicitly marked as internal by wrapping internal content +in `(--` and `--)`. + +Non-public links, internal implementation notes (such as `TODO` and `FIXME` +directives), and other such material **must** be marked as internal. + +**Note:** Comments **should** use only leading comments (not trailing comments +or detached comments). In particular, comments **must not** use both a leading +and trailing comment to describe any component, because this is a common source +of inadvertent omissions of the internal content annotation. + +[commonmark]: https://commonmark.org/ +[ibm]: https://en.wikipedia.org/wiki/IBM \ No newline at end of file diff --git a/aep/general/0192/aep.yaml b/aep/general/0192/aep.yaml index 02f70ad3..d8b58765 100644 --- a/aep/general/0192/aep.yaml +++ b/aep/general/0192/aep.yaml @@ -1,7 +1,7 @@ --- id: 192 -state: reviewing +state: approved slug: documentation -created: 2023-01-22 +created: 2024-05-04 placement: category: best-practices From b0cab3429c71712525e728fd404b1c60179a1fa0 Mon Sep 17 00:00:00 2001 From: Richard Frankel Date: Wed, 15 May 2024 18:43:34 -0400 Subject: [PATCH 09/10] Adopt AIP-205: Beta-blocking changes (#138) * Adopt AIP-205: Beta-blocking changes * De-Google * Add proto/OAS tabs --- aep/general/0205/aep.md.j2 | 48 +++++++++++++++++++++++++++++++++++--- aep/general/0205/aep.yaml | 5 ++-- 2 files changed, 48 insertions(+), 5 deletions(-) diff --git a/aep/general/0205/aep.md.j2 b/aep/general/0205/aep.md.j2 index 24c65be4..3fda8227 100644 --- a/aep/general/0205/aep.md.j2 +++ b/aep/general/0205/aep.md.j2 @@ -1,5 +1,47 @@ # Beta-blocking changes -**Note:** This AEP has not yet been adopted. See -[this GitHub issue](https://github.com/aep-dev/aep.dev/issues/50) for more -information. +APIs often release an alpha version of their API in order to get early feedback +from customers. This API is provisional and can change many times before the +important feedback is incorporated and the API is made stable for beta. + +Since the purpose of alpha is to gather feedback, the API does not need to be +perfect yet, and it's not strictly necessary for API authors to address every +usability concern or address every point in the API standards. Often, API +authors and API reviewers will not agree on the best design, and the best way +to find out is by having users try out the API. + +However, once the feedback has been collected and the API is going to be +promoted to beta, usability concerns and style issues need to be addressed. +In order to ensure that these issues are not forgotten, they **should** be +explicitly documented in the API. + +## Guidance + +If an API has usability concerns or violates API standards, and the present +design should receive additional scrutiny before being carried through to the +beta version, there **must** be an internal comment linking to this document +using its descriptive link ([aep.dev/beta-blocker]()) to ensure that the design +is corrected before the API is released to beta. + +The comment **must** also indicate what kind of change should be made for beta. +For example: + +{% tab proto %} + +```proto +message InputConfig { + // Parameters for input. + // (-- aep.dev/beta-blocker: Convert well-known parameters into explicit + // fields before the beta launch. --) + map parameters = 1; +} +``` + +{% tab oas %} + +**Note:** OAS example not yet written. + +{% endtabs %} + +If an exception to API standards _does_ need to be carried through to beta and +GA, see AEP-200. diff --git a/aep/general/0205/aep.yaml b/aep/general/0205/aep.yaml index 843ece9a..a4c65ffd 100644 --- a/aep/general/0205/aep.yaml +++ b/aep/general/0205/aep.yaml @@ -1,7 +1,8 @@ --- id: 205 -state: reviewing +state: approved slug: beta-blocking-changes -created: 2023-01-22 +created: 2024-03-14 placement: category: governance +redirect_from: /beta-blocker \ No newline at end of file From 2e9aa800f165c3d53f7549fb1df097907839acce Mon Sep 17 00:00:00 2001 From: Yusuke Tsutsumi Date: Wed, 15 May 2024 22:29:17 -0700 Subject: [PATCH 10/10] feat(pagination): re-importing from AIP-158 (#168) AEP-158 forked from the general aip.dev, whose guidance diverged a bit from gooogle.aip.dev. This re-imports those, resulting in the following set of changes: - switching typescript examples to proto (with placeholder for openapi). - minor wording fixes. --------- Co-authored-by: Richard Frankel --- aep/general/0003/aep.md.j2 | 6 ++ aep/general/0158/aep.md.j2 | 189 +++++++++++++++---------------------- aep/general/0158/aep.yaml | 1 + 3 files changed, 83 insertions(+), 113 deletions(-) diff --git a/aep/general/0003/aep.md.j2 b/aep/general/0003/aep.md.j2 index 9a0669be..987de638 100644 --- a/aep/general/0003/aep.md.j2 +++ b/aep/general/0003/aep.md.j2 @@ -99,6 +99,12 @@ Examples of complexities that declarative clients abstract away include: [Terraform][] is an example of such a client. +### Schema + +A schema describes the structure of the request or response of an [API +method](#api-method), or a [resource](#api-resource). It refers both to an +OpenAPI schema as well as a protobuf message. + ### User A human being which is using an API directly, such as with cURL. This term is diff --git a/aep/general/0158/aep.md.j2 b/aep/general/0158/aep.md.j2 index 43184e94..be3d908f 100644 --- a/aep/general/0158/aep.md.j2 +++ b/aep/general/0158/aep.md.j2 @@ -1,100 +1,105 @@ # Pagination -APIs often need to provide collections of data, most commonly in the -[List][aip-132] standard method. However, collections can often be arbitrarily -sized, and also often grow over time, increasing lookup time as well as the -size of the responses being sent over the wire. Therefore, it is important that -collections be paginated. +APIs often need to provide collections of data, most commonly in the [List][] +standard method. However, collections can often be arbitrarily sized, and also +often grow over time, increasing lookup time as well as the size of the +responses being sent over the wire. Therefore, it is important that collections +be paginated. ## Guidance -Operations returning collections of data **must** provide pagination _at the -outset_, as it is a [backwards-incompatible change](#backwards-compatibility) -to add pagination to an existing method. +Methods returning collections of data **must** provide pagination _at the outset_, +as it is a [backwards-incompatible change](#backwards-compatibility) to add +pagination to an existing method. -```typescript +{% tab proto %} + +```proto // The request structure for listing books. -interface ListBooksRequest { +message ListBooksRequest { // The parent, which owns this collection of books. // Format: publishers/{publisher} - parent: string; + string parent = 1 [ + (google.api.field_behavior) = REQUIRED, + (google.api.resource_reference) = { + child_type: "library.googleapis.com/Book" + }]; // The maximum number of books to return. The service may return fewer than // this value. // If unspecified, at most 50 books will be returned. // The maximum value is 1000; values above 1000 will be coerced to 1000. - maxPageSize: bigint; + int32 max_page_size = 2; // A page token, received from a previous `ListBooks` call. // Provide this to retrieve the subsequent page. // // When paginating, all other parameters provided to `ListBooks` must match // the call that provided the page token. - pageToken: string; + string page_token = 3; } // The response structure from listing books. -interface ListBooksResponse { +message ListBooksResponse { // The books from the specified publisher. - books: Book[]; + repeated Book books = 1; // A token that can be sent as `page_token` to retrieve the next page. // If this field is omitted, there are no subsequent pages. - nextPageToken: string; + string next_page_token = 2; } ``` -- Request definitions for collections **should** define an - `int32 max_page_size` field, allowing users to specify the maximum number of - results to return. +- The field containing pagination results **should** be the first field in + the message and have a field number of `1`. + +{% tab oas %} + +**Note:** OAS example not yet written. + +{% endtabs %} + +- Request messages for collections **should** define an integer (`int32` for +protobuf) `max_page_size` field, allowing users to specify the maximum number of +results to return. - If the user does not specify `max_page_size` (or specifies `0`), the API chooses an appropriate default, which the API **should** document. The API **must not** return an error. - - If the user specifies `max_page_size` greater than the maximum permitted by - the service, the service **should** coerce down to the maximum permitted - page size. - - If the user specifies a negative value for `max_page_size`, the API - **must** return a `400 Bad Request` error. - - The service **should** the number of results requested, unless the end of - the collection is reached. - - However, occasionally this is infeasible, especially within expected time - limits. In these cases, the service **may** return fewer results than the - number requested (including zero results), even if not at the end of the - collection. -- Request definitions for collections **should** define a `string page_token` + - If the user specifies `max_page_size` greater than the maximum permitted by the + API, the API **should** coerce down to the maximum permitted page size. + - If the user specifies a negative value for `max_page_size`, the API **must** + send an `INVALID_ARGUMENT` error. + - The API **may** return fewer results than the number requested (including + zero results), even if not at the end of the collection. +- Request schemas for collections **should** define a `string page_token` field, allowing users to advance to the next page in the collection. - - If the user changes the `max_page_size` in a request for subsequent pages, - the service **must** honor the new page size. - - The user is expected to keep all other arguments to the operation request - the same; if any arguments are different, the API **should** send a - `400 Bad Request` error. + - The `page_token` field **must not** be required. + - If the user changes the `max_page_size` in a request for subsequent pages, the + service **must** honor the new page size. + - The user is expected to keep all other arguments to the method the same; if + any arguments are different, the API **should** send an `INVALID_ARGUMENT` + error. - The response **must not** be a streaming response. -- Services **may** support using page tokens across versions of a service, but - are not required to do so. -- Response definitions for collections **must** define a +- Response messages for collections **must** define a `string next_page_token` field, providing the user with a page token that may be used to retrieve the next page. - - The field containing pagination results **should** be the first field - specified. It **should** be a repeated field containing a list of resources - constituting a single page of results. + - The field containing pagination results **must** be a repeated + field containing a list of resources constituting a single page of results. - If the end of the collection has been reached, the `next_page_token` field **must** be empty. This is the _only_ way to communicate "end-of-collection" to users. - If the end of the collection has not been reached (or if the API can not - determine in time), the service **must** provide a `next_page_token`. -- Response definitions **may** include a `string next_page_url` field - containing the full URL for the next page. -- Response definitions for collections **may** provide an `int32 total_size` - field, providing the user with the total number of items in the list. - - This total **may** be an estimate (but the API **should** explicitly - document that). - -[rfc-8288]: https://tools.ietf.org/html/rfc8288 + determine in time), the API **must** provide a `next_page_token`. +- Response messages for collections **may** provide an integer (`int32` for +protobuf) `total_size` field, providing the user with the total number of items +in the list. + - This total **may** be an estimate. If so, the API **should** explicitly + document that. ### Skipping results -The request definition for a paginatied operation **may** define an -`int32 skip` field to allow the user to skip results. +The request definition for a paginated operation **may** define an integer +(`int32` for protobuf ) `skip` field to allow the user to skip results. The `skip` value **must** refer to the number of individual resources to skip, not the number of pages. @@ -111,9 +116,9 @@ If a `skip` value is provided that causes the cursor to move past the end of the collection of results, the response **must** be `200 OK` with an empty result set, and not provide a `next_page_token`. -### Opacity +### Page Token Opacity -Page tokens provided by services **must** be opaque (but URL-safe) strings, and +Page tokens provided by APIs **must** be opaque (but URL-safe) strings, and **must not** be user-parseable. This is because if users are able to deconstruct these, _they will do so_. This effectively makes the implementation details of your API's pagination become part of the API surface, and it becomes @@ -133,32 +138,21 @@ authorization to the underlying resources, and authorization **must** be performed on the request as with any other regardless of the presence of a page token. -### Expiring page tokens - -Many services store page tokens in a database internally. In this situation, -the service **may** expire page tokens a reasonable time after they have been -sent, in order not to needlessly store large amounts of data that is unlikely -to be used. It is not necessary to document this behavior. - -**Note:** While a reasonable time may vary between services, a good rule of -thumb is three days. - -### Consistency +### Page Token Expiration -When discussing pagination, consistency refers to the question of what to do if -the underlying collection is modified while pagination is in progress. The most -common way that this occurs is for a resource to be added or deleted in a place -that the pagination cursor has already passed. +Many APIs store page tokens in a database internally. In this situation, APIs +**may** expire page tokens a reasonable time after they have been sent, in +order not to needlessly store large amounts of data that is unlikely to be +used. It is not necessary to document this behavior. -Services **may** choose to be strongly consistent by approximating the -"repeatable read" behavior in databases, and returning exactly the records that -exist at the time that pagination begins. +**Note:** While a reasonable time may vary between APIs, a good rule of thumb +is three days. ### Backwards compatibility -Adding pagination to an existing operation is a backwards-incompatible change. -This may seem strange; adding fields to interface definitions is generally -backwards compatible. However, this change is _behaviorally_ incompatible. +Adding pagination to an existing method is a backwards-incompatible change. This +may seem strange; adding fields to proto messages is generally backwards +compatible. However, this change is _behaviorally_ incompatible. Consider a user whose collection has 75 resources, and who has already written and deployed code. If the API later adds pagination fields, and sets the @@ -167,48 +161,17 @@ now is only getting the first 50 (and does not know to advance pagination). Even if the API set a higher default limit, such as 100, the user's collection could grow, and _then_ the code would break. -For this reason, it is important to always add pagination to operations -returning collections _up front_; they are consistently important, and they can -not be added later without causing problems for existing users. +For this reason, it is important to always add pagination to RPCs returning +collections _up front_; they are consistently important, and they can not be +added later without causing problems for existing users. **Warning:** This also entails that, in addition to presenting the pagination fields, they **must** be _actually implemented_ with a non-infinite default value. Implementing an in-memory version (which might fetch everything then paginate) is reasonable for initially-small collections. -## Implementation +[list]: ./list -Page tokens **should** be versioned independently of the public API, so that -page tokens can be used with any version of the service. - -The simplest form of a page token only requires an offset. However, offsets -pose challenges when a distributed database is introduced, so a more robust -page token needs to store the information needed to find a "logical" position -in the database. The simplest way to do this is to include relevant data from -the last result returned. Primarily, this means the resource ID, but also -includes any other fields from the resource used to sort the results (for the -event where the resource is changed or deleted). - -This information is from the resource itself, and therefore might be sensitive. -Sensitive data **must** be encrypted before being used in a page token. -Therefore, the token also includes the date it was created, to allow for the -potential need to rotate the encryption key. - -This yields the following interface, which **may** be base64 encoded and used -as a page token: - -```typescript -interface PageTokenSecrets { - // The ID of the most recent resource returned. - lastId: string; - - // Any index data needed, generally 1:1 with the fields used for ordering. - indexData: Buffer[]; - - // When this token was minted. - createTime: Date; -} -``` +## Changelog -**Note:** This section does not preclude alternative page token implementations -provided they conform to the guidelines discussed in this document. +- **2024-04-14**: Imported from https://google.aip.dev/158. \ No newline at end of file diff --git a/aep/general/0158/aep.yaml b/aep/general/0158/aep.yaml index 128130de..500b34a5 100644 --- a/aep/general/0158/aep.yaml +++ b/aep/general/0158/aep.yaml @@ -3,6 +3,7 @@ id: 158 state: approved slug: pagination created: 2019-02-18 +updated: 2024-04-14 placement: category: design-patterns order: 60