Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Propagate unknown when an explicit flag is set, in the form unknown=...|PROPAGATE #1634

Open
wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

Propagate unknown when an explicit flag is set, in the form unknown=...|PROPAGATE #1634

wants to merge 6 commits into from

Commits on Sep 4, 2020

  1. Propagate unknown parameter to nested fields

    @mahenzon @sirosen
    mahenzon authored and sirosen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    95d1828 View commit details
    Browse the repository at this point in the history

  2. Make "propagate_unknown" behavior explicit 

    This adds a new option, `propagate_unknown` which defaults to False.

It controls the behavior of `unknown` with respect to nested
structures. Anywhere that `unknown` can be set, `propagate_unknown`
can be set. That means it can be applied to a schema instance, a load
call, schema.Meta, or to fields.Nested .

When set, nested deserialize calls will get the same value for
`unknown` which their parent call got and they will receive
`propagate_unknown=True`.

The new flag is completely opt-in and therefore backwards
compatible with any current usages of marshmallow.
Once you opt in to this behavior on a schema, you don't need to
worry about making sure it's set by nested schemas that you use.

In the name of simplicity, this sacrifices a bit of flexibility. A
schema with `propagate_unknown=True, unknown=...` will override the
`unknown` settings on any of its child schemas.

Tests cover usage as a schema instantiation arg and as a load arg
for some simple data structures.
    @sirosen
    sirosen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    9b97235 View commit details
    Browse the repository at this point in the history

  3. Make propagate_unknown respect any explicit value 

    propagate_unknown will still traverse any series of nested documents,
meaning that once you set propagate_unknown=True, it is true for the
whole schema structure.

However, this introduces tracking for whether or not `unknown` was set
explicitly. If `unknown=RAISE` is set because no value was specified,
we will set a new flag on the schema, `auto_unknown=True`.

propagate_unknown now has the following behavior:
- if the nested schema has auto_unknown=False, use the current value
  for `unknown` in the nested `load` call
- if a nested field has its `unknown` attribute set, use that in place
  of any value sent via `propagate_unknown`

Effectively, this means that if you set `unknown` explicitly anywhere
in a nested schema structure, it will propagate downwards from that
point.

Combined with the fact that propagate_unknown=True propagates
downwards across all schema barriers, including if
`propagate_unknown=False` is set explicitly somewhere, this could be
confusing. However, because the idea is for `propagate_unknown=True`
to eventually be the only supported behavior for marshmallow,
this is acceptable as a limitation.

auto_unknown is an attribute of schema opts and of schema instances,
with the same kind of inheritance behavior as other fields.
    @sirosen
    sirosen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    9e1f75f View commit details
    Browse the repository at this point in the history

  4. Add changelog and docs for propagate_unknown 

    The changelog entry, including credit to prior related work, covers
the closed issues and describes how `propagate_unknown` is expected to
behave.

Inline documentation attempts to strike a balance between clarify and
brevity.

closes marshmallow-code#1428, marshmallow-code#1429, marshmallow-code#1490, marshmallow-code#1575
    @sirosen
    sirosen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    3e711d7 View commit details
    Browse the repository at this point in the history

  5. Replace propagate_unknown with or-able unknown 

    Rather than having a new parameter which allows `propagate_unknown`
behavior, convert this into an option which can be set on the
``unknown`` parameter itself.

Drawing from other interfaces which present flags as or-able
values (traditionally bitmasks), express the setting of this option as
`x | PROPAGATE` for any supported `x`.

This makes it harder to propagate the value of `propagate` separately
from the rest of the value of `unknown`. For simplicity, change the
behavior here to suit this implementation.
    @sirosen
    sirosen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    e495fbe View commit details
    Browse the repository at this point in the history

  6. Make unknown propagation override child schemas 

    In order to simplify, the value of `unknown` will not be respected
in child schemas if `PROPAGATE` is used. This removes any need for the
`auto_unknown` tracking, so there's less complexity added to schemas
in order to implement this behavior.

Adjust tests and changelog to match.

Also make minor tweaks to ensure UnknownParam usage is consistent and
keep the diff with the dev branch smaller.
    @sirosen
    sirosen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    499d608 View commit details
    Browse the repository at this point in the history