Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove overloaded use of modified property #80

Open
ldodds opened this issue Dec 12, 2017 · 1 comment
Open

Remove overloaded use of modified property #80

ldodds opened this issue Dec 12, 2017 · 1 comment

Comments

@ldodds
Copy link
Contributor

ldodds commented Dec 12, 2017

The ordering strategy that uses change numbers means that the modified property no longer contains a reliable date that the record changed. In this case it contains a version number.

When a publisher is using an alternative strategy, it should be indicated, e.g. by including the change number as a separate key. The modified property can then be retained as a time stamp if/when available.

@nickevansuk
Copy link
Contributor

nickevansuk commented Feb 21, 2018

In both cases the modified property's invariant holds, which is "It must be appropriately comparable to itself as either a string or integer, representing a chronological ordering". It is more useful to think about it as a ordered E-Tag rather than a modified date in this respect, as the primary purpose is cache invalidation and ensuring that any consuming System 2 can process items in parallel without depending solely on the order in which they appear in the feed.

To make this example real: as long as every page is processed and the pages are retrieved following the next URL, the items themselves can e.g. enter geolocation queue or a manual validation queue based on their contents without concern for halting reading of the feed until such items are resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants