How to replay projections?

[CumpsD]

Replaying Projections

Problem

Sometimes bugs occur which cause projections to project the wrong data. Fixing the bug by itself is not enough to get the system back in a correct state. For this, the projection needs to be rebuilt with the fixed code.

Projections take time however, some more than others, and during this period it is not acceptable to have missing data in the system. The old buggy data should remain available until the new corrected data is done projecting.

How do we deal with this?

Constraints

• Old data should remain available during a rebuild.
• Automation is a nice to have, but not a requirement, manual deploys are permitted for (rare) cases like this.

Brainstorm Conclusion

During a brainstorm among developers, we came to the conclusion automating this is not worth the investment at the time. Given the fact that bugs requiring a full rebuild are rather rare, it is acceptable to have some manual effort to coordinate these rebuilds.

The chosen path is to:

• Duplicate the existing buggy projection code as a new projection.
• Deploy the new projection and let it populate.
• Deploy a new version of the API which reads from the new fixed projection.
• Remove the buggy projection from the codebase.

Additionally, we have to keep in mind breaking data changes for any of these and utilize versioning in the API if required.