FIO-8723: Clear values from submission for hidden comp with clearOnHide flag

[mikekotikov]

Link to Jira Ticket

https://formio.atlassian.net/browse/FIO-8723

Description

What changed?

Added reset of component's 'hidden' property in conditionals and logic processors, for cases when array data components (like Edit Grid/Data Grid) with multiple rows were processed.
The problem: the first time 'hidden' value got set in conditionals/logic processor, it was applied to component json and it was then thought to be present on every next row component, so the conditionals/logic wasn't recalculated for them.

Dependencies

None

How has this PR been tested?

Automated test was already there, but the assertion was incorrect, so fixed it

Checklist:

• I have completed the above PR template
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (if applicable)
• My changes generate no new warnings
• My changes include tests that prove my fix is effective (or that my feature works as intended)
• New and existing unit/integration tests pass locally with my changes
• Any dependent changes have corresponding PRs that are listed above

[brendanbond]

I'm confused on how this fixes the problem that the ticket describes. It seems to me that the problematic example in FIO-8723 just has two components with default values; how do we get from there to row components such as edit grid or data grid?

[mikekotikov]

I'm confused on how this fixes the problem that the ticket describes. It seems to me that the problematic example in FIO-8723 just has two components with default values; how do we get from there to row components such as edit grid or data grid?

Sorry, wrote not good description for the ticket. So, the original problem from the ticket is fixed by adding || component.hidden check in src/process/clearHidden.ts.
And other stuff with deleting component.hidden was then added to fix the tests, since there was a wrong behavior for array data components, where in case of logic/conditions, the result of first row calculation was thought to be the same for all the next rows.

[brendanbond]

Up to now, we've used the hidden component JSON property as a "source of truth" to determine whether or not a component is hidden. A component is conditionally hidden? Add the component path to the conditionallyHidden scope and set the hidden property on the component's JSON to true. A component is logically hidden? Add the component path to the conditionallyHidden scope and set the hidden property on the component's JSON to true.

My problem with using the delete operator to get around the fact that rows in nested data components need independent hidden contexts is that it essentially does an end run around this system. Rather than determining during processing that a component should be hidden or not, it always "resets" the hidden property to false, meaning that the scope will never be aware of the component despite the fact that it has technically been logically or conditionally hidden.

I think a better solution to this problem might be to rethink the way in which we evaluate for hidden. Rather than setting/unsetting the component's hidden property when we determine a component should be hidden, could we instead utilize the scope more effectively and say "if this component appears in the conditionallyHidden scope or if it is manually hidden, then we will clear the value." In other words, we're going to stop using the component's hidden property as a source of truth (and hopefully even stop setting/unsetting it) and use the scope as the source of truth, and ensure that each and every component (which would include nested rows in data grids, e.g.) is correctly added to the conditionallyHidden scope.

Feel free to ping me to discuss further.

[mikekotikov]

@brendanbond refactored, removed all the places where component.hidden was mutated and my changes with deleting it

[mikekotikov]

This test appeared to be doing invalid assertion, since clearOnHide is false in this case, the fields should be left. Corrected it.

[TanyaGashtold]

This PR fixes one more issue related to conditional components inside EditGrid: https://formio.atlassian.net/browse/FIO-8992

[brendanbond]

Really nice work @mikekotikov ! I went ahead and added back in a test case (it still works) and merged master to verify that formio-server tests were passing. All looks good, thanks dude