Run scenarios with CHC enabled

[c12i]

This PR addreses #69

[ThetaSinner]

What are we going to do about needing different config for some scenarios?

[c12i]

I think the embed_conductor_config function might need to be refactored slightly, it should return a type rather than the yaml in the form of a string, so that values can be mutated by scenarios that need custom conductor config different from the one in the yaml files.

What do you think?

[ThetaSinner]

I'm not sure about the scenario modifying the config. We want to be able to run scenarios with different config. In this case that means we want to run with CHC on and off. We don't then want to be modifying the scenario code to run in different environments etc.

What would be really nice is if you could provide a conductor configuration "overlay". So the embed_conductor_config would load a base configuration then at runtime you could configure which overlay to load. It might make sense to merge those as conductor configuration objects rather than yaml.

[c12i]

Sorry I am late, got distracted with other work

I'm not sure about the scenario modifying the config. We want to be able to run scenarios with different config. In this case that means we want to run with CHC on and off. We don't then want to be modifying the scenario code to run in different environments etc.

I see what you mean, what would a good compromise be? perhaps having -chcvariants of the existing yaml configs, not the cleanest approach.

Or rather than reading from the conductor config files, we programmatically build the conductor configs from the scenario, with an env var i.e CHC_ENABLED determining whether a chc_url should be set.

I am open to discuss this further

[ThetaSinner]

When I'm talking about overlays, I mean this kind concept -> https://carvel.dev/ytt/

Not necessarily that tool but something along those lines. So we could have a with-chc.yaml and a with-dpki.yaml that we can choose to load.

I think choosing to load that with an environment variable like CHC_ENABLED is good. So the current macro can load all the config files and embed them. Then we can choose at runtime which ones we merge with the base conductor config.

I know that YAML merging is a bit of a pain in general but if it's purely additive for now, then it shouldn't be so bad I hope.

What do you think?

[c12i]

I see, would the contents of the with-chc.yaml and with-dpki.yaml only contain the CHC and DPKI configurations?

[ThetaSinner]

Exactly that, they'd just bring in the piece that they configure. So we might do other things the same way, like different tunings of the network or the conductor itself.

We probably couldn't easily guarantee that a given combination of overlays makes sense but I'm okay with leaving that up to the user for now

[c12i]

Understood. The only question I have now is how to run scenarios with CHC enabled via trycp. I believe the holoports would need to have the reference implementation running?

[ThetaSinner]

I think we need to deploy somewhere that the HoloPorts can see yes. We have a couple of servers but nowhere ideal really. I think this might need to be a question we ask to techops