Network intents when applied using update API creates a new service instance

[amcopma]

Description:
This issue is observed when using update API for updating network intent of an application

Steps to reproduce:

1. create logcal cloud with cluster1
2. specify cluster1 as the application placement intent.
3. specify network intents for application
4. Approve and instantiate the dig.
5. Edit the application network intent.
6. Call update API.
7. A new service instance is created for updated network intents, and the previous service instance still remains.

[snackewm]

Please add examples of the intents used to reproduce - specifically step 3 and step 5.

[amcopma]

Please add examples of the intents used to reproduce - specifically step 3 and step 5.

For step 3, we are using below json payload
wlIntent := WorkloadIntent{ Metadata: Metadata{ Name: workloadIntentName, Description: "NA", UserData1: customData, UserData2: "data2"}, Spec: WorkloadIntentSpec{ AppName: appName, WorkloadResource: intentData.DigVersion + "-" + appName, Type: "Deployment", }, }
We use below URL to createworkload intent
url := "http://" + orch.MiddleendConf.OvnService + "/v2/projects/" + p + "/composite-apps/" + ca + "/" + v + "/deployment-intent-groups/" + digName + "/network-controller-intent/" + nwControllerIntentName + "/workload-intents" resp, err := orch.apiPost(jsonLoad, url, intentName)

For step 5, we are deleting the workload intent and then creating it
url := "http://" + orch.MiddleendConf.OvnService + "/v2/projects/" + p + "/composite-apps/" + ca + "/" + v + "/deployment-intent-groups/" + digName + "/network-controller-intent/" + nwControllerIntent + "/workload-intents/" + workloadIntentName + "/interfaces/" + ifaceName resp, err := orch.apiDel(url, ca+"_delIface")
and then creating workload intent using step mentioned as part of step3

Hope it helps

[snackewm]

@amcopma - I don't know the details of your example test case, so I'll describe some observations I made using the vfw test case (which uses network intents).

Let me know if any of the following may be relevant to your test case.

1. I instantiated the vfw test case. Then, I deleted one of the interface intents (on the 'firewall' app) and recreated an interface intent, changing the IP address of the interface in the intent. After running the 'update' command, I saw the the new 'firewall' pod trying to start up (never fully came up), but the existing pod continued to run.
2. I suspected the Deployment spec.strategy.type was a factor in my case, so I started over after changing the firewall deployment strategy type to be "Recreate". In this case, (doing the same test as I did in Error with Helm rendering while Instantiating composite app #1), the 'firewall' pod got deleted first and then the new one started.
3. I suspected the issue I had in Error with Helm rendering while Instantiating composite app #1 was I only change 1 IP address, but the new pod was trying to use the same IP on other interfaces, so that was why new pod did not come up. So, I reset the firewall to have the default strategy (as in Error with Helm rendering while Instantiating composite app #1) and this time, during the test, I changed all 3 firewall interface intents to have different IPs (so as not to conflict with initial pod). In this case, on doing the 'update' command, the new pod came up and the old pod was deleted.

What I observed in case #1 sounds kind of similar to what you are reporting.
If none of what I saw in #1-#3 is relevant to your issue, please post the full details of your test case, so we can attempt to reproduce.

[amcopma]

Hope this output helps

1. v1-firewall instantiated
ubuntu@nodeXXXX:~/allauddin/aarna-stream/awe/src/guimiddleend$ k get pod NAME READY STATUS RESTARTS AGE mongo-7dd4599d65-9v2jj 1/1 Running 0 9m21s v1-firewall-5cc964bb47-kd77j 0/1 Pending 0 71s
ubuntu@nodeXXXX:~/allauddin/aarna-stream/awe/src/guimiddleend$ k get pod -o yaml | grep nfn-network k8s.plugin.opnfv.org/nfn-network: '{"type":"ovn4nfv","Interface":[{"interface":"net0","name":"protected-private-net","defaultGateway":"false","ipAddress":"192.168.20.5"},{"interface":"net1","name":"emco-private-net","defaultGateway":"false","ipAddress":"10.10.20.5"},{"interface":"net2","name":"unprotected-private-net","defaultGateway":"false","ipAddress":"192.168.10.5"}]}' f:k8s.plugin.opnfv.org/nfn-network: {} ubuntu@nodeXXXX:~/allauddin/aarna-stream/awe/src/guimiddleend$ ^C

2. IP addresses updated by updating network intent using Update API
ubuntu@nodeXXXX:~/allauddin/aarna-stream/awe/src/guimiddleend$ k get pod NAME READY STATUS RESTARTS AGE mongo-7dd4599d65-9v2jj 1/1 Running 0 11m v1-firewall-5cc964bb47-kd77j 0/1 Pending 0 3m v1-firewall-f5bfbbc6b-862zn 0/1 Pending 0 6s
ubuntu@nodezXXXX:~/allauddin/aarna-stream/awe/src/guimiddleend$ k get pod v1-firewall-f5bfbbc6b-862zn -o yaml | grep nfn-network k8s.plugin.opnfv.org/nfn-network: '{"type":"ovn4nfv","Interface":[{"interface":"net0","name":"protected-private-net","defaultGateway":"false","ipAddress":"192.168.20.6"},{"interface":"net1","name":"emco-private-net","defaultGateway":"false","ipAddress":"10.10.20.6"},{"interface":"net2","name":"unprotected-private-net","defaultGateway":"false","ipAddress":"192.168.10.6"}]}' f:k8s.plugin.opnfv.org/nfn-network: {}
ubuntu@nodez00b03:~/allauddin/aarna-stream/awe/src/guimiddleend$ k get pod v1-firewall-5cc964bb47-kd77j -o yaml | grep nfn-network k8s.plugin.opnfv.org/nfn-network: '{"type":"ovn4nfv","Interface":[{"interface":"net0","name":"protected-private-net","defaultGateway":"false","ipAddress":"192.168.20.5"},{"interface":"net1","name":"emco-private-net","defaultGateway":"false","ipAddress":"10.10.20.5"},{"interface":"net2","name":"unprotected-private-net","defaultGateway":"false","ipAddress":"192.168.10.5"}]}' f:k8s.plugin.opnfv.org/nfn-network: {} ubuntu@nodez00b03:~/allauddin/aarna-stream/awe/src/guimiddleend$

[snackewm]

@amcopma - on the first instantiation of your firewall (before doing an update), does it ever get out of Pending status (and get to a Running status)?

[amcopma]

No, it does not get to running state in our first instantiation, and always is in pending state.
Could you please share your scripts that you have used for validating update of network intents? I can validate the same in our setups too.

[snackewm]

I was using the /kud/emcoctl-tests/test-vfw.yaml as my test case. I just created an addtional file (pasted below) which I used to delete interfaces from the firewall and then apply again (to change the IP address of the interfaces). So the full sequence looks something like this (I'm just running from command line):

./setup.sh cleanup
./setup.sh create
../../bin/emcoctl/emcoctl --config emco-cfg.yaml apply -f prerequisites.yaml -v values.yaml
../../bin/emcoctl/emcoctl --config emco-cfg.yaml apply -f test-vfw.yaml -v values.yaml

vfw had been instantiated, now delete some interface intents and apply again (with changes), then call update:
../../bin/emcoctl/emcoctl --config emco-cfg.yaml delete -f update1.yaml -v values.yaml
../../bin/emcoctl/emcoctl --config emco-cfg.yaml apply -f update1.yaml -v values.yaml
../../bin/emcoctl/emcoctl --config emco-cfg.yaml apply projects/proj1/composite-apps/compositevfw/v1/deployment-intent-groups/vfw_deployment_intent_group/update

emcoctl-tests]$ cat update1.yaml

version: emco/v2
resourceContext:
anchor: projects/{{.ProjectName}}/composite-apps/compositevfw/v1/deployment-intent-groups/vfw_deployment_intent_group/network-controller-intent/vfw_ovnaction_intent/workload-intents/firewall_workload_intent/interfaces
metadata :
name: firewall_emco_if
spec:
interface: eth3
name: emco-private-net
defaultGateway: "false"
ipAddress: 10.10.20.3

---

version: emco/v2
resourceContext:
anchor: projects/{{.ProjectName}}/composite-apps/compositevfw/v1/deployment-intent-groups/vfw_deployment_intent_group/network-controller-intent/vfw_ovnaction_intent/workload-intents/firewall_workload_intent/interfaces
metadata :
name: firewall_unprotected_if
spec:
interface: eth1
name: unprotected-private-net
defaultGateway: "false"
ipAddress: 192.168.10.3

---

version: emco/v2
resourceContext:
anchor: projects/{{.ProjectName}}/composite-apps/compositevfw/v1/deployment-intent-groups/vfw_deployment_intent_group/network-controller-intent/vfw_ovnaction_intent/workload-intents/firewall_workload_intent/interfaces
metadata :
name: firewall_protected_if
spec:
interface: eth2
name: protected-private-net
defaultGateway: "false"
ipAddress: 192.168.20.2

[snackewm]

No, it does not get to running state in our first instantiation, and always is in pending state.
Could you please share your scripts that you have used for validating update of network intents? I can validate the same in our setups too.

Also, it is worth investigating why your workload does not leave the Pending state. I suggest trying to look at logs, 'kubectl describe', etc. to see if you can determine that first.

If the initial Pod does not get to running state, then I guess the second one will also not get there - especially with default Deployment strategy of RollingUpdate.

[snackewm]

@amcopma - hi - is there any update to this? Have you determined why the initial Pod does not leave the Pending state?

[amcopma]

@snackewm Yes, we did identified the reason for the firewall service not getting into running state, I will retry once the fix is done and update this ticket