Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HA Manifest - error converting YAMLto JSON #20532

Closed
xelab04 opened this issue Oct 25, 2024 · 10 comments
Closed

HA Manifest - error converting YAMLto JSON #20532

xelab04 opened this issue Oct 25, 2024 · 10 comments
Labels
bug Something isn't working

Comments

@xelab04
Copy link

xelab04 commented Oct 25, 2024

Checklist:

[* ] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
[* ] I've included steps to reproduce the bug.
[ ] I've pasted the output of argocd version.

Describe the bug

Yaml file provided appears to be invalid.

error parsing https://github.com/argoproj/argo-cd/blob/master/manifests/ha/install.yaml: error converting YAML to JSON: yaml: line 216: mapping values are not allowed in this context

To Reproduce

kubectl apply -n argocd -f https://github.com/argoproj/argo-cd/blob/master/manifests/ha/install.yaml

Expected behavior

Manifest is applied properly and all resources are created.

Version

Not applicable

@xelab04 xelab04 added the bug Something isn't working label Oct 25, 2024
@MaroIT
Copy link

MaroIT commented Nov 5, 2024

Is there any patch for this bug?

@andrii-korotkov-verkada
Copy link
Contributor

I don't know the exact issue, but see some lines where there's a multiline statement without any special char like | or |> before the value, e.g. https://github.com/argoproj/argo-cd/blob/5d0a3e6e9abd7e922e72e27c992ed0cdc0824f99/manifests/ha/install.yaml#L219-L220. This seems like a rendering issue. Do you have some minimal repro?
If not, I can try to avoid this kind of rendering, since I guess different Yaml parsers can have an issue with that.

@andrii-korotkov-verkada
Copy link
Contributor

There's this issue kubernetes-sigs/kustomize#947

@andrii-korotkov-verkada
Copy link
Contributor

I've found a way to fix it, following advice from https://stackoverflow.com/questions/45004464/yaml-dump-adding-unwanted-newlines-in-multiline-strings/45004775#45004775 and modifying the repr to also apply literal style if the length of the value is > 40. I'll send a fix soon.

@andrii-korotkov-verkada
Copy link
Contributor

Ah, when I use kustomize it apparently uses a regular type of the formatter, so the combined manifests are not getting updated. I either need to fix kustomize or have a hacky script to normalize all manifests.

@andrii-korotkov-verkada
Copy link
Contributor

Seems like ArgoCD repo doesn't have Python files as of now, so not sure adding that is a good idea. I'll try and see if I get rejected.

@andrii-korotkov-verkada
Copy link
Contributor

Python used to be there, but got removed.

@andrii-korotkov-verkada
Copy link
Contributor

Hm, though manifests update script are local only, so I think it'd be okay to use Python script.

andrii-korotkov-verkada added a commit to andrii-korotkov-verkada/argo-cd that referenced this issue Nov 10, 2024
…rgoproj#20532)

Can help with argoproj#20532

The manifests often used the plain style of strings, which could move a part of value to a new line, which could cause some parsers fail to parse. Using this style is not recommended. It was switched to the literal style when relevant and manifests were updated. To make sure next updates are good, the helper script to fix manifests is a part of the script to update manifests. It's in Python, but it's run only locally.

Signed-off-by: Andrii Korotkov <[email protected]>
andrii-korotkov-verkada added a commit to andrii-korotkov-verkada/argo-cd that referenced this issue Nov 10, 2024
…rgoproj#20532)

Can help with argoproj#20532

The manifests often used the plain style of strings, which could move a part of value to a new line, which could cause some parsers fail to parse. Using this style is not recommended. It was switched to the literal style when relevant and manifests were updated. To make sure next updates are good, the helper script to fix manifests is a part of the script to update manifests. It's in Python, but it's run only locally.

Let's cherry pick to 2.13, as the apply of install manifests is currently broken.

Signed-off-by: Andrii Korotkov <[email protected]>
andrii-korotkov-verkada added a commit to andrii-korotkov-verkada/argo-cd that referenced this issue Nov 10, 2024
…rgoproj#20532)

Can help with argoproj#20532

The manifests often used the plain style of strings, which could move a part of value to a new line, which could cause some parsers fail to parse. Using this style is not recommended. It was switched to the literal style when relevant and manifests were updated. To make sure next updates are good, the helper script to fix manifests is a part of the script to update manifests. It's in Python, but it's run only locally.

There's one weird value for haproxy.cfg which is rendered as a quotes-enclosed string, but it was correct when I tried to load yamls from upstream and updated ones. In any case, it's more readable now.

Let's cherry pick to 2.13, as the apply of install manifests is currently broken.

Signed-off-by: Andrii Korotkov <[email protected]>
@andrii-korotkov-verkada
Copy link
Contributor

@xelab04, can you try to load my branch and apply the manifests, please?

andrii-korotkov-verkada added a commit to andrii-korotkov-verkada/argo-cd that referenced this issue Nov 10, 2024
…rgoproj#20532)

Can help with argoproj#20532

The manifests often used the plain style of strings, which could move a part of value to a new line, which could cause some parsers fail to parse. Using this style is not recommended. It was switched to the literal style when relevant and manifests were updated. To make sure next updates are good, the helper script to fix manifests is a part of the script to update manifests. It's in Python, but it's run only locally.

There's one weird value for haproxy.cfg which is rendered as a quotes-enclosed string, but it was correct when I tried to load yamls from upstream and updated ones. In any case, it's more readable now.

Let's cherry pick to 2.13, as the apply of install manifests is currently broken.

Signed-off-by: Andrii Korotkov <[email protected]>
andrii-korotkov-verkada added a commit to andrii-korotkov-verkada/argo-cd that referenced this issue Nov 10, 2024
…rgoproj#20532)

Can help with argoproj#20532

The manifests often used the plain style of strings, which could move a part of value to a new line, which could cause some parsers fail to parse. Using this style is not recommended. It was switched to the literal style when relevant and manifests were updated. To make sure next updates are good, the helper script to fix manifests is a part of the script to update manifests. It's in Python, hope that's okay as it doesn't get added to images.

There's one weird value for haproxy.cfg which is rendered as a quotes-enclosed string, but it was correct when I tried to load yamls from upstream and updated ones. In any case, it's more readable now.

Let's cherry pick to 2.13, as the apply of install manifests is currently broken.

Signed-off-by: Andrii Korotkov <[email protected]>
@crenshaw-dev
Copy link
Member

@xelab04 the problem is that you're applying the GitHub page for install.yaml, not install.yaml itself.

To demonstrate the problem, try this:

wget https://github.com/argoproj/argo-cd/blob/master/manifests/ha/install.yaml
kubectl apply -n argocd -f install.yaml
less install.yaml

The first two commands should produce the same error as what you got. The last command should show that the file you're trying to apply is HTML rather than YAML.

Use the raw install.yaml link:

kubectl apply -n argocd https://raw.githubusercontent.com/argoproj/argo-cd/refs/heads/master/manifests/ha/install.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants
@crenshaw-dev @xelab04 @MaroIT @andrii-korotkov-verkada and others