✨ Configurable qos resources for deployments managed by clustermanager and klusterlet #316

dongbeiqing91 · 2024-01-18T06:24:36Z

Summary

Support configurable qos resources for deployments managed by clustermanager and klusterlet because "Default" deployment resources w/ only hard coded "resources" but no "limits" is not good enough and "BestEffort" is usually not allowed in production.

Related issue(s)

Fixes #

dongbeiqing91 · 2024-01-18T06:25:12Z

/assign @qiujian16

qiujian16 · 2024-01-18T06:47:59Z

operator/v1/types_klusterlet.go

 )

 // ResourceRequirement allow user override the default pod QoS classes
 type ResourceRequirement struct {
 	// +kubebuilder:validation:Enum=Default;BestEffort
 	// +kubebuilder:default:=Default
 	Type ResourceQosClass `json:"type"`
+	// Resources defines resource requests and limits when Type is ResourceQosClassConfigurable
+	// +optional
+	Resources *corev1.ResourceList `json:"resources,omitempty"`


You would need to add ResourceQosClassConfigurable in the // +kubebuilder:validation:Enum=Default;BestEffort for type.

I would suggest the type is ResourceRequirements and the field name should be ResourceRequirements too.

Added ResourceQosClassConfigurable in the validation comment

I would suggest the type is ResourceRequirements and the field name should be ResourceRequirements too

Do you mean changing

ResourceRequirement *ResourceRequirement `json:"resourceRequirement,omitempty"`

to

ResourceRequirements *ResourceRequirements `json:"resourceRequirement,omitempty"`

But json:"resourceRequirement,omitempty" has to stay unchanged, right?

ResourceRequirement *ResourceRequirement json:"resourceRequirement,omitempty" is ok

qiujian16 · 2024-01-18T06:50:46Z

operator/v1/types_klusterlet.go

 )

 // ResourceRequirement allow user override the default pod QoS classes
 type ResourceRequirement struct {
 	// +kubebuilder:validation:Enum=Default;BestEffort
 	// +kubebuilder:default:=Default
 	Type ResourceQosClass `json:"type"`
+	// Resources defines resource requests and limits when Type is ResourceQosClassConfigurable
+	// +optional
+	Resources *corev1.ResourceList `json:"resources,omitempty"`


I do not think *corev1.ResourceList is the correct type. What you want is https://github.com/kubernetes/api/blob/master/core/v1/types.go#L2547?

Yes you are right. Fixed.

qiujian16 · 2024-01-18T06:53:21Z

operator/v1/types_clustermanager.go

+
+	// ResourceRequirement specify QoS classes of deployments managed by clustermanager
+	// +optional
+	ResourceRequirement *ResourceRequirement `json:"resourceRequirement,omitempty"`


it should be noted that it applies to all containers since there are multiple containers running in the cluster manager.

comment added

zhiweiyin318 · 2024-01-18T07:00:19Z

the deepcopy go file is not updated, is there any error durning running the make update?

dongbeiqing91 · 2024-01-18T07:16:40Z

the deepcopy go file is not updated, is there any error durning running the make update?

It indeeded had some errors I didn't pay attention to. Let me check and fix it

dongbeiqing91 · 2024-01-18T07:34:07Z

the deepcopy go file is not updated, is there any error durning running the make update?

The project path was wrong. I have fixed it. deepcopy is correctly generated now.

qiujian16 · 2024-01-18T07:34:37Z

operator/v1/types_klusterlet.go

 	ResourceQosClassDefault ResourceQosClass = "Default"
 	// If all containers in the pod don't set resource request and limits, the pod is treated as BestEffort.
 	ResourceQosClassBestEffort ResourceQosClass = "BestEffort"
+	// Configurable resources with requests and limits
+	ResourceQosClassConfigurable ResourceQosClass = "Configurable"


the type name should be ResourceRequirement

ResourceQosClassResourceRequirement ResourceQosClass = "ResourceRequirement"

qiujian16 · 2024-01-18T07:52:29Z

operator/v1/types_klusterlet.go

 	// +kubebuilder:default:=Default
 	Type ResourceQosClass `json:"type"`
+	// Resources defines resource requests and limits when Type is ResourceQosClassResourceRequirement
+	// +optional
+	Resources *corev1.ResourceRequirements `json:"resources,omitempty"`


Suggested change

Resources *corev1.ResourceRequirements `json:"resources,omitempty"`

ResourceRequirement *corev1.ResourceRequirements `json:"resourceRequirement,omitempty"`

if this change is made, the yaml will look like:

resourceRequirement: resourceRequirement: limits: ...

duplicated resourceRequirements look weild I suppose

In kubernetes Container struct, it is

Resources ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,8,opt,name=resources"`

https://github.com/kubernetes/api/blob/master/core/v1/types.go#L2682

it is intended to follow the union pattern https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/1027-api-unions/README.md. So either the type should be Resources or the filed name should be ResourceRequirement

zhiweiyin318 · 2024-01-18T14:49:41Z

the DCO Check requires a Signed-off-by trailer in the commit log. so you need to add -s when run git commit.

dongbeiqing91 · 2024-01-19T01:39:28Z

the DCO Check requires a Signed-off-by trailer in the commit log. so you need to add -s when run git commit.

Done

Signed-off-by: Dong Beiqing <[email protected]>

qiujian16 · 2024-01-22T08:41:15Z

/approve
/lgtm

openshift-ci · 2024-01-22T08:41:22Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: promid, qiujian16

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [qiujian16]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot requested review from deads2k and mdelder January 18, 2024 06:24

openshift-ci bot assigned qiujian16 Jan 18, 2024

dongbeiqing91 changed the title ~~configurable qos resources for deployments managed by clustermanager and klusterlet~~ ✨ Configurable qos resources for deployments managed by clustermanager and klusterlet Jan 18, 2024

qiujian16 reviewed Jan 18, 2024

View reviewed changes

dongbeiqing91 force-pushed the dev/qos branch from 6c35afe to 9fa5b15 Compare January 18, 2024 07:01

dongbeiqing91 force-pushed the dev/qos branch from 9fa5b15 to cc2b204 Compare January 18, 2024 07:31

qiujian16 reviewed Jan 18, 2024

View reviewed changes

dongbeiqing91 force-pushed the dev/qos branch from cc2b204 to f0a8b03 Compare January 18, 2024 07:39

qiujian16 reviewed Jan 18, 2024

View reviewed changes

dongbeiqing91 force-pushed the dev/qos branch from fd5489a to f0a8b03 Compare January 18, 2024 08:37

dongbeiqing91 force-pushed the dev/qos branch from f0a8b03 to 3e5a4c0 Compare January 19, 2024 01:29

configurable qos resources for clustermanager and klusterlet deployments

24a2a0e

Signed-off-by: Dong Beiqing <[email protected]>

dongbeiqing91 force-pushed the dev/qos branch 2 times, most recently from e303165 to ee4bb4c Compare January 19, 2024 08:00

dongbeiqing91 mentioned this pull request Jan 19, 2024

✨ Configurable qos resources for containers managed by cluster-manager and klusterlet open-cluster-management-io/ocm#351

Merged

dongbeiqing91 force-pushed the dev/qos branch 2 times, most recently from b15428a to 15ae4da Compare January 19, 2024 08:54

move ResourceRequirement to a separate file

25e246a

Signed-off-by: Dong Beiqing <[email protected]>

dongbeiqing91 force-pushed the dev/qos branch from 15ae4da to 25e246a Compare January 22, 2024 01:51

openshift-ci bot added the lgtm label Jan 22, 2024

openshift-ci bot added the approved label Jan 22, 2024

openshift-merge-bot bot merged commit e7bd1bd into open-cluster-management-io:main Jan 22, 2024
10 checks passed

dongbeiqing91 mentioned this pull request Jan 30, 2024

✨ Configurable QoS resources for cluster-manager and klusterlet operators and their managed containers open-cluster-management-io/clusteradm#400

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Configurable qos resources for deployments managed by clustermanager and klusterlet #316

✨ Configurable qos resources for deployments managed by clustermanager and klusterlet #316

dongbeiqing91 commented Jan 18, 2024

dongbeiqing91 commented Jan 18, 2024

qiujian16 Jan 18, 2024

dongbeiqing91 Jan 18, 2024 •

edited

Loading

qiujian16 Jan 18, 2024

qiujian16 Jan 18, 2024

dongbeiqing91 Jan 18, 2024

qiujian16 Jan 18, 2024

dongbeiqing91 Jan 18, 2024

zhiweiyin318 commented Jan 18, 2024

dongbeiqing91 commented Jan 18, 2024

dongbeiqing91 commented Jan 18, 2024

qiujian16 Jan 18, 2024 •

edited

Loading

dongbeiqing91 Jan 18, 2024

qiujian16 Jan 18, 2024

dongbeiqing91 Jan 18, 2024

dongbeiqing91 Jan 18, 2024 •

edited

Loading

qiujian16 Jan 19, 2024 •

edited

Loading

dongbeiqing91 Jan 19, 2024

zhiweiyin318 commented Jan 18, 2024

dongbeiqing91 commented Jan 19, 2024

qiujian16 commented Jan 22, 2024

openshift-ci bot commented Jan 22, 2024

	Resources *corev1.ResourceRequirements `json:"resources,omitempty"`
	ResourceRequirement *corev1.ResourceRequirements `json:"resourceRequirement,omitempty"`

✨ Configurable qos resources for deployments managed by clustermanager and klusterlet #316

✨ Configurable qos resources for deployments managed by clustermanager and klusterlet #316

Conversation

dongbeiqing91 commented Jan 18, 2024

Summary

Related issue(s)

dongbeiqing91 commented Jan 18, 2024

Choose a reason for hiding this comment

dongbeiqing91 Jan 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiweiyin318 commented Jan 18, 2024

dongbeiqing91 commented Jan 18, 2024

dongbeiqing91 commented Jan 18, 2024

qiujian16 Jan 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dongbeiqing91 Jan 18, 2024 • edited Loading

Choose a reason for hiding this comment

qiujian16 Jan 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiweiyin318 commented Jan 18, 2024

dongbeiqing91 commented Jan 19, 2024

qiujian16 commented Jan 22, 2024

openshift-ci bot commented Jan 22, 2024

dongbeiqing91 Jan 18, 2024 •

edited

Loading

qiujian16 Jan 18, 2024 •

edited

Loading

dongbeiqing91 Jan 18, 2024 •

edited

Loading

qiujian16 Jan 19, 2024 •

edited

Loading