Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upgrade es and kb to version 8 #3598

Merged
merged 2 commits into from
Nov 19, 2024
Merged

upgrade es and kb to version 8 #3598

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
af72eab
upgrade es and kb to version 8
vara2504 Nov 14, 2024
f1b9f62
Fix make check dirty
vara2504 Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions config/enterprise_versions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@ components:
image: tigera/dex
version: master
eck-kibana:
version: 7.17.25
version: 8.15.3
kibana:
image: tigera/kibana
version: master
eck-elasticsearch:
version: 7.17.25
version: 8.15.3
elasticsearch:
image: tigera/elasticsearch
version: master
Expand Down
4 changes: 2 additions & 2 deletions pkg/components/enterprise.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,12 @@ var (
}

ComponentEckElasticsearch = Component{
Version: "7.17.25",
Version: "8.15.3",
Registry: "",
}

ComponentEckKibana = Component{
Version: "7.17.25",
Version: "8.15.3",
Registry: "",
}

Expand Down
29 changes: 28 additions & 1 deletion pkg/crds/calico/crd.projectcalico.org_felixconfigurations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,25 @@ spec:
for debugging purposes. \n Deprecated: Use BPFConnectTimeLoadBalancing
[Default: true]"
type: boolean
bpfConntrackLogLevel:
description: 'BPFConntrackLogLevel controls the log level of the BPF
conntrack cleanup program, which runs periodically to clean up expired
BPF conntrack entries. [Default: Off].'
enum:
- "Off"
- Debug
type: string
bpfConntrackMode:
description: 'BPFConntrackCleanupMode controls how BPF conntrack entries
are cleaned up. `Auto` will use a BPF program if supported, falling
back to userspace if not. `Userspace` will always use the userspace
cleanup code. `BPFProgram` will always use the BPF program (failing
if not supported). [Default: Auto]'
enum:
- Auto
- Userspace
- BPFProgram
type: string
bpfDSROptoutCIDRs:
description: BPFDSROptoutCIDRs is a list of CIDRs which are excluded
from DSR. That is, clients in those CIDRs will access service node
Expand All @@ -95,7 +114,8 @@ spec:
that Calico workload traffic flows over as well as any interfaces
that handle incoming traffic to nodeports and services from outside
the cluster. It should not match the workload interfaces (usually
named cali...).
named cali...) or any other special device managed by Calico itself
(e.g., tunnels).
type: string
bpfDisableGROForIfaces:
description: BPFDisableGROForIfaces is a regular expression that controls
Expand Down Expand Up @@ -217,6 +237,13 @@ spec:
connection. Warning: changing the size of the conntrack map can
cause disruption.'
type: integer
bpfMapSizeConntrackCleanupQueue:
description: BPFMapSizeConntrackCleanupQueue sets the size for the
map used to hold NAT conntrack entries that are queued for cleanup. This
should be big enough to hold all the NAT entries that expire within
one cleanup interval.
minimum: 1
type: integer
bpfMapSizeIPSets:
description: BPFMapSizeIPSets sets the size for ipsets map. The IP
sets map must be large enough to hold an entry for each endpoint
Expand Down
23 changes: 11 additions & 12 deletions pkg/render/logstorage.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ import (
"github.com/tigera/operator/pkg/common"
"github.com/tigera/operator/pkg/components"
"github.com/tigera/operator/pkg/dns"
"github.com/tigera/operator/pkg/ptr"
relasticsearch "github.com/tigera/operator/pkg/render/common/elasticsearch"
rmeta "github.com/tigera/operator/pkg/render/common/meta"
"github.com/tigera/operator/pkg/render/common/networkpolicy"
Expand Down Expand Up @@ -358,15 +359,10 @@ func (es *elasticsearchComponent) podTemplate() corev1.PodTemplateSpec {
},
}

sc := securitycontext.NewRootContext(false)
// These capabilities are required for docker-entrypoint.sh.
// See: https://github.com/elastic/elasticsearch/blob/7.17/distribution/docker/src/docker/bin/docker-entrypoint.sh.
// TODO Consider removing for Elasticsearch v8+.
sc.Capabilities.Add = []corev1.Capability{
"SETGID",
"SETUID",
"SYS_CHROOT",
}
sc := securitycontext.NewNonRootContext()
// Set the user and group to be the default elasticsearch ID
sc.RunAsUser = ptr.Int64ToPtr(1000)
Copy link
Member

@rene-dekker rene-dekker Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, the init containers still need to run as root to:

  • set os settings
  • chown the file system

and so the namespace still needs to have the privileged context. The main container now runs as 1000:1000, which is the es user. This is also the user the prepareFS script will chown to.

Is my understanding correct? Have we considered changing the user to 10000:10000 (our default user)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this SecurityContext is for the main ElasticSearch container. The SecurityContext for each init container still uses the root context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Init container "elastic-internal-init-filesystem" still run as root because we need to set the vm max map count "echo 262144 > /proc/sys/vm/max_map_count". This change is only for elasticsearch container.
Currently, Elasticsearch 7, we override the security context of the Elasticsearch container to run as the root user and group. The docker-entrypoint.sh script in Elasticsearch contained logic to switch to user 1000 when it encountered a root user. However, this logic has been removed in Elasticsearch 8. Since the ownership of the /data folder is set to the default Elasticsearch UID, we are encountering an unwritable error when running as the root user.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We spoke offline about a potential follow-up to explore using uid:gid of 10001:10001, but continue with the PR in its current form.

sc.RunAsGroup = ptr.Int64ToPtr(1000)

esContainer := corev1.Container{
Name: "elasticsearch",
Expand Down Expand Up @@ -717,9 +713,12 @@ func (es *elasticsearchComponent) nodeSets() []esv1.NodeSet {
// NodeSet
func (es *elasticsearchComponent) nodeSetTemplate(pvcTemplate corev1.PersistentVolumeClaim) esv1.NodeSet {
config := map[string]interface{}{
"node.master": "true",
"node.data": "true",
"node.ingest": "true",
"node.roles": []string{
"data",
"ingest",
"master",
"remote_cluster_client",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove this role?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might need these role - data, ingest, master same as the previous version.
Reg:remote_cluster_client : if we are setting node.roles, then remote_cluster_client role must be explicitly included to enable cross-cluster search. Otherwise, this could result in an access denied issue when loading the Stack Monitoring page in Kibana.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:/ so be it...

},
"cluster.max_shards_per_node": 10000,
// Disable geoip downloader. This removes an error from the startup logs, because our network policy blocks it.
"ingest.geoip.downloader.enabled": false,
Expand Down
55 changes: 22 additions & 33 deletions pkg/render/logstorage_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,13 +217,12 @@ var _ = Describe("Elasticsearch rendering tests", func() {
esContainer := resultES.Spec.NodeSets[0].PodTemplate.Spec.Containers[0]
Expect(*esContainer.SecurityContext.AllowPrivilegeEscalation).To(BeFalse())
Expect(*esContainer.SecurityContext.Privileged).To(BeFalse())
Expect(*esContainer.SecurityContext.RunAsGroup).To(BeEquivalentTo(0))
Expect(*esContainer.SecurityContext.RunAsNonRoot).To(BeFalse())
Expect(*esContainer.SecurityContext.RunAsUser).To(BeEquivalentTo(0))
Expect(*esContainer.SecurityContext.RunAsGroup).To(BeEquivalentTo(1000))
Expect(*esContainer.SecurityContext.RunAsNonRoot).To(BeTrue())
Expect(*esContainer.SecurityContext.RunAsUser).To(BeEquivalentTo(1000))
Expect(esContainer.SecurityContext.Capabilities).To(Equal(
&corev1.Capabilities{
Drop: []corev1.Capability{"ALL"},
Add: []corev1.Capability{"SETGID", "SETUID", "SYS_CHROOT"},
},
))
Expect(esContainer.SecurityContext.SeccompProfile).To(Equal(
Expand All @@ -242,9 +241,7 @@ var _ = Describe("Elasticsearch rendering tests", func() {

// Check that the expected config made it's way to the Elastic CR
Expect(nodeSet.Config.Data).Should(Equal(map[string]interface{}{
"node.master": "true",
"node.data": "true",
"node.ingest": "true",
"node.roles": []string{"data", "ingest", "master", "remote_cluster_client"},
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
}))
Expand Down Expand Up @@ -970,12 +967,10 @@ var _ = Describe("Elasticsearch rendering tests", func() {
},
}))
Expect(nodeSets[0].Config.Data).Should(Equal(map[string]interface{}{
"node.master": "true",
"node.data": "true",
"node.ingest": "true",
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2a",
"node.roles": []string{"data", "ingest", "master", "remote_cluster_client"},
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2a",
"cluster.routing.allocation.awareness.attributes": "zone",
}))

Expand All @@ -991,12 +986,10 @@ var _ = Describe("Elasticsearch rendering tests", func() {
},
}))
Expect(nodeSets[1].Config.Data).Should(Equal(map[string]interface{}{
"node.master": "true",
"node.data": "true",
"node.ingest": "true",
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2b",
"node.roles": []string{"data", "ingest", "master", "remote_cluster_client"},
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2b",
"cluster.routing.allocation.awareness.attributes": "zone",
}))
})
Expand Down Expand Up @@ -1063,13 +1056,11 @@ var _ = Describe("Elasticsearch rendering tests", func() {
},
}))
Expect(nodeSets[0].Config.Data).Should(Equal(map[string]interface{}{
"node.master": "true",
"node.data": "true",
"node.ingest": "true",
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2a",
"node.attr.rack": "rack1",
"node.roles": []string{"data", "ingest", "master", "remote_cluster_client"},
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2a",
"node.attr.rack": "rack1",
"cluster.routing.allocation.awareness.attributes": "zone,rack",
}))

Expand All @@ -1094,13 +1085,11 @@ var _ = Describe("Elasticsearch rendering tests", func() {
},
}))
Expect(nodeSets[1].Config.Data).Should(Equal(map[string]interface{}{
"node.master": "true",
"node.data": "true",
"node.ingest": "true",
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2b",
"node.attr.rack": "rack1",
"node.roles": []string{"data", "ingest", "master", "remote_cluster_client"},
"cluster.max_shards_per_node": 10000,
"ingest.geoip.downloader.enabled": false,
"node.attr.zone": "us-west-2b",
"node.attr.rack": "rack1",
"cluster.routing.allocation.awareness.attributes": "zone,rack",
}))
})
Expand Down
Loading