Skip to content

Commit

Permalink
merge main
Browse files Browse the repository at this point in the history
  • Loading branch information
wind57 committed Dec 22, 2023
2 parents 783446a + f6707d5 commit 9ce53b2
Show file tree
Hide file tree
Showing 31 changed files with 1,246 additions and 57 deletions.
61 changes: 46 additions & 15 deletions docs/modules/ROOT/pages/discovery-client.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -222,31 +222,62 @@ NOTE: `spring.application.name` has no effect as far as the name registered for

'''

Spring Cloud Kubernetes can also watch the Kubernetes service catalog for changes and update the
`DiscoveryClient` implementation accordingly. By "watch" we mean that we will publish a heartbeat event every `spring.cloud.kubernetes.discovery.catalog-services-watch-delay`
milliseconds (by default it is `30000`). The heartbeat event will contain the target references (and their namespaces of the addresses of all endpoints
Spring Cloud Kubernetes can also watch the Kubernetes service catalog for changes and update the `DiscoveryClient` implementation accordingly. In order to enable this functionality you need to add
`@EnableScheduling` on a configuration class in your application. By "watch", we mean that we will publish a heartbeat event every `spring.cloud.kubernetes.discovery.catalog-services-watch-delay`
milliseconds (by default it is `30000`). For the http discovery server this must be an environment variable set in deployment yaml:

----
containers:
- name: discovery-server
image: springcloud/spring-cloud-kubernetes-discoveryserver:3.0.5-SNAPSHOT
env:
- name: SPRING_CLOUD_KUBERNETES_DISCOVERY_CATALOGSERVICESWATCHDELAY
value: 3000
----

The heartbeat event will contain the target references (and their namespaces of the addresses of all endpoints
(for the exact details of what will get returned you can take a look inside `KubernetesCatalogWatch`). This is an implementation detail, and listeners of the heartbeat event
should not rely on the details. Instead, they should see if there are differences between two subsequent heartbeats via `equals` method. We will take care to return a correct implementation that adheres to the equals contract.
The endpoints will be queried in either :
- all namespaces (enabled via `spring.cloud.kubernetes.discovery.all-namespaces=true`)
- `all-namespaces` (enabled via `spring.cloud.kubernetes.discovery.all-namespaces=true`)

- `selective namespaces` (enabled via `spring.cloud.kubernetes.discovery.namespaces`), for example:

- `one namespace` via xref:property-source-config.adoc#namespace-resolution[Namespace Resolution] if the above two paths are not taken.

- specific namespaces (enabled via `spring.cloud.kubernetes.discovery.namespaces`), for example:
NOTE: If, for any reasons, you want to disable catalog watcher, you need to set `spring.cloud.kubernetes.discovery.catalog-services-watch.enabled=false`. For the http discovery server, this needs to be an environment variable set in deployment for example:

[source]
----
spring:
cloud:
kubernetes:
discovery:
namespaces:
- namespace-a
- namespace-b
SPRING_CLOUD_KUBERNETES_DISCOVERY_CATALOGSERVICESWATCH_ENABLED=FALSE
----

- we will use: xref:property-source-config.adoc#namespace-resolution[Namespace Resolution] if the above two paths are not taken.
The functionality of catalog watch works for all 3 discovery clients that we support, with some caveats that you need to be aware of in case of the http client.

- The first is that this functionality is disabled by default, and it needs to be enabled in two places:

* in discovery server via an environment variable in the deployment manifest, for example:
+
----
containers:
- name: discovery-server
image: springcloud/spring-cloud-kubernetes-discoveryserver:3.0.5-SNAPSHOT
env:
- name: SPRING_CLOUD_KUBERNETES_HTTP_DISCOVERY_CATALOG_WATCHER_ENABLED
value: "TRUE"
----
+

* in discovery client, via a property in your `application.properties` for example:
+
----
spring.cloud.kubernetes.http.discovery.catalog.watcher.enabled=true
----
+

In order to enable this functionality you need to add
`@EnableScheduling` on a configuration class in your application.
- The second point is that this is only supported since version `3.0.6` and upwards.
- Since http discovery has _two_ components : server and client, we strongly recommend to align versions between them, otherwise things might not work.
- If you decide to disable catalog watcher, you need to disable it in both server and client.

By default, we use the `Endpoints`(see https://kubernetes.io/docs/concepts/services-networking/service/#endpoints) API to find out the current state of services. There is another way though, via `EndpointSlices` (https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/). Such support can be enabled via a property: `spring.cloud.kubernetes.discovery.use-endpoint-slices=true` (by default it is `false`). Of course, your cluster has to support it also. As a matter of fact, if you enable this property, but your cluster does not support it, we will fail starting the application. If you decide to enable such support, you also need proper Role/ClusterRole set-up. For example:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,16 @@ public static ApiClient kubernetesApiClient() {
return apiClient;
}
catch (Exception e) {
LOG.info("Could not create the Kubernetes ApiClient in a cluster environment, because : ", e);
LOG.info("Trying to use a \"standard\" configuration to create the Kubernetes ApiClient");
if (e instanceof IllegalStateException illegalStateException
&& illegalStateException.getCause() instanceof NumberFormatException) {
LOG.info("Could not create the Kubernetes ApiClient in a cluster environment, because connection port " +
"was not provided.");
}
else {
LOG.info("Could not create the Kubernetes ApiClient in a cluster environment, because : ", e);
}
LOG.info("""
Trying to use a "standard" configuration to create the Kubernetes ApiClient""");
try {
ApiClient apiClient = ClientBuilder.defaultClient();
LOG.info("Created standard API client. Unless $KUBECONFIG or $HOME/.kube/config is defined, "
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import io.kubernetes.client.openapi.apis.CoreV1Api;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;

import org.springframework.boot.DefaultBootstrapContext;
Expand All @@ -31,6 +32,8 @@
import org.springframework.boot.context.properties.bind.Binder;
import org.springframework.boot.context.properties.source.ConfigurationPropertySources;
import org.springframework.boot.logging.DeferredLogFactory;
import org.springframework.boot.test.system.CapturedOutput;
import org.springframework.boot.test.system.OutputCaptureExtension;
import org.springframework.cloud.kubernetes.commons.KubernetesClientProperties;
import org.springframework.cloud.kubernetes.commons.config.ConfigDataRetryableConfigMapPropertySourceLocator;
import org.springframework.cloud.kubernetes.commons.config.ConfigDataRetryableSecretsPropertySourceLocator;
Expand All @@ -43,6 +46,7 @@
/**
* @author wind57
*/
@ExtendWith(OutputCaptureExtension.class)
class KubernetesClientConfigDataLocationResolverTests {

private static final DeferredLogFactory FACTORY = Supplier::get;
Expand Down Expand Up @@ -142,7 +146,7 @@ void testBothPresent() {
* these are not retryable beans.
*/
@Test
void testBothPresentExplicitly() {
void testBothPresentExplicitly(CapturedOutput capturedOutput) {
MockEnvironment environment = new MockEnvironment();
environment.setProperty("spring.cloud.kubernetes.config.enabled", "true");
environment.setProperty("spring.cloud.kubernetes.secrets.enabled", "true");
Expand Down Expand Up @@ -172,6 +176,10 @@ void testBothPresentExplicitly() {
SecretsPropertySourceLocator secretsPropertySourceLocator = context.get(SecretsPropertySourceLocator.class);
Assertions.assertSame(KubernetesClientSecretsPropertySourceLocator.class,
secretsPropertySourceLocator.getClass());

Assertions.assertTrue(capturedOutput.getOut()
.contains("Could not create the Kubernetes ApiClient in a cluster environment, because connection port "
+ "was not provided."));
}

/*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
import org.springframework.core.log.LogAccessor;
import org.springframework.scheduling.annotation.Scheduled;

import static org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryConstants.CATALOG_WATCH_PROPERTY_WITH_DEFAULT_VALUE;
import static org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryConstants.DISCOVERY_GROUP;
import static org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryConstants.DISCOVERY_VERSION;
import static org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryConstants.ENDPOINT_SLICE;
Expand Down Expand Up @@ -67,7 +68,7 @@ public void setApplicationEventPublisher(ApplicationEventPublisher publisher) {
this.publisher = publisher;
}

@Scheduled(fixedDelayString = "${spring.cloud.kubernetes.discovery.catalogServicesWatchDelay:30000}")
@Scheduled(fixedDelayString = "${" + CATALOG_WATCH_PROPERTY_WITH_DEFAULT_VALUE + "}")
void catalogServicesWatch() {
try {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,10 @@
import io.kubernetes.client.openapi.apis.CoreV1Api;

import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnCloudPlatform;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.cloud.CloudPlatform;
import org.springframework.cloud.client.ConditionalOnDiscoveryEnabled;
import org.springframework.cloud.kubernetes.client.KubernetesClientAutoConfiguration;
import org.springframework.cloud.kubernetes.commons.KubernetesNamespaceProvider;
import org.springframework.cloud.kubernetes.commons.discovery.ConditionalOnKubernetesCatalogEnabled;
import org.springframework.cloud.kubernetes.commons.discovery.ConditionalOnKubernetesCatalogWatcherEnabled;
import org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryProperties;
import org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryPropertiesAutoConfiguration;
import org.springframework.context.annotation.Bean;
Expand All @@ -39,9 +36,7 @@
* @author wind57
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnDiscoveryEnabled
@ConditionalOnKubernetesCatalogEnabled
@ConditionalOnCloudPlatform(CloudPlatform.KUBERNETES)
@ConditionalOnKubernetesCatalogWatcherEnabled
@AutoConfigureAfter({ KubernetesClientAutoConfiguration.class, KubernetesDiscoveryPropertiesAutoConfiguration.class })
class KubernetesCatalogWatchAutoConfiguration {

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Copyright 2019-2023 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.kubernetes.commons.discovery;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;

/**
* Provides a more succinct conditional for:
* <code>spring.cloud.kubernetes.http.discovery.client.catalog.watcher.enabled</code>.
*
* @author wind57
*/
@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@ConditionalOnProperty(value = "spring.cloud.kubernetes.http.discovery.catalog.watcher.enabled", matchIfMissing = false)
public @interface ConditionalOnHttpDiscoveryCatalogWatcherEnabled {

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
* Copyright 2013-2023 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.kubernetes.commons.discovery;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.springframework.boot.autoconfigure.condition.ConditionalOnCloudPlatform;
import org.springframework.boot.cloud.CloudPlatform;
import org.springframework.cloud.client.ConditionalOnDiscoveryEnabled;

/**
* Provides common conditionals to be used for catalog watcher.
*
* @author wind57
*/
@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@ConditionalOnDiscoveryEnabled
@ConditionalOnKubernetesCatalogEnabled
@ConditionalOnCloudPlatform(CloudPlatform.KUBERNETES)
public @interface ConditionalOnKubernetesCatalogWatcherEnabled {

}
Original file line number Diff line number Diff line change
Expand Up @@ -82,4 +82,20 @@ private KubernetesDiscoveryConstants() {
*/
public static final String SECURED = "secured";

/**
* catalog watch delay property name.
*/
public static final String CATALOG_WATCH_PROPERTY_NAME = "spring.cloud.kubernetes.discovery.catalogServicesWatchDelay";

/**
* default delay for the configuration watcher.
*/
public static final String CATALOG_WATCHER_DEFAULT_DELAY = "30000";

/**
* catalog watch delay property name with default value.
*/
public static final String CATALOG_WATCH_PROPERTY_WITH_DEFAULT_VALUE = CATALOG_WATCH_PROPERTY_NAME + ":"
+ CATALOG_WATCHER_DEFAULT_DELAY;

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
* Copyright 2013-2023 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.kubernetes.discoveryserver;

import java.util.List;

import reactor.core.publisher.Mono;

import org.springframework.cloud.kubernetes.commons.discovery.ConditionalOnHttpDiscoveryCatalogWatcherEnabled;
import org.springframework.cloud.kubernetes.commons.discovery.ConditionalOnKubernetesCatalogEnabled;
import org.springframework.cloud.kubernetes.commons.discovery.EndpointNameAndNamespace;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

/**
* @author wind57
*/
@RestController
@ConditionalOnKubernetesCatalogEnabled
@ConditionalOnHttpDiscoveryCatalogWatcherEnabled
class DiscoveryCatalogWatcherController {

private final HeartBeatListener heartBeatListener;

DiscoveryCatalogWatcherController(HeartBeatListener heartBeatListener) {
this.heartBeatListener = heartBeatListener;
}

@GetMapping("/state")
Mono<List<EndpointNameAndNamespace>> state() {
return Mono.defer(() -> Mono.just(heartBeatListener.lastState().get()));
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,13 @@

import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.scheduling.annotation.EnableScheduling;

/**
* @author Ryan Baxter
*/
@SpringBootApplication
@EnableScheduling
public class DiscoveryServerApplication {

public static void main(String[] args) {
Expand Down
Loading

0 comments on commit 9ce53b2

Please sign in to comment.