Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCP Pub/Sub Publisher Issue: ManagedChannel isn't shutdown properly #2045

Open
ankurg03 opened this issue May 27, 2024 · 1 comment
Open
Assignees
Labels
api: pubsub Issues related to the googleapis/java-pubsub API.

Comments

@ankurg03
Copy link

We are using Debezium version 2.5 to capture SQL CDC (Change Data Capture) events, which are then published to Google Cloud Pub/Sub.
This uses the PubSubChangeConsumer for this purpose. However, we encountered an issue where a timeout occurred during the publish event, leading to an instruction to shut down the Debezium process.

The shutdown call got stuck because of an infinite wait on the wait() method inside the waitComplete method of Waiter class.

Implementing a time-bounded wait call may resolve this issue.

Please find the thread dump of a waiting thread.

java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)

  • waiting on
    at java.lang.Object.wait([email protected]/Object.java:328)
    at com.google.cloud.pubsub.v1.Waiter.waitComplete(Waiter.java:44)
  • waiting to re-lock in wait() <0x00000000d26dc200> (a com.google.cloud.pubsub.v1.Waiter)
    at com.google.cloud.pubsub.v1.Publisher.shutdown(Publisher.java:617)
    at io.debezium.server.pubsub.PubSubChangeConsumer.lambda$close$1(PubSubChangeConsumer.java:210)
    at io.debezium.server.pubsub.PubSubChangeConsumer$$Lambda$1156/0x00000008401aac40.accept(Unknown Source)
    at java.util.HashMap$Values.forEach([email protected]/HashMap.java:977)
    at io.debezium.server.pubsub.PubSubChangeConsumer.close(PubSubChangeConsumer.java:208)
    at io.debezium.server.pubsub.PubSubChangeConsumer_Bean.doDestroy(Unknown Source)
    at io.debezium.server.pubsub.PubSubChangeConsumer_Bean.destroy(Unknown Source)
    at io.debezium.server.pubsub.PubSubChangeConsumer_Bean.destroy(Unknown Source)
    at io.debezium.server.DebeziumServer.stop(DebeziumServer.java:242)
    at io.debezium.server.DebeziumServer_Observer_stop_d470e05516c8d86f955c7275c00cebdc903562ab.notify(Unknown Source)
    at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:346)
    at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:328)
    at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:82)
    at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:155)
    at io.quarkus.arc.runtime.ArcRecorder$2.run(ArcRecorder.java:111)
    at io.quarkus.runtime.StartupContext.runAllInReverseOrder(StartupContext.java:84)
    at io.quarkus.runtime.StartupContext.close(StartupContext.java:73)
    at io.quarkus.runner.ApplicationImpl.doStop(Unknown Source)
    at io.quarkus.runtime.Application.stop(Application.java:208)
    at io.quarkus.runtime.Application.stop(Application.java:155)
    at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:227)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
    at io.debezium.server.Main.main(Main.java:15)
@product-auto-label product-auto-label bot added the api: pubsub Issues related to the googleapis/java-pubsub API. label May 27, 2024
@michaelpri10
Copy link
Contributor

Hello! I've been attempting to reproduce this issue but have been unsuccessful. Would you be able to provide a code snippet that would allow us to test and reproduce this issue?

@ankurg03 ankurg03 changed the title GCP Pub/Sub Publisher Issue: ManagedChannel Not Shut Down Properly GCP Pub/Sub Publisher Issue: ManagedChannel isn't shutdown properly Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the googleapis/java-pubsub API.
Projects
None yet
Development

No branches or pull requests

2 participants