cppkafka::Consumer hangs on thread join in destructor

[ydsun90]

I use cppkafka 0.3.1 and librdkafka 2.2.0. Sometimes the cppkafka::Consumer hangs in destructor. The stack traces show one of the threads is waiting on an internal thread to join.

Thread 27 (Thread 0x7f7fa7a55700 (LWP 66854)):
#0 0x00007f7fab0bc017 in pthread_join () from /usr/lib64/libpthread.so.0
#1 0x00000000005bed12 in thrd_join (thr=thr@entry=140185233770240, res=res@entry=0x7f7fa7a3d2dc) at tinycthread.c:692
#2 0x00000000005157d7 in rd_kafka_destroy_app (rk=0x7f7f679e3ff0, flags=) at rdkafka.c:1109
#3 0x00007f7fbdcfc1fa in ~unique_ptr (this=0x7f7f67985f98, __in_chrg=) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/move.h:99
#4 ~KafkaHandleBase (this=0x7f7f67985dd0, __in_chrg=) at /agent_home/exposure/var/conan.home.RelWithDebInfo/.conan/storage/cppkafka/0.3.1///build/5e493e26de4dea68af41b6bfadef6d8d5c0f572a/source_subfolder/src/../include/cppkafka/kafka_handle_base.h:66
#5 cppkafka::Consumer::~Consumer() () at /agent_home/exposure/var/conan.home.RelWithDebInfo/.conan/storage/cppkafka/0.3.1///build/5e493e26de4dea68af41b6bfadef6d8d5c0f572a/source_subfolder/src/consumer.cpp:75
#6 0x00007f7fbdcfc429 in cppkafka::Consumer::~Consumer (this=0x7f7f67985dd0, __in_chrg=) at /agent_home/exposure/var/conan.home.RelWithDebInfo/.conan/storage/cppkafka/0.3.1///build/5e493e26de4dea68af41b6bfadef6d8d5c0f572a/source_subfolder/src/consumer.cpp:75

cppkafka/consumer.cpp:75 is the destructor
Consumer::~Consumer() {

cppkafka/kafka_handle_base.h:66 is also the destructor
virtual ~KafkaHandleBase() = default;

librdkafka/rdkafka.c:1109 in rd_kafka_destroy_app() is a thread join which calls pthread_join as below
if (thrd_join(thrd, &res) != thrd_success)

librdkafka/tinycthread.c:692 in thrd_join() is pthread join
if (pthread_join(thr, &pres) != 0)

The consumer polls with a short timeout which will not hang indefinitely. I can't find anything that could block the consumer to close. Does anybody have any idea about the possible cause and fix? Thanks a lot!

[ydsun90]

My issue seems similar to the issue being fixed by the PR #4724. Waiting to try it.