Producer crash at attempt to produce within re-created Topic object

[kfdx87]

Description

C++ binding of librdkafka library tries to access deleted memory and crashes after a produce attempt to the re-created Topic object.

How to reproduce

1. Use C++ binding of librdkafka library.
2. Create Producer object and some Topic object.
3. Delete Topic object.
4. Create a new Topic object, use the same topic name!
5. Try to produce any message within re-created Topic object --> access memory violation is detected.

It seems that topic context, created in plain C domain, is not freed when the appropriate Topic object is deleted in C++ domain, meanwhile topic context in C domain keeps an opaque pointer to the Topic config object. So it keeps a pointer to the deleted memory.

Checklist

Please provide the following information:

• librdkafka version (release number or git tag): v0.9.5
• Apache Kafka version: 0.10.1
• librdkafka client configuration:
• Operating system: Windows 8.1
• Using the legacy Consumer: no
• Using the high-level KafkaConsumer: no
• Provide logs (with debug=.. as necessary) from librdkafka: no, it's reproduced in 100% cases. Will add a backtrace info soon.
• Provide broker log excerpts: no
• Critical issue: major

[edenhill]

C Topic objects are not destroyed when the last C++ or application reference is lost, so when you re-create the C++ Topic object librdkafka looks up the previous existing C Topic object for the same topic name and reuses it.
Unfortunately that previous C Topic object has its opaque pointer pointing to the previously deleted C++ Topic object.

The workaround at this time is not to re-create C++ Topic objects, but keep them in the application code until the Producer/Consumer instance is going to be destroyed.

[kfdx87]

@edenhill, my investigation has shown the same results, thank you.

[louisth]

Running into this bug (in 1.5.2) while trying to make our long-running producer reconfigurable.