kafka: Add topic_prefix and escape topic names
#1570
Conversation
These are especially useful when paired with `topic_variable`. Since the variable is being extracted from a field, we can't guarantee that the characters being used for the topic name are valid, so we implement a slightly modified version of stdlib's `QueryEncode` which sanitizes the name to the range of valid topic chars. Example: ```toml [KafkaOutput] topic_prefix = "heka-" topic_variable = "Fields[ContainerName]" ```
| } | ||
|
|
||
| switch c { | ||
| case '.', '_', '-': |
There was a problem hiding this comment.
So there is one problem here, is the - character needs to be escaped, since that's being used as the escape character above on L394.
This will prevent backwards compatibility though if we escape -. So unsure how to proceed best without changing the topic names on people.
There was a problem hiding this comment.
Not sure I understand. Can't we use a different character for escaping? Back-slashes aren't allowed in a topic name, but they could still be used as an escape character, right? I might be missing something, though...
|
Sorry to take so long to get to this... things have been a bit crazy for me lately. Thanks for your contribution! Anyway, I didn't love the two mutually exclusive topic options in the first place, and this feels like it goes a bit farther in the wrong direction. I'd rather see all of the settings reduced down to a single Also, any changes to plugin configuration settings need to be reflected in both the documentation and the CHANGES.txt changelog. Thanks! |
These are especially useful when paired with
topic_variable.Since the variable is being extracted from a field, we can't guarantee
that the characters being used for the topic name are valid, so
we implement a slightly modified version of stdlib's
QueryEncodewhich sanitizes the name to the range of valid topic chars.
Example: